Tour of Parser-Printers: API Clients for Free

Episode #189 • May 9, 2022 • Free Episode

We conclude our tour of swift-parsing with a look at how URL routers defined as parser-printers can be automatically transformed into fully-fledged API clients, which we will drop into an iOS application and immediately use.

This episode is free for everyone.

Subscribe to Point-Free

Access all past and future episodes when you become a subscriber.

See plans and pricing

Already a subscriber? Log in

Introduction

This is all looking pretty amazing. If we are willing to do the upfront work of building a parser-printer for our router, we can easily plug it into a Vapor application to power the website. Doing so allows us to remove a lot of logic from our handlers that doesn’t need to be there, such as extracting, coercing and validating data in the path or query params. And with no additional work we instantly get the ability to link to any page in our entire website.

But if you think all of this sounds interesting, you haven’t see anything yet.

Not only do we get all these benefits in the server side code, but we also get benefits in our client side iOS code that needs to talk to the server. We can instantly derive an API client that can speak to our server without doing much work at all. And the iOS client and server side client will always be in sync. If we add a new endpoint to the server it will instantly be available to us in the client with no additional work whatsoever.

Sound too good to be true? Let’s build a small iOS application that makes requests to our server side application.

Setting up the iOS Client

We’ll start by creating a brand new Xcode project for an iOS application, which we will call Client and we will put it in a sibling directory to the server.

And then we can open this new Client Xcode project and drag the entire Server directory into it. This gives us one single Xcode project that can access all of the targets and libraries from both projects. With a little more work you can even consolidate all of this into a single SPM package for which your server and client targets consist of just a single file that act as the entry point into those respective applications. We won’t do that now, but we have covered these ideas in our modularization episodes, so we highly recommend you check those out if you are interested in that.

Now what we’d like to do is create an API client that could be used in our iOS application for making requests to our server, downloading data, and decoding it into models so that we can make use of that data in the application. There are a few popular ways of doing this in the iOS community, but at the end of the day they are all variations on a central theme, which is providing class, structs, methods and function helpers for constructing URL requests, firing off those URL requests (typically with URLSession), and then decoding the data it gets back.

The first half of this process is usually the lion’s share of the work the API client needs to accomplish. We somehow need to model the data for describing an API endpoint, such as IDs, filter options, post data and more, and then we need some way to turn that data into a URL request.

This is precisely what our parser-printer router handles for us. We get to simultaneously describe the data needed for an API endpoint as well as how that endpoint is parsed from and printed to a URL request. So we don’t have to worry about this responsibility of the API client because our parser-printers have taken care of it.

The other API client responsibility is that of actually making the network request and decoding the data. That part is quite simple thanks to URLSession. As long as we can generate a URL request, it only takes a few steps to hand that to URLSession, let it do its thing to return some data to us, and then we decode it into a model.

So it sounds like we are more than halfway to having an API client by virtue of the fact that we have described our server routing as a parser-printer. But in reality we are actually 100% there, because our URL routing library actually vends a type that automates everything we just discussed for us.

In order to make use of it we need to access to the site router in the iOS application, which means we need to extract it to its own library so that it can be simultaneously used from the server and client. Let’s do that:

.package(url: "https://github.com/pointfreeco/swift-parsing", from: "0.9.1 ")

…

.target(
  name: "SiteRouter",
  dependencies: [
    .product(name: "_URLRouting", package: "swift-parsing"),
  ]
),

The main server “App” target will depend on it:

.target(
  name: "App",
  dependencies: [
    "SiteRouter",
    …
  ]
)

And we just need to add a few imports to the server code and everything should build and run just like it did before, we just need to publicize a few things in the SiteRouter module.

Setting up the API Client

Now we are set up to use the site router in our iOS application. First we need to export SiteRouter from the package as a library.

products: [
  .library(name: "SiteRouter", targets: ["SiteRouter"])
]

Which allows us to make the Client target depend on the SiteRouter.

And just with that we are able to import the SiteRouter into our iOS application:

import SiteRouter

Which means we can easily construct requests to API endpoints on our server:

router.request(for: .users(.user(1, .books())))

So right off the bat we have accomplished the first major responsibility we outlined for API clients, that of giving us the tools to construct requests to the server.

As we alluded to before, it wouldn’t be much work to wrap our site router into a new type that can further make a request to the server using URLSession, but luckily for us there’s no need to. The _URLParsing library ships with a tool that can automatically derive an API client from our site router, and it even comes with some conveniences that make using it and testing with a really nice experience.

All we have to do is import our _URLRouting library, which remember currently is underscored because its an experimental library and its public API is still being refined, but someday soon we will be making it public:

import _URLRouting

Once we do that we get access to a type called URLRoutingClient, which can be constructed as a live dependency if we hand it a parser-printer of the right shape:

let apiClient = URLRoutingClient.live(router: router)

That’s all it takes and we now have something that can make API requests, decode the response into response models, and hand the results back to us. Although technically we have to make one small change, which is to specify the base URL of the router, just like we did in the server:

let apiClient = URLRoutingClient.live(
  router: router
    .baseURL("http://127.0.0.1:8080")
)

With a little more work we could make it so that this base URL is set based on the build of the application, such as DEBUG versus RELEASE, or could even make it changeable from within the application itself. But we aren’t going to worry about those kinds of things right now.

So we now have an API client, let’s start using it! We’re going to build a simple SwiftUI view that loads data from the API and displays it in a list. We could make the stubbed ContentView that Xcode provides us hold onto an array of books:

struct ContentView: View {
  @State var books: [BooksResponse.Book] = []

  …
}

And then in the view’s body we can create a list for each of the books in the array:

var body: some View {
  List {
    ForEach(self.books, id: \.id) { book in
      Text(book.title)
    }
  }
}

But in order for this to work we need access to BooksResponse, which currently is in the server code. In order for us to have a chance at encoding this response on the server to send out and then decoding the response on the client we need to share this model between both platforms. For that reason we will move our response types to the SiteRouter module and make them public, but we will relax their conformance to Vapor’s Content protocol to Codable instead. This way we don’t have to have our site router depend on all of Vapor, which would force our iOS client to depend on all of Vapor as well and something we do not want to do. Instead, our server code will be responsible for importing these response types and retroactively extending them to Content.

extension UserResponse: Content {}
extension BookResponse: Content {}
extension BooksResponse: Content {}
extension BooksResponse.Book: Content {}

In order to populate the array of books we need to make an API request, which our API client makes very easy. The client comes with a method called request that allows you to specify the route you want to hit as well as the response model you want to decode into:

.task {
  do {
    self.books = try await apiClient.request(
      .users(.user(1, .books())),
      as: BooksResponse.self
    ).value.books
  } catch {

  }
}

Now this compiles. However, if we run this nothing appears in the SwiftUI preview.

This is happening because our server isn’t running. We need to have the server running so that the playground can actually hit “127.0.0.1:8080” and load data from our server code. So let’s quickly switch to the “Server” target, hit cmd+R to run it, and then switch back to the “Client” target and re-run our preview.

Now it magically loads data! We are actually hitting our server from the SwiftUI playground, and we can see this not only because data is actually showing in the list, but we can also see logs appearing in the console of Xcode that Vapor spits out anytime a request is made.

This is pretty amazing. With zero work on our part we have magically created an API client that is capable of constructing a URL request that is proven to be understandable by our server, and then it makes the request to the server, and then decodes the response into a model we can easily use in our client side code. All the messiness of constructing URL paths, appending query parameters, setting post bodies and more is hidden away from us in the router.

This is a pretty huge win. In fact, remember that there was one site route endpoint that was responsible for creating a user. Under the hood it needs to make sure the HTTP method is a POST, and that it encodes some data into JSON to be added to the POST body of the request. The server cares about all of that, but the client doesn’t. The client just wants to tell the server “hey, create a user for me with this data”, and that’s exactly what the API client can accomplish:

apiClient.request(
  .users(.create(.init(bio: "Blobbed around the world", name: "Blob"))),
  as: <#Decodable.Protocol#>
)

We don’t need to know any of the particulars of how we are communicating with the server: whether it’s a GET or a POST, how to encode the JSON. All of that is handled for us behind the scenes.

So, this is pretty amazing. We have been able to take the router that we built for our server application, which is capable of both parsing incoming requests to figure out what application logic we want to execute as well as generating outgoing URLs for embedding in responses, and from that object we derived an API client that can be used in our iOS application for communicating to the server. And we did so with no additional work.

It’s worth mentioning that this technique can be used even if you are not building a server-side application in Swift. If you just need to make requests to your company’s server API, rather than building an API client in the more traditional style, you can instead build a parser-printer for the API’s specification, and from that get an API client out of it for free. This will give you a bunch of tools for constructing complex requests and make everything nice and type safe.

A complex API-driven feature

This is definitely looking incredible, but let’s make our little client app a little more complicated so that we can see how powerful these parser-printer routers are. What if we wanted to add a segmented control at the top of the UI that chooses between sorting directions, either ascending or descending.

We can add some state to our view:

struct ContentView: View {
  @State var direction: SearchOptions.Direction = .asc

  …
}

And we can bundle our list into a VStack so that we can put a picker view above it:

VStack {
  Picker(selection: self.$direction) {
    Text("Ascending").tag(SearchOptions.Direction.asc)
    Text("Descending").tag(SearchOptions.Direction.desc)
  } label: {
    Text("Direction")
  }
  .pickerStyle(.segmented)

  List {
    ForEach(self.books, id: \.id) { book in
      Text(book.title)
    }
  }
}

And then we can tack on a .onChange modifier to the end of our view so that whenever the direction changes we re-request the books from the server:

.onChange(of: self.direction) { _ in
  Task {
    do {
      let response = try await apiClient.request(
        .users(.user(1, .books(.search(.init(direction: self.direction))))),
        as: BooksResponse.self
      )
      self.books = response.books
    } catch {
    }
  }
}

Now the .task and .onChange modifier contain very similar code, and so you’d probably want to extract out this common code to a view model, which is exactly what we will do in a moment.

But, with these few changes we can now change the sort direction directly in the SwiftUI preview. If we run the preview we will see that changing the selection of the segmented control causes the results to sort differently, and we can even see from the logs that indeed the server is being hit each time we change the selection.

Testable/previewable API clients

So, this all seems pretty amazing, but it gets even better. By constructing our API client in this way we immediately get the ability to override certain endpoints to return mock data synchronously. This is great for testing since we don’t want to make requests to our live server when testing. Our API client type comes with some really fantastic tools for accomplishing this, and it dovetails with many ideas we have discussed on Point-Free in the past.

To explore this we are going to refactor our view really quickly to use a proper view model, because that’s the only way we have a chance at testing this feature. We can basically copy-and-paste a bunch of stuff from the view in order to make a basic view model, and we’ll go ahead and preemptively require the view model to take an explicit URLRoutingClient as a dependency so that we can control it from the outside:

class ViewModel: ObservableObject {
  @Published var books: [BooksResponse.Book] = []
  @Published var direction: SearchOptions.Direction = .asc
  let apiClient: URLRoutingClient<SiteRoute>

  init(apiClient: URLRoutingClient<SiteRoute>) {
    self.apiClient = apiClient
  }

  @MainActor
  func fetch() async {
    do {
      let response = try await self.apiClient.request(
        .users(.user(1, .books(.search(.init(direction: self.direction))))),
        as: BooksResponse.self
      )
      self.books = response.books
    } catch {

    }
  }
}

Then our view can use this view model instead of having a bunch of local state, which is not easy to test:

struct ContentView: View {
  @ObservedObject var viewModel: ViewModel

  …
}

And then our view’s body can reach into the view model to get access to data instead of accessing it directly on the ContentView:

  var body: some View {
    VStack {
      Picker(
        selection: self.$viewModel.direction
      ) {
        Text("Ascending").tag(SearchOptions.Direction.asc)
        Text("Descending").tag(SearchOptions.Direction.desc)
      } label: {
        Text("Direction")
      }
      .pickerStyle(.segmented)

      List {
        ForEach(self.viewModel.books, id: \.id) { book in
          Text(book.title)
        }
      }
    }
    .task {
      await self.viewModel.fetch()
    }
    .onChange(of: self.viewModel.direction) { _ in
      Task {
        await self.viewModel.fetch()
      }
    }
  }

Now our view is compiling, but anywhere we try to construct a view is failing because we need to supply a view model. This is our chance to supply dependencies so that we can control the environment we are working in. For now we’ll just use the live API client in the preview:

struct ContentView_Previews: PreviewProvider {
  static var previews: some View {
    ContentView(
      viewModel: .init(
        apiClient: .live(
          router: router.baseURL("http://127.0.0.1:8080")
        )
      )
    )
  }
}

And in the app entry point:

import SiteRouter

@main
struct ClientApp: App {
  var body: some Scene {
    WindowGroup {
      ContentView(
        viewModel: .init(
          apiClient: .live(router: router.baseURL("http://127.0.0.1:8080"))
        )
      )
    }
  }
}

In order to write a test for this feature we need to construct a view model and then invoke endpoints on the view model to assert on what happens after:

class ClientTests: XCTestCase {
  func testBasics() async throws {
    let viewModel = ViewModel(apiClient: ???)

    await viewModel.fetch()
  }
}

However, to do this we need to supply an API client. We definitely do not want to supply a live API client because that would leave us exposed to the vagaries of the outside world, such as the quality of our internet connection, the stability of the server, and the unpredictable data the server could send back.

So what we like to do is skip the live logic of the API client entirely, and just synchronously return the data we want to test with. Technically we can build a whole new API client from scratch by just supplying a function that can transform SiteRoute values into data and response:

let viewModel = ViewModel(
  apiClient: .init(request: <#(SiteRoute) async throws -> (Data, URLResponse)#>)
)

But implementing such a function takes a lot of work. We’d have to destructure the SiteRoute to match on the exact route we care about and then construct some Data and URLResponse from scratch, and then I guess return some kind of default data and response in all other cases.

So, this is why our library comes with tools for starting the API client off in a state where none of the site routes are implemented, and then we can override just the specific endpoints we think will be invoked.

Even better, if an API endpoint is invoked that we did not override we will get a test failure, which allows us to exhaustively prove what parts of our API the view model actually needs to do its job. And in the future if we start using new API endpoints we will be instantly notified of which tests need to be updated to account for the new behavior.

So, we can start our view model off with a fully failing API client, which means no matter which route you try to hit you will get a test failure:

class ClientTests: XCTestCase {
  func testBasics() async throws {
    let viewModel = ViewModel(apiClient: .failing)

    await viewModel.fetch()
  }
}

Failed to respond to route: SiteRoute.users(.user(1, .books(.search(SearchOptions(sort: .name, direction: .asc, count: 10)))))

Use ‘URLRoutingClient.override’ to supply a default response for this route.

And of course this immediately fails because the .fetch method tries to hit an API endpoint. The test failure is very helpful in showing us exactly what API endpoint was hit, and so all we have to do is override this specific endpoint so that it returns some real data.

There is a method that allows us to do this:

let viewModel = ViewModel(
  apiClient: .failing
    .override
)

There are a few options for describing how we want to override the API client, and each is interesting and as their own uses, but the one we are most interested in is the one that allows us to specify the exact endpoint we want to stub in for some synchronous mock data.

In order to get access to that override we need to make SiteRoute equatable so that override can understand which route it is we are overriding.

And now we get access to an override method that allows us to specify the exact route we want to override, as well as the response we want to synchronously and immediately return:

let viewModel = ViewModel(
  apiClient: .failing
    .override(
      <#T##route: SiteRoute##SiteRoute#>,
      with: <#T##() throws -> Result<(data: Data, response: URLResponse), URLError>#>
    )
)

The route we want to override is the exact enum value we expect to be invoked in the view model. Currently it’s just a hardcoded value, but in a real application there may be logic that determines which endpoint is invoked, and this style of testing gives us the ability to test that logic.

let viewModel = ViewModel(
  apiClient: .failing
    .override(
      .users(.user(42, .books(.search(.init(direction: .asc)))),
      with: <#T##() throws -> Result<(data: Data, response: URLResponse), URLError>#>
    )
)

Next we need to construct the data and response we want the API client to send back when this exact API endpoint is invoked. This gives us full freedom to describe exactly how the API client responds, but in terms of its data but also in terms of its status code, headers and more.

The library comes with a helper to make constructing this result a bit nicer:

let viewModel = ViewModel(
  apiClient: .failing
    .override(
      .users(.user(1, .books())),
      with: {
        try .ok(
          BooksResponse(
            books: [
              .init(
                id: .init(uuidString: "deadbeef-dead-beef-dead-beefdeadbeef")!,
                title: "Blobbed around the world",
                bookURL: "/books/deadbeef-dead-beef-dead-beefdeadbeef"
              )
            ]
          )
        )
      }
    )
)

So now we have forced the failing API client to not fail on this one, specific route, in which case a concrete BooksResponse will be returned.

In fact, now that we have this, our test suite is passing because the fetch method will invoke this specific endpoint under the hood, and all is fine. That means we can now finally make assertions on how the view model’s state changes after the fetch endpoint is invoked:

XCTAssertEqual(
  viewModel.books,
  [
    .init(
      id: .init(uuidString: "deadbeef-dead-beef-dead-beefdeadbeef")!,
      title: "Blobbed around the world",
      bookURL: "/books/deadbeef-dead-beef-dead-beefdeadbeef"
    )
  ]
)

But to do that we need to make Book equatable:

public struct Book: Codable, Equatable {
}

And now this test passes!

So this is pretty incredible. Not only could we generate an API client from our router for free, with no extra work on our part, but it’s even infinitely testable right out of the box. We can tap into the API client to override just a specific endpoint, and force it to return the data that we want. And then we get to test how that data flows into the rest of the system, which we can do by making assertions on the view model’s state.

But there are more applications of this override concept than just for tests. Currently in our preview we are using a live API client, which is cool for when we want to demo how the screen integrates with a live, running web server, but often we do not want to do that. Many times running a full server is a bit too heavy handed for something like a preview, and so we just want to supply some simple data to the preview to show, and we can use a different overload of override to accomplish this.

We can start our preview off in a failing state:

struct ContentView_Previews: PreviewProvider {
  static var previews: some View {
    ContentView(
      viewModel: .init(
        apiClient: .failing
      )
    )
  }
}

Which will cause the preview to show no data, because the API client we supplied just immediately fails for each endpoint.

So, we can chain onto this failing client the .override method, but this time we can supply a predicate that determines which routes we want to override rather than supplying just one single route that we want to override:

viewModel: .init(
  apiClient: .failing
    .override(
      <#Tpredicate: (SiteRoute) -> Bool##(SiteRoute) -> Bool#>,
      with: <#() throws -> Result<(data: Data, response: URLResponse), URLError>#>
    )
)

So, we could use pattern matching to say that we only want to support the search endpoint:

viewModel: .init(
  apiClient: .failing
    .override {
      guard case .users(.user(_, .books(.search))) = $0
      else { return false }
      return true
    } with: {

    }
)

And then when that predicate is true we will override that endpoint with a mock BooksResponse:

viewModel: .init(
  apiClient: .failing
    .override {
      guard case .users(.user(_, .books(.search))) = $0
      else { return false }
      return true
    } with: {
      try .ok(
        BooksResponse(
          books: (1...100).map { n in
            .init(
              id: .init(),
              title: "Book \(n)",
              bookURL: URL(string: "/books/\(n)")!
            )
          }
        )
      )
    }
)

And now our preview is running again, but it’s no longer hitting a server. We are just providing the data immediately to the preview.

So, not only have we installed a router into a Vapor application that is statically checked and type safe, and in doing so not only have we made it possible to immediately link to other parts of the application in a static and type safe way, but once all of that was done we get an API client out of it for free. This allowed us to make API requests from an iOS application to our server without doing any additional work.

And if that wasn’t amazing enough, we were able to accomplish all of this in an infinitely testable and flexible manner. We can easily construct all new API clients that stub out just a small portion of the routes in order to use the client in tests and previews.

Conclusion

Well, that concludes this tour of our swift-parsing library. We think it’s pretty incredible that in just 4 episodes we have touched upon small parsing problems such as what one encounters in Advent of Code challenges, as well as comparing regular expressions to incremental parsing, and then somehow ended on type safe routing and derived API clients.

We think this just goes to show how incredibly powerful a generic, composable parsing library can be. Believe it or not there is still more parsing topics we want to cover in the future, but we want to cover some different topics, so we’ll leave it here for now.

Until next time!

This episode is free for everyone.

Subscribe to Point-Free

Access all past and future episodes when you become a subscriber.

See plans and pricing

Already a subscriber? Log in

References

Vapor Routing

Brandon Williams & Stephen Celis • Monday May 2, 2022

A bidirectional Vapor router with more type safety and less fuss.

Swift Parsing

Brandon Williams & Stephen Celis • Tuesday Dec 21, 2021

A library for turning nebulous data into well-structured data, with a focus on composition, performance, generality, and invertibility.

Invertible syntax descriptions: Unifying parsing and pretty printing

Tillmann Rendel and Klaus Ostermann • Thursday Sep 30, 2010

Parsers and pretty-printers for a language are often quite similar, yet both are typically implemented separately, leading to redundancy and potential inconsistency. We propose a new interface of syntactic descriptions, with which both parser and pretty-printer can be described as a single program using this interface. Whether a syntactic description is used as a parser or as a pretty-printer is determined by the implementation of the interface. Syntactic descriptions enable programmers to describe the connection between concrete and abstract syntax once and for all, and use these descriptions for parsing or pretty-printing as needed. We also discuss the generalization of our programming technique towards an algebra of partial isomorphisms.

This publication (from 2010!) was the initial inspiration for our parser-printer explorations, and a much less polished version of the code was employed on the Point-Free web site on day one of our launch!

Vapor

A popular Swift web framework. It comes with a router that is clearly inspired by frameworks like Express, but as a result is less type safe than it could be.

Downloads