Consumer API

[shaps80]

Consumer API

• Proposal: [SGQL-194]
• Authors: Shaps
• Status: In-progress
• Release: v6

Overview

The current APIs have a few issues IMO that I'd like to try and resolve.

• The API has a fairly steep initial learning curve since the look-and-feel isn't exactly Swift API
• It has some great features like type-safety however it has limited auto-complete (type-inference) until you've filled in some of the details first
• Its cumbersome (imo) to write reusable queries/fragments
• It relies heavily on GraphQL (non-swift) terms

It was some amazing upsides, but I feel the developer experience leaves a lot to be desired and the current approach is problematic to really solve that well.

---

Proposal

Given that, I started working on several different approaches over several months. Eventually I landed on a resultBuilder implementation that I feel solves all of the above problems, and only introduces very minor (almost irrelevant) trade-offs.

Lets cover the issues and I'll demonstrate how I feel the new approach could improve on those issues.

Learning curve and auto-complete

I'm a firm believer in leaning into the plethora of API Apple has provided since developers are already likely familiar and therefore have certain expectations that we can easily fulfil. SwiftUI is a great example, so taking a queue from that writing a Query is now much simpler.

struct PersonQuery: Query {
    // generated model
    typealias Model = API.Model.Person

    var body: some Query {
        Attribute(\.twitter)
        Attribute(\.email)

        // custom alias
        Attribute("phone", \.number)
    }
}

As you can see it looks extremely familiar, however it's even simpler than SwiftUI since there is only ONE type to learn, Attribute.

The Model (a protocol requirement) works in tandem with the QueryBuilder to provide full type-safety and autocomplete support against the generated models.

And since only models are scoped to API.Model its easy to find the appropriate type. From there auto-complete helps you discover everything else without issue.

Note: I won't cover more advanced usage here, but some Attribute initialisers will also be included via the code generation, providing schema based features like filtering, sorting, etc provided by your API

Reusable queries

Another great feature of resultBuilder APIs, is that its easy to write and compose Query's. Lets say we had a query with a nested element:

struct PersonQuery: Query {
    typealias Model = API.Model.Person
    var body: some Query {
        Attribute(\.email)
        Attribute(\.name) {
            Attribute(\.givenName)
            Attribute(\.familyName)
        }
    }
}

We can easily decompose the name by simply copying its content and pasting it into a new Query:

struct NameQuery: Query {
    typealias Model = API.Model.Name
    var body: some Query {
        Attribute(\.givenName)
        Attribute(\.familyName)
    }
}

And now our updated PersonQuery becomes:

struct PersonQuery: Query {
    typealias Model = API.Model.Person
    var body: some Query {
        Attribute(\.email)
        Attribute(\.name) {
            NameQuery()
        }
    }
}

Note we don't need to worry about whether name refers to an array or dict, and we don't need to know whether an attribute is optional or not. The QueryBuilder automatically selects the right Attribute.init for us. Another huge benefit over the existing API.

As you can see, it's an extremely flexible API and makes it easy to write, compose and decompose queries. It's uncomplicated, extremely familiar and easy to work with.

Swift language

Lastly I want to touch on the language. Here you only need to remember TWO terms, Query and Attribute – both of which are familiar terms that have well understood semantic meaning.

Unlike SwiftUI you don't need to work with many types of View's or other types, everything is either built-in or generated, but is provided via extremely discoverable auto-complete APIs.

In fact, API availability and behaviour is also scoped to the initialisers similarly to most SwiftUI APIs. For example nesting a query under name that isn't scoped to API.Model.Name will fail to compile with an error indicating as such.

struct PersonQuery: Query {
    typelias Model = API.Model.Person
    var body: some Query {
        Attribute(\.email)
        Attribute(\.name) {
            AddressQuery() // ERROR: Unexpected query for type `API.Model.Name`
        }
    }
}

The same applies for nesting, trying to include nesting on a non-nested type (dict or array) simply fails to compile. So the compiler support is strong at all times.

---

Added benefits

I'd like to also discuss some of the added benefits this approach brings over the existing implementation (and others I explore).

Actor support

When you execute a query, the Client automatically scopes the query to the GraphQLActor. This ensures thread-safety end-to-end for any given query. This includes query execution right up until model decoding and the developer doesn't have to do anything to get this behaviour.

"It just works"

Aliasing

Wherever possible the API provides strongly typed compile-time warnings or errors to ensure you can't compile a query that isn't correct. The one exception to this is related to attributes being unique at any given level.

This is a GraphQL requirement since queries are represented as JSON during transport. The existing v5 implementation uses an automatic hashing approach to ensure you don't need to think about this. However in practice this can be problematic and there's no easy way to opt-out and control that behaviour if you need to.

There are several Issues related to this already.

The new approach uses the following approach:

• Attribute names are respected by default
• An optional alias can be provided giving you full control
• When the Request executes the body of the query, the library collects the attributes and throws an error if it encounters a duplicate
  • In addition, the console output identifies which attribute was duplicated and provides an appropriate solution to help the developer out.
  • The error reports the name, a solution and also the file/line where the duplicate exists – this ensures reusable queries are still easy to debug

In addition, you can set a naming policy on the Request that changes this behaviour. The alternative strategy uses hashes similar to the existing implementation in an attempt to 'solve' this for the developer, the tradeoff being you give up some control.

Note: while attribute names will be hashed using that policy, you can still choose to set an alias (something you can't do in v5) – however if duplicate aliases are also found, these will also be hashed to remove ambiguity.

Conditional queries

Another added benefit of using resultBuilder APIs, is that we can write conditional queries:

struct UserQuery: Query {
    var isAuthenticated: Bool

    var body: some Query {
        Attribute(\.username)
        
        if isAuthenticated {
            Attribute(\.email)
            PermissionsQuery()
        }
    }
}

This is of course possible in v5 as well, but it's slightly cleaner here since we can write nested queries more easily as well.

Note also how we're able to store a variable for controlling our condition. This feels extremely natural and is a powerful way to build reusable Query's that are then used in other contexts.

---

Generated models

API.Model.xxx types are generated models from your GraphQL Schema. Therefore a lot of the existing implementations will likely remain with some subtle improvements.

These will also continue to be automatically decoded by the library through the definition of Scalar types. These are similar to the existing protocol however with a slightly more up-to-date and familiar API. It leans more heavily into the CoreTransferable for design inspiration.

@GraphQLActor
public protocol Scalar {
    associatedtype Representation: ScalarRepresentation
    static var scalarRepresentation: Representation { get }
}

@GraphQLActor
public protocol ScalarRepresentation<Item>: Sendable {
    associatedtype Item: Scalar
    func scalar(from data: AnyCodable) throws -> Item
}

The library includes default implementations for many commonly used types and if your custom type can be simple cast you can opt-in more easily:

extension Decimal: Scalar {
    public static var scalarRepresentation: DefaultScalarRepresentation<Self> { .init() }
}

No implementation is required, since a default is already provided:

public static var scalarRepresentation: some ScalarRepresentation {
    DefaultScalarRepresentation { try $0.value(as: Self.self) }
}

However you can instead implement a custom representation for a type if you need greater control:

extension UUID: Scalar {
    public static var scalarRepresentation: DefaultScalarRepresentation<Self> {
        .init(decoding: Self.self) { data in
            // value(as:) is a built-in convenience function
            let value = try data.value(as: String.self)
            guard let uuid = Foundation.UUID(uuidString: value) else {
                throw DecodingError.unexpectedScalarValue(
                    expected: Self.self,
                    value: value
                )
            }
            return uuid
        }
    }
}

Decoding

One of the strongest implementations (imo) of the existing API that Mat did a great job on, is how the decoding happens alongside the query generation. While this is not my personal preference, it does solve a unique problem, where my solution decides to make the trade off.

In the current implementation, you are essentially 'decoding' at the same time as you're writing the 'selection' – this is really cool because it means you can't forget to 'select' something you want to 'decode'.

However, it's slightly unnatural to me (and I'd assume most other Swift developers) given other API like Decodable. In addition, it makes it difficult to separate concerns (engineering principle) between query definition and model decoding.

In addition there are caveats you need to take of as a developer and without reading the docs, can lead to confusion. In fact, Mat has a special note on the website specifically for this.

It's important that you always make selection before any logical operation on it. Otherwise, it could happen that some fields are missing from the selection.

• Docs

Given that, I decided to take an alternative approach that borrows from some of Mat's ideas but pushes in a different direction.

struct Name: GraphQLDecodable {
    typealias Model = API.Model.Name
    var name: PersonNameComponents

    // this 'looks' like a `Decodable` method but the Decoder is actually
    // code-generated with included type-safety with existing `KeyPath`
    // knowledge, thus no CodingKey requirements, etc...
    init(from decoder: Decoder) throws {
        let given = try decoder.decode(\.givenName)
        let family = try decoder.decode(\.familyName)
        name = .init("\(given) \(family)")
    }
}

• We can now separate our decoding logic nicely, even across models and queries.
• Since the Model typealias requirement is identical to a Query you could choose (if preferred) to share this model with both your query and the decoding. So it's still possible to keep it on a single type, file etc if you prefer.
• As you can see its still easy to perform value transformations where required
• The Decoder pattern is familiar and makes it instantly clear in terms of expected behaviour.

Decode will throw an error if the value doesn't exist since we're inferring here that it should.

To assist the developer, an error is reported, reminding the developer to include this attribute in their Query and we could potentially even indicate which query was used since we constrain the Query and Decodable to the Request where this knowledge can be discerned.

Giving the developer more control means you can do things like decide to never throw, rather just revert to a default value:

let given = (try? decoder.decode(\.givenName)) ?? ""

We could additionally include a similar API to decodeIfPresent that would automatically fall back to the Scalar's default value, essentially never returning nil.

This approach is preferable IMO because we ensure the user has maximum flexibility while keeping things intuitive and simple. However we include convenience where appropriate, but not at the expense of customisation where that's the common-case.

Final thoughts

There are still a lot of details to work through, while I do have a compiling/working POC for the above, there are many pieces left to sort through. For one thing many nested attributes accept 'arguments' – therefore an API must be generated for those.

This will be possible since the code-gen will simply extend Attribute to include those init methods. But this needs to be fleshed out of course.

I look forward to sharing more and I am open to any and all feedback at this stage, no solution is perfect but I feel like this has the right trade-offs while still improving the current experience and taking it even further.

[CrownedPhoenix]

One issue I'm noticing given:

struct CustomPerson: Query, GraphQLDecodable {
    typealias Model = API.Model.Person
    var name: PersonNameComponents
    var email: String

   var body: some Query {
        Attribute(\.email)
        Attribute(\.name) {
            Attribute(\.givenName)
            Attribute(\.familyName)
        }
    }

    init(from decoder: Decoder) throws {
        let email = try decoder.decode(\.email)
        let given = try decoder.decode(\.name.givenName)
        let family = try decoder.decode(\.name.familyName)
        self.email = email
        self.name = .init("\(given) \(family)")
    }
}

How does the GraphQLDecodable decoder use the KeyPath to resolve the value?
After receiving the JSON response data from the server, that has to be decoded into some intermediary structure - which has to be something like a [String: Any] in order to represent arbitrary JSON.

It looks like it's the code generation that is responsible for creating the mapping between \.email and "email" such that the "email" string can be used to look up the member of the JSON data? That part definitely seems doable.

However, the two-hop KeyPaths (above) don't seem like they could be resolved effectively without having to code generate a case match for every single possible path through the GQL schema models.

Specifically, I'm imagining a code generated function like this:

func lookup<Value>(_ kp: KeyPath<API.Model.Person, Value>) -> String? {
    switch kp {
        case \.email: "email"
        case \.name: "name"
    }
}

Such that the decoder would have some code like this:

func decode<Value>(_ kp: KeyPath<API.Model.Person, Value>) throws -> Value {
    guard
        let topLevelMemberName = lookup(kp),
        let value = self.jsonDataStructure[topLevelMemberName] as? Value
    else { /* throw */ }
    return value
}

That seems fine for top level members. But (unless I'm missing something) it seems to break down with nested queries.
You'd need something like:

func lookup<Value>(_ kp: KeyPath<API.Model.Person, Value>) -> [String] {
    switch kp {
        case \.email: ["email"]
        case \.name: ["name"]
        // Having to do this ends up creating explosive combinatoric
        // over every possible KeyPath through the GQL schema
        case \.name.givenName: ["name", "givenName"]
    }
}


func decode<Value>(_ kp: KeyPath<API.Model.Person, Value>) throws -> Value {
    guard
        let memberPath = lookup(kp),
        let value = self.jsonDataStructure.recursiveLookup(memberPath) as? Value
    else { /* throw */ }
    return value
}

Possible Solutions

• If KeyPaths could be "popped" such that \.name.givenName could be decomposed into \Person.name and \Name.givenName then this could be done recursively without code-gen combinatoric explosion.
• The decomposition of the KeyPaths could be indicated by the API somehow. The decoder.decode(\.name) API would have to prevent users from supplying multi-hop KeyPaths etc

Is there something I'm missing?

[shaps80]

I unfortunately do not have the time atm to explain this in detail, but what you've described above has over-complicated a lot of the intent. This is actually implemented and working directly via JSONDecoder in fact, the specified keyPath is PURELY to generate the correct API from pre-known names (code-gen).

For example:

Attribute(\.email)

We store the keyPath as AnyKeyPath with no reference to what it actually holds. However when you print this you get Person.email – we simply trim off the namespace and voila we have email.

Don't know if that makes sense or not, it's an oversimplification, however it works including for nested queries etc as well. This is in fact one piece of work that's been completed already.

One issue that does exist that I hadn't previously anticipated is that we cannot correctly allow alias as per my above design. This is because the code-gen will always generate the name based on the schema.

If we allowed the Attribute to supply an alias – effectively changing the expected name – JSON decoding would fail since the associated key would not be found.

I'm currently looking into this to determine the appropriate step forward.

[shaps80]

Small addition, you're right we use an intermediary decoded value btw, but not [String: Any] since this would require a ton of extra effort.

Rather I have constructed a special GraphQLValue<Value> that's generic and as such allows us to 'know' at compile-time the type of the value as well as whether its Optional or not.

Every property is wrapped in this type as an optional requirement, allowing some queries to exclude the property safely.

We then have a special decoding process that will throw if you try and decode a value that wasn't included in the original query, we even give you nice console output to help you resolve those situations, rare as they may be.

[shaps80]

Final note:

let given = try decoder.decode(\.name.givenName)
let family = try decoder.decode(\.name.familyName)

These two lines you've reference are not from my code and would also not be supported. Keypaths would be 'single level' only, since the first thing you get back is a GraphQLValue and you have to use the decoding method to gain access to its internal value.

This is by design and ensures nested access isn't allowed since it makes little sense in this context.

[CrownedPhoenix]

We store the keyPath as AnyKeyPath with no reference to what it actually holds. However when you print this you get Person.email – we simply trim off the namespace and voila we have email.

Gotcha. This was my first intuition about what you were probably doing. But I second-guessed it because I've run into issues using this feature before (to do the exact same thing) but had to find an alternative because it is a best-effort recovery that isn't consistent between platforms (see: this forum post). But, as you've said, linux isn't on your list of platforms to specifically support so you probably won't have any issues there. It may not even be an issue on linux since I'm guessing none of the generated fields would be computed properties.

What you describe about GraphQLValue and the KeyPath nesting makes sense.
Thank you.

What would (roughly) be the appropriate way to write the query I mentioned with the new v6 API?

{
    people {
        email
        name {
            givenName
            familyName
        }
    }
}

Such that it decodes into:

struct CustomPerson: Query, GraphQLDecodable {
    typealias Model = API.Model.Person
    var fullName: PersonNameComponents
    var email: String
}

[tonyarnold]

This looks great - did this work eventuate? The whole repo looks very quiet since before WWDC.

[NicholasTD07]

I am curious. Any update on the v6 work? It's almost a year since the talk in #185.