Replace Result<> with Raw Arrays for Better React Integration

[SMhdAsadi]

In the React ecosystem, queries are designed to re-render components whenever data updates, ensuring reactivity and seamless integration with state changes. While Result is valuable for live data and reactivity with listeners, it introduces complexity when integrating with React components. Returning a Result<> type requires an additional step to convert it into a raw array before it can be used in our applications. This extra step complicates the workflow and diminishes the ease of use that React developers expect.

[kraenhansen]

Thank you for creating this discussion 💙

While Result is valuable for live data and reactivity with listeners

Another benefit of the result is that it lazily loads data. I.e. it doesn't eagerly load all objects matching your query into memory as an array would do.

it introduces complexity when integrating with React components.

I see - this could be a significant drawback 🤔

For what it's worth, our Collection implements Iterable:

realm-js/packages/realm/src/Collection.ts

Line 65 in 7540fe7

> implements Iterable<T>

And the OrderedCollection (which Results inherit from) implements ReadonlyArray:

realm-js/packages/realm/src/OrderedCollection.ts

Line 166 in 7540fe7

implements Omit<ReadonlyArray<T>, "entries">

I have a few clarifying questions:

• Are you using the "@realm/react" package?
• Can you supply concrete examples of how this degrades the developer experience?
• Is it the liveness of the Results that cause an issue for you (i.e. that it doesn't behave immutable) or that you're experiencing that the types doesn't match Array exactly?
• Is your suggestion that our useQuery hook should actually return an array or that it should simply "lie" and declares itself as an array although it actually returns a Results?

[SMhdAsadi]

Thanks for your quick response.

• Yes I'm using Realm/React package.
• I'll try to give an example at the end of this message.
• The types are mostly the issue.
• I meant to return the actual array instead of Result.

I'm integrating Realm into an existing app. Since there was a concrete data flow in components, it is almost impossible to use full potential of live Results. I really appreciate it if there was a hook that actually returned the concrete, non-live and non-lazy values as an array so I can instantly work with.

Here is an example. I have a schema for my users called UserSummaryDbModel. One of its properties store a list of userId that reference other users. I want to access all of them. Currently I've used the useQuery to find the initial user. Here I need to use that property which stores the list of userId immediately, so live and lazy Results doesn't help me in this case. I use another useQuery to fetch other users.

I usually need the data immediately in my components and custom hooks. I wonder if it would be possible to have this feature (data being lazy and live) as optional or have it in another hook like useLazyQuery? Or introducing another hook that returns the actual data?

[kraenhansen]

I really appreciate it if there was a hook that actually returned the concrete, non-live and non-lazy values as an array so I can instantly work with.

Agree - we're aware that the liveness of our collections and object are not ideal in a React (or any other declarative UI framework) which expects immutable data. We've also seen that our hook-based API trigger re-renders reliably and as long as users pass primitive values through props. If they don't, it primarily cause over-renders (which is annoying but at least doesn't cause stale UIs). We currently expose APIs to capture a non-live snapshot of Results via the snapshot() method. That being said, we don't have this of Realm.Object and this DX is something we want to improve.

The lazy part of the Realm.Results and Realm.Object should be completely opaque to you as a developer. Items in collections and properties on objects are retrievable using synchronous APIs and if it's in the database, it'll return. The lazy part of the architecture pertain more to how the JavaScript layer of the SDK access the database on the filesystem, i.e. it doesn't actually load data into JS before it's accessed, making it possible to query the data and get a collection of millions of objects without paying the cost of "hydration" that an ORM would normally pay. This is also why we don't see a strong need for pagination (limit and offset) that a typical SQL / ORM based database would need.

As such, while it's likely we'll improve the DX around accessing collections and objects as immutable (or frozen as other of our SDKs refer to these) data, we'll likely not expose additional APIs (beyond array and object spreading [...results] and {...object}) to eagerly load data into memory, by-passing the lazy architecture of our SDK that we consider a feature and shouldn't be visible to developers anyway. I'd love to learn more on how you see the laziness affect your codebase negatively.

Here I need to use that property which stores the list of userId immediately

I don't fully understand this statement. Although the Realm.Results and Realm.Object are lazy, if the property with a list of userId is in the database, you should be able to read it right away. I'd appreciate a bit of code here to better understand your access pattern. One potential optimization would be to store object links instead of a list of ids, if thats possible in the wider context of your app and its architecture.

[SMhdAsadi]

I don't fully understand this statement. Although the Realm.Results and Realm.Object are lazy, if the property with a list of userId is in the database, you should be able to read it right away. I'd appreciate a bit of code here to better understand your access pattern. One potential optimization would be to store object links instead of a list of ids, if thats possible in the wider context of your app and its architecture.

Thank you for pointing that out. I was storing only the IDs, thinking they acted like foreign keys. Now, I am using relationships to other objects as you suggested, which has resolved many of my issues.

I can see that the problem isn't inherently about the Result type. My real challenge was using nested embedded objects. It's really difficult to work with TypeScript and embedded objects. Although this might be off-topic, here's an example to illustrate:

class Embedded2Schema {
  array!: boolean;

  static schema: ObjectSchema = {
    name: 'Embedded2',
    embedded: true,
    properties: {
      array: 'bool',
    },
  };
}

class Test2Schema extends Realm.Object<Test2Schema> {
  id!: string;
  name!: string;

  static schema: ObjectSchema = {
    name: 'Test2',
    primaryKey: 'id',
    properties: {
      id: 'string',
      name: 'string',
    },
  };
}

class Embedded1Schema {
  embedded2!: Realm.List<Embedded2Schema>;
  relationship!: Test2Schema;

  static schema: ObjectSchema = {
    name: 'Embedded1',
    embedded: true,
    properties: {
      embedded2: {
        type: 'list',
        objectType: Embedded2Schema.schema.name,
      },
      relationship: Test2Schema.schema.name,
    },
  };
}

class Test1Schema extends Realm.Object<Test1Schema> {
  userId!: string;
  embedded1!: Embedded1Schema;

  static schema: ObjectSchema = {
    name: 'Test1',
    primaryKey: 'userId',
    properties: {
      userId: 'string',
      embedded1: Embedded1Schema.schema.name,
    },
  };
}

type Test1 = {
  userId: string;
  embedded1: {
    embedded2: {array: boolean}[];
    relationship: {
      id: string;
      name: string;
    };
  };
};

const obj: Test1 = {
  userId: 'userId',
  embedded1: {
    embedded2: [{array: true}],
    relationship: {
      id: 'id',
      name: 'name',
    },
  },
};

const realm = getRealmInstance();
realm.write(() => {
  realm.create(Test1Schema, obj);
});

I had to remove extends Realm.Object for all embedded objects. As you can see, the realm.create statement has type errors for both embedded2 and relationship.