Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RKOTLIN-612] MongoClient API #1593

Merged
merged 44 commits into from
May 22, 2024
Merged

[RKOTLIN-612] MongoClient API #1593

merged 44 commits into from
May 22, 2024

Conversation

rorbech
Copy link
Contributor

@rorbech rorbech commented Dec 5, 2023

This PR adds MongoClient-APIs to query and update remote Atlas App Service data sources directly according to these docs

This overall flow is to obtain a MongoCollection though:

  • App->User->MongoClient->MongoDatabase->MongoCollection

with:

val app = App.create(APP_NAME)
val user = app.emailPasswordAuth.registerUser(email, password).run { app.logIn(email, password) }
val client = user.mongoClient(
    SERVICE_NAME,
    EJson(
        serializersModule = realmSerializerModule(
            setOf(
                ParentCollectionDataType::class,
                ChildCollectionDataType::class
            )
        )
    )
)
val database = client.database("databasename")
val collection: MongoCollection<BsonDocument>  = 
    database.collection("collectionName")
val primaryKey = 
    bsonTypedCollection.insertOne(BsonDocument("""{ "name" : "object-name, "_id" : 5} """)) as Int
val bsonDocument: BsonDocument? = 
    bsonTypedCollection.findOne(BsonDocument("""{ "name" : "object-name, "_id" : 5} """))

You can obtain typed collection instances that will automatically serialize arguments by specifying the object types of the collection.

// Typed collection with automatic serialization
val typedCollection = database.collection<CollectionDataType>("collectionName")

// Insert and object instance and get the primary key
val newDocumentId: Int = typedCollection.insertOne(CollectionDataType("object-name", _id = 6)) as Int

// Querying for object with BsonDocument-filter.
val document: CollectionDataType? = typedCollection.findOne(BsonDocument("name", "object-name"))

If sync is enabled and there is a schema defined for a specific RealmObject you can obtain the MongoCollection directly from the MongoClient. This allows obtaining a MongoCollection directly though:

  • App->User->MongoClient->MongoCollection

with:

val collection = client.collection<CollectionDataType>()

This will use the sync schema definition to map from the RealmObject to the collection of the corresponding type on the server.

Serialization of custom types and links between RealmObjects
The typed MongoCollection<T> supports serialization to/from Kotlin Serialization's @Serializable types by default.
Since MongoDB represents links only by their primary keys, special serializers needs to be configured if serializing links to/from existing RealmObject model classes. These can be added by supplying a SerializerModule to the ejson-serializer infrastructure when obtaining any of the relevant MongoDB API MongoClient, MongoDatabase or MongoCollection instances with:

// RealmObject-aware serializer module
val ejsonSerializer = EJson(
    serializersModule = realmSerializerModule(
        setOf( ... list of RealmObject model classes ...)
    )
)

// Above serializer can be injected into the hierarchy at the appropriate level and will be inherited from the parent entity if left unspecified
val client = user.mongoClient(<SERVICE_NAME>, ejson)
// or
val database = user.database(<DATABASE_NAME>, ejson)
// or
val collection = database.collection(COLLECTION_NAME, ejson)

The main entry points and main public APIs are:

Input to review
One of the main issues with this PR is how to control serialization. Kotlins serialization framework is heavily dependant of compile time derivable information which has made it a bit tricky. Some of the concerns was:

  • Not require extensive annotations throughout the model definition
  • Chosen not to generate serializers from the compiler plugin as it is tedious to produce and difficult to debug
  • Allow usage of standard Kotlin serialization framework serializers for non-RealmObject types
  • Lack of reflection requires some upfront definition of known classes to be able to create object instances from typed links (links in mixed fields) where the schema information can't dictate which class instance to.

The above led me to write a generic serializer that reuses existing EJSON<->BsonDocument serializer from github.com/mongodb/kbson and control serialization from statically stored model metadata. This however needs runtime information of the KClass and cannot easily be added through class annotations. Alternative would have been to write a new decoder that could hold the set of className->RealmObjectCompanion needed to construct typed link instances, as there is no other way to maintain a runtime-context during serialization. Instead I continued the approach from AppConfiguration.ejson of allowing to inject the full StringFormat'er. To ease configuration all realm object serialializers must be added to the StringFormat'er as a speciel SerializerModule constructed from realmSerializerModule.

TODOs

Closes #972

@rorbech rorbech marked this pull request as ready for review December 18, 2023 16:27
Copy link
Contributor

@cmelchior cmelchior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First round of feedback. Haven't gone over the tests yet. Most things look great.

Some docs seems to be missing and I have some reservations about the return types in MongoCollection

Copy link
Contributor

@cmelchior cmelchior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, mostly just minor documentation issues at this point.

rorbech and others added 3 commits January 2, 2024 10:01
Co-authored-by: Christian Melchior <[email protected]>
# Conflicts:
#	CHANGELOG.md
#	packages/test-sync/src/commonMain/kotlin/io/realm/kotlin/test/mongodb/util/AppAdmin.kt
#	packages/test-sync/src/commonMain/kotlin/io/realm/kotlin/test/mongodb/util/TestAppInitializer.kt
#	packages/test-sync/src/commonTest/kotlin/io/realm/kotlin/test/mongodb/common/utils/Utils.kt
@rorbech rorbech changed the title MongoClient API [RKOTLIN-612] MongoClient API Apr 5, 2024
@rorbech rorbech mentioned this pull request Apr 10, 2024
@rorbech rorbech requested a review from nielsenko April 10, 2024 10:45
Copy link
Contributor

@nielsenko nielsenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but then again my exposure to this SDK is very limited

Copy link
Contributor

@clementetb clementetb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, amazing job. Some minor comments.

.github/actions/run-android-device-farm-test/action.yml Outdated Show resolved Hide resolved
}

@Test
open fun findOne() = runBlocking<Unit> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why it has to be open?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was to overwrite the test in the actual implementations. I have remove the open. Actually it is now possible to select which implementing class you want to run the test for when you click the gutter icon for a test in an interface in latest Android Studio. That is quite neat.

fun findOne_extraFieldsAreDiscarded() = runBlocking<Unit> {
val withDocumentClass = collection.withDocumentClass<BsonDocument>()
withDocumentClass.insertOne(
BsonDocument(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if it could be written as:

Bson {
    "_id" to Random.nextInt(),
    "name" to "object-1",
    "extra" to "extra",
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added this convenience extension constructor in https://github.com/realm/realm-kotlin/pull/1593/files#diff-62866c3bf368e1c82bfc172e3aa985f0d097e7ae6bb93002d12e5b78978c235dR1418. More clever DSL-construct should probably be filed as an issue in https://github.com/mongodb/kbson

@rorbech rorbech requested a review from clementetb May 15, 2024 11:46
Copy link
Contributor

@clementetb clementetb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great

@rorbech rorbech merged commit cd6cc63 into main May 22, 2024
56 checks passed
@rorbech rorbech deleted the cr/mongo-client branch May 22, 2024 09:53
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mongo Client API's
4 participants