Regression(?) infinitely growing active versions in 1.10.2 when using Singleton to retrieve realm instance

[EdwardvanRaak]

How frequently does the bug occur?

Always

Description

During some debugging I discovered that our app is suffering from infinitely growing active versions after realm write transactions.

This is a topic with similarities to earlier issues such as #1007 and #1081

Minimum reproducible example:
With the setup below on Android, rotate the screen about 20 times (or make a button, whatever) and observe logs.

object Singleton { // Or Koin Singleton doesn't matter
    val config = RealmConfiguration.create(schema = setOf(Item::class))
    val realm = Realm.open(config)
}

class MainActivity : ComponentActivity() {

    @SuppressLint("SetTextI18n")
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        repeat(100) {
            realm.writeBlocking {
                copyToRealm(Item().apply {
                    summary = "Do the laundry"
                    isComplete = false
                })
            }
        }
        println(
            "Number of active versions: ${realm.getNumberOfActiveVersions()}"
        )
    }
}

class Item() : RealmObject {
    @PrimaryKey
    var _id: ObjectId = ObjectId()
    var isComplete: Boolean = false
    var summary: String = ""
    var owner_id: String = ""

    constructor(ownerId: String = "") : this() {
        owner_id = ownerId
    }
}

Log output:

com.example.myapplication            I  Number of active versions: 101
com.example.myapplication            I  Number of active versions: 201
com.example.myapplication            I  Number of active versions: 301
com.example.myapplication            I  Number of active versions: 401
... // Another 20+ rotations later
com.example.myapplication            I  Number of active versions: 2501
com.example.myapplication            I  Number of active versions: 2601
com.example.myapplication            I  Number of active versions: 2701

During rotating, I triggered several forced GCs through the Android Studio Memory profiler. However this has no effect on the number of active version.

Reverting the realm version back to 1.6.0 (because of this comment ) and running the exact same example, garbage collection does result in the number of versions decreasing.

Log output on v1.6.0

com.example.myapplication            I  Number of active versions: 101
com.example.myapplication            I  Number of active versions: 201
com.example.myapplication            I  Number of active versions: 301
<trigger manual GC in Android Studio>
com.example.myapplication            I  Number of active versions: 101

Interestingly, omitting the usage of the Singleton on version 1.10.2 such as this:

 @SuppressLint("SetTextI18n")
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val realm = Realm.open(Singleton.config)

        repeat(100) {
            realm.writeBlocking {
                copyToRealm(Item().apply {
                    summary = "Do the laundry"
                    isComplete = false
                })
            }
        }
        println(
            "Number of active versions: ${realm.getNumberOfActiveVersions()}"
        )
    }

Also results in GCs correctly decreasing the number of active version.

Stacktrace & log output

No response

Can you reproduce the bug?

Always

Reproduction Steps

No response

Version

1.10.2

What Atlas App Services are you using?

Local Database only

Are you using encryption?

No

Platform OS and version(s)

Android 13

Build environment

Android Studio version: Android Studio Flamingo | 2022.2.1 Patch 2

[cmelchior]

Hi @EdwardvanRaak

Thank you for reporting this. We have made changes to our version tracking in between those versions, but that should not cause this issue, so I'm not aware of any regressions. But we will investigate.

[ggajews]

I checked different versions starting from 1.6.0 and it seems that this issue was introduced in 1.8.0.
Could this be connected to this change? #1320

[EdwardvanRaak]

@cmelchior I can confirm what @ggajews found.

From my tests, huge realm file size is possible as long as an app process isn't killed and realm isn't compacted. Isn't this quite worrisome since these library versions are widely spread at this point? Or are you considering this a minor bug and can we release with something like this? I understand that support for free Realm products is limited so that's why I'm asking.

The setup described in my original post is one that is suggested by Realm (for example on the Kotlin Slack) and isn't uncommon I feel.

[cmelchior]

Sorry for not responding to this issue sooner. The purpose of #1320 was actually to make the filesize growth less likely to happen, not more. And at least our internal testing showed that to be the case, but it sounds like we have missed something when running these tests.

So I would consider this a bug we do want to fix. Unfortunately, we had some other issues taking priority on being able to dive into this one, but I hope to get back to it next week.

[clementetb]

@EdwardvanRaak The issue with the growing number of active versions is that the getNumberOfActiveVersions is not reporting the actual active version number but the difference between the latest and earlier version numbers.

Yes, since #1320 the numbers reported with getNumberOfActiveVersions change, we keep a reference open with the initial version, thus why the growing numbers, but it has a minimum effect because any intermediate versions are reclaimed automatically by Realm.

We are working on reporting the actual number of active versions in getNumberOfActiveVersions.

#1478 (comment) - Here you say that you have seen that the database size grows until it is compacted. What kind of data, and operations do you do? Could you share a sample project that shows this huge growth?

[EdwardvanRaak]

@clementetb
Thanks for the clarifications so far.

Yes here is a sample project https://gist.github.com/EdwardvanRaak/9097f7b642e8b13f5f901692f93c79eb for Android (Jetpack Compose) with a simple UI so you can see the file size slowly increasing over time.

[rorbech]

The number reported from getNumberOfActiveVersions was just not right. This is fixed in #1516 and available in the 1.11.2-SNAPSHOT.

[ggajews]

Now we are hitting 20 active versions max, so the fix worked 👍, thanks.
Any leaks on the release date for 1.11.2?

[clementetb]

@EdwardvanRaak The latest release 1.12.0 fixes some memory leaks and version pinning. Would you mind testing it and seeing if it addresses your issues?

[github-actions]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.