Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix JVM memory leaks #1526

Merged
merged 5 commits into from
Oct 6, 2023
Merged

Fix JVM memory leaks #1526

merged 5 commits into from
Oct 6, 2023

Conversation

clementetb
Copy link
Contributor

This PR fixes some native memory leaks caused when handling core:

  • Schema creation leaked strings buffers.
  • Querying the schema leaked property pointer buffers.
  • Converting sync errors leaked a map.
  • Retrieving all user identities leaked an array.
  • Collection changesets leaked indexes and change arrays.
  • Query parsing leaked arguments buffers.

@clementetb clementetb self-assigned this Sep 25, 2023
@cla-bot cla-bot bot added the cla: yes label Sep 25, 2023
@clementetb clementetb marked this pull request as ready for review September 25, 2023 12:47
Copy link
Contributor

@rorbech rorbech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catches. Wouldn't it make sense to encapsulate all these allocations (PropertyInfo, PropertyInfo[], ClassInfo, ClassInfo[], etc.) into the MemTrackingAllocator so that we could just use our normal inputScope { } for these kind of blocks?

Copy link
Contributor

@cmelchior cmelchior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great find. I do have some questions about some of the implementations though.

Comment on lines +125 to +148
for (classIndex in 0 until count) {
val classInfo = realmc.classArray_getitem(cclasses, classIndex)

// Clean properties
val propertyArray = realmc.propertyArrayArray_getitem(cproperties, classIndex)

val propertyCount = classInfo.getNum_properties() + classInfo.getNum_computed_properties()
for (propertyIndex in 0 until propertyCount) {
val property = realmc.propertyArray_getitem(propertyArray, propertyIndex.toInt())

realmc.realm_property_info_t_cleanup(property)
property.delete()
}

realmc.delete_propertyArray(propertyArray)

// end clean properties

realmc.realm_class_info_t_cleanup(classInfo)
classInfo.delete()
}

realmc.delete_propertyArrayArray(cproperties)
realmc.delete_classArray(cclasses)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I get why we need to do cleanup this way? Also, this is quite a lot of JNI traversals to do cleanup. Couldn't we move this entire block into a helper function in realm_api_helpers?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, doesn't realmc.classArray_getitem cause a new allocation rather than returning the old one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But now that you mention, in realm_get_class we could free the allocated struct afterwards.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still doing a ton of JNI traversals. My comment was about us moving the entire for-loop into C++. I assume that is possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is possible. I wanted to use, if possible, swig APIs, and then in a separate PR move this logic to c++ to measure the actual performance gain from doing it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, fair enough 👍

Copy link
Contributor

@rorbech rorbech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per discussion I am OK with just fixing this now, and making a nicer scoped solution when optimizing further ... but I think we are missing a leak of list query-arguments 🧐

)
)
} finally {
realmc.delete_queryArgArray(args.head)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we release everything here. The query argument array holds a pointer to a value array allocated in JvmMemAllocator.allocRealmValueList. Maybe not showing up in the benchmarks if you are not testing with list query-arguments (query("x in $0", [1, 2, 3])

Copy link
Collaborator

@nhachicha nhachicha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@clementetb clementetb merged commit 330a06c into releases Oct 6, 2023
1 check passed
@clementetb clementetb deleted the ct/jvm-memory-leaks branch October 6, 2023 07:59
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 14, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants