Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Database Synchronization Function for Game Playtime #586

Merged
merged 40 commits into from
Oct 20, 2024
Merged

Conversation

bagusnl
Copy link
Member

@bagusnl bagusnl commented Oct 5, 2024

Main Goal

Synchronize user game playtime to a database that the user can sync with other systems with privacy in mind.

Every user needs to have their own SQLite server (eg. Turso https://turso.tech/ free account should be sufficient) then supply all the necessary informations to Collapse (Database URL in https and the token). Collapse will generate the GUIDv7 as a user identifier table inside the database.

Caution

Changing User ID without taking notes of your current ID will cause the data to be dangling in the database and not being used anymore.
While you can edit the database yourself, it's a bit tedious, so please do NOT lose your user ID.

Playtime synced to the database in a few mechanism:

  1. At first region load

When region is loaded, playtime stats is first loaded from your local machine, then the system fetch and compare when the last time that region play time is updated, if the local is out of date then database value will be used and saved locally. If the database is out of date, then local value will be used and saved to the database.

  1. At game first launch (force push)
  2. Every 5 minutes when the game is running
  3. At game close (force push)

PR Status :

  • Overall Status : In Progress
  • Commits : In Progress
  • Synced to base (Collapse:main) : Yes
  • Build status : OK
  • Crashing : Yes
  • Bug found caused by PR : 0

bagusnl and others added 14 commits September 28, 2024 23:17
untested lollmao
This took waaaay too long for me

- Switched to Libsql.Client
not prod ready? fine by me
- Fixed config not loading due to my stupid ass putting the bracket
- sphagget over at DBHandler.cs, yum!
- SQL injection fixes
- Reinitialize DB instance when its stream expired
- Retry operation when error happens
- Use UserID hash as the table name
- Break on success in StoreKeyValue
- Fix log retry logic in QueryKey
Practically force the data in DB to be updated when user clicking either reset or update playtime
@bagusnl bagusnl linked an issue Oct 5, 2024 that may be closed by this pull request
@bagusnl bagusnl requested review from neon-nyan and gablm and removed request for neon-nyan October 5, 2024 18:16
@bagusnl bagusnl marked this pull request as draft October 5, 2024 18:16
@bagusnl bagusnl self-assigned this Oct 5, 2024
@bagusnl bagusnl added Enhancement New feature or request Area: Mirror/API Server Issue labeled for Mirror/API Server issue releated Area: Runtime Issue labeled for runtime, Hi3Helper.Core and other libraries used by Collapse Priority: Low labels Oct 5, 2024
Copy link

github-actions bot commented Oct 5, 2024

Qodana for .NET

It seems all right 👌

No new problems were found according to the checks applied

💡 Qodana analysis was run in the pull request mode: only the changed files were checked
☁️ View the detailed Qodana report

Contact Qodana team

Contact us at [email protected]

@bagusnl bagusnl requested a review from neon-nyan October 5, 2024 18:26
Copy link
Member

@gablm gablm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core functionality seems to be working fine.
However, there are some problems related to error handling and file initialisation when running Collapse for the first time.

Some Front-end related suggestions
  • Allow DB Url to not contain https:// or have a different protocol from https://

As the database link copied from Turso is in the format libsql://example.turso.io, it would be nice to be able to input the url directly like that.

  • Show a dialog after closing game is syncing wasn't successful.
  • Show a dialog if database connection is unsuccessful at start-up.

- Early return if DB function is disabled
- Put all the props init in the try-catch to prevent stopping error
- Not disposing _enabled to reduce amount of calls to config
- Removed test code
also random comments here and there
Removed unnecessary database query usage

smol optimization bby

Also prevent pull function to crash the app by returning Task
(learnt my lesson)
by discarding the return value, HUEHUEHUE
@bagusnl bagusnl requested a review from a team October 12, 2024 13:08
@bagusnl
Copy link
Member Author

bagusnl commented Oct 12, 2024

@CollapseLauncher/core-maintainers please finish review in 12 hours if you can, planning to merge this PR on the weekend

Copy link
Contributor

@shatyuka shatyuka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a quick review.

Copy link
Collaborator

@Cryotechnic Cryotechnic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks ok, I share some concerns with shatyuka as well as some other things.

}
catch (Exception e)
{
LogWriteLine($"[DBHandler::Init] Error when (re)initializing database system!\r\n{e}", LogType.Error, true);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how pertinent it would be to show user-facing error/warning in the form of a dialog but could be good.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error will never be displayed as public because I want to set the database system to be completely in the background and non-invasive to the UX. I tried to do push in-app notifications but it seems currently the caller for it doesn't work outside HomePage methods (maybe i am dum, probably, most likely)

Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Hi3Helper.Core/Lang/en_US.json Outdated Show resolved Hide resolved
Copy link
Member

@gablm gablm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functionality wise everything seems to be working apart from statistic syncing.
I agree with cryo that could be good to show more errors in the UI when they occur.

Related to changing the url/token/uid in the settings, I believe it would be better to have a "save & validate" button instead of instantly changing the fields (or showing some dialog if first connection fails)
As it is, it is possible to insert a incorrect url and/or token and no error would ever be displayed if the user never clicks the "Validate Settings" button.

gablm and others added 4 commits October 14, 2024 00:25
This commit allows user to input any string as User ID, but still allows and auto generates GUIDv7 as User ID when needed

This also adds a warning badge next to save and validate button, notifying user to save the config when any fields has been changed
@bagusnl
Copy link
Member Author

bagusnl commented Oct 20, 2024

After discussing with neon in chat, I'm merging this PR. Should there be any concern of the code, hit me up in this PR comment or directly to me

@bagusnl bagusnl merged commit 3d53419 into main Oct 20, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Mirror/API Server Issue labeled for Mirror/API Server issue releated Area: Runtime Issue labeled for runtime, Hi3Helper.Core and other libraries used by Collapse Enhancement New feature or request Priority: Low
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Request]: Sync/save playtime between devices via authentication
5 participants