Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test dt adapter #5524

Open
wants to merge 37 commits into
base: wip-NXDRIVE-2929-upgrade-python-from-3.9.5-to-3.12.3
Choose a base branch
from

Conversation

gitofanindya
Copy link
Collaborator

@gitofanindya gitofanindya commented Dec 18, 2024

Summary by Sourcery

Upgrade the Python version to 3.12 across various CI workflows and update the pywin32 dependency to version 306. Modify datetime handling to use timezone-aware datetime objects. Update documentation to reflect the new Python version support.

Enhancements:

  • Update the pywin32 dependency to version 306 in the requirements file.

CI:

  • Upgrade the Python version to 3.12 in multiple GitHub Actions workflows, including functional, unit, integration, lint, spell, style, translations, and types tests.

Documentation:

  • Update documentation to reflect support for Python 3.12.3 instead of 3.9.5.

dependabot bot and others added 22 commits September 27, 2024 12:34
* Bump platformdirs from 4.2.2 to 4.3.6 in /tools/deps

Bumps [platformdirs](https://github.com/tox-dev/platformdirs) from 4.2.2 to 4.3.6.
- [Release notes](https://github.com/tox-dev/platformdirs/releases)
- [Changelog](https://github.com/tox-dev/platformdirs/blob/main/CHANGES.rst)
- [Commits](tox-dev/platformdirs@4.2.2...4.3.6)

---
updated-dependencies:
- dependency-name: platformdirs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump faker from 22.0.0 to 29.0.0 in /tools/deps (#5271)

Bumps [faker](https://github.com/joke2k/faker) from 22.0.0 to 29.0.0.

* Bump build from 1.2.1 to 1.2.2 in /tools/deps (#5274)

Bumps [build](https://github.com/pypa/build) from 1.2.1 to 1.2.2.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
* Bump watchdog from 3.0.0 to 5.0.2 in /tools/deps

Bumps [watchdog](https://github.com/gorakhargosh/watchdog) from 3.0.0 to 5.0.2.
- [Release notes](https://github.com/gorakhargosh/watchdog/releases)
- [Changelog](https://github.com/gorakhargosh/watchdog/blob/master/changelog.rst)
- [Commits](gorakhargosh/watchdog@v3.0.0...v5.0.2)

---
updated-dependencies:
- dependency-name: watchdog
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump mypy from 1.10.0 to 1.11.2 in /tools/deps (#5207)

Bumps [mypy](https://github.com/python/mypy) from 1.10.0 to 1.11.2.

* Bump responses from 0.24.1 to 0.25.3 in /tools/deps (#5006)

Bumps [responses](https://github.com/getsentry/responses) from 0.24.1 to 0.25.3.

* Bump python-dateutil from 2.8.2 to 2.9.0.post0 in /tools/deps (#4680)

Bumps [python-dateutil](https://github.com/dateutil/dateutil) from 2.8.2 to 2.9.0.post0.

* Bump types-python-dateutil in /tools/deps (#5230)

Bumps [types-python-dateutil](https://github.com/python/typeshed) from 2.8.19.20240106 to 2.9.0.20240906.


---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
* Bump pyobjc-framework-cocoa from 10.1 to 10.3.1 in /tools/deps

Bumps [pyobjc-framework-cocoa](https://github.com/ronaldoussoren/pyobjc) from 10.1 to 10.3.1.
- [Release notes](https://github.com/ronaldoussoren/pyobjc/releases)
- [Changelog](https://github.com/ronaldoussoren/pyobjc/blob/master/docs/changelog.rst)
- [Commits](ronaldoussoren/pyobjc@v10.1...v10.3.1)

---
updated-dependencies:
- dependency-name: pyobjc-framework-cocoa
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump pyobjc-framework-scriptingbridge from 10.1 to 10.3.1 in /tools/deps (#4987)

Bumps [pyobjc-framework-scriptingbridge](https://github.com/ronaldoussoren/pyobjc) from 10.1 to 10.3.1.

* Bump pyobjc-core from 10.1 to 10.3.1 in /tools/deps (#4990)

Bumps [pyobjc-core](https://github.com/ronaldoussoren/pyobjc) from 10.1 to 10.3.1.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
)

* Bump pyobjc-framework-fsevents from 10.1 to 10.3.1 in /tools/deps

Bumps [pyobjc-framework-fsevents](https://github.com/ronaldoussoren/pyobjc) from 10.1 to 10.3.1.
- [Release notes](https://github.com/ronaldoussoren/pyobjc/releases)
- [Changelog](https://github.com/ronaldoussoren/pyobjc/blob/master/docs/changelog.rst)
- [Commits](ronaldoussoren/pyobjc@v10.1...v10.3.1)

---
updated-dependencies:
- dependency-name: pyobjc-framework-fsevents
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump pyobjc-framework-coreservices from 10.1 to 10.3.1 in /tools/deps (#4986)

Bumps [pyobjc-framework-coreservices](https://github.com/ronaldoussoren/pyobjc) from 10.1 to 10.3.1.

* Bump pyobjc-framework-systemconfiguration in /tools/deps (#4989)

Bumps [pyobjc-framework-systemconfiguration](https://github.com/ronaldoussoren/pyobjc) from 10.1 to 10.3.1.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
* Bump send2trash from 1.7.1 to 1.8.3 in /tools/deps

Bumps [send2trash](https://github.com/arsenetar/send2trash) from 1.7.1 to 1.8.3.
- [Release notes](https://github.com/arsenetar/send2trash/releases)
- [Changelog](https://github.com/arsenetar/send2trash/blob/master/CHANGES.rst)
- [Commits](https://github.com/arsenetar/send2trash/commits)

---
updated-dependencies:
- dependency-name: send2trash
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump altgraph from 0.17 to 0.17.4 in /tools/deps (#4565)

Bumps [altgraph](https://github.com/ronaldoussoren/altgraph) from 0.17 to 0.17.4.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
* Bump requests from 2.31.0 to 2.32.3 in /tools/deps

Bumps [requests](https://github.com/psf/requests) from 2.31.0 to 2.32.3.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](psf/requests@v2.31.0...v2.32.3)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump future from 0.18.3 to 1.0.0 in /tools/deps (#4693)

Bumps [future](https://github.com/PythonCharmers/python-future) from 0.18.3 to 1.0.0.

* Bump pefile from 2023.2.7 to 2024.8.26 in /tools/deps (#5191)

Bumps [pefile](https://github.com/erocarrera/pefile) from 2023.2.7 to 2024.8.26.

* Bump pycparser from 2.21 to 2.22 in /tools/deps (#4753)

Bumps [pycparser](https://github.com/eliben/pycparser) from 2.21 to 2.22.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
* Bump black from 23.12.1 to 24.10.0 in /tools/deps

Bumps [black](https://github.com/psf/black) from 23.12.1 to 24.10.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.12.1...24.10.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pooja Ramkrishna Daine <[email protected]>
Co-authored-by: unknown <[email protected]>
Copy link
Contributor

sourcery-ai bot commented Dec 18, 2024

Reviewer's Guide by Sourcery

This PR implements datetime handling improvements and upgrades Python version from 3.9 to 3.12. The main changes focus on proper timezone handling in datetime operations and adapting datetime objects for SQLite storage. The PR also includes dependency updates and removal of unused code.

Class diagram for AutoRetryCursor modifications

classDiagram
    class AutoRetryCursor {
        +adapt_datetime_iso(val)
        +execute(sql: str, parameters: Iterable[Any])
    }
    note for AutoRetryCursor "Added adapt_datetime_iso method and modified execute method to handle datetime objects with timezone."
Loading

Class diagram for datetime handling changes

classDiagram
    class datetime {
        +fromtimestamp(timestamp, tz)
        +now(tz)
    }
    note for datetime "Updated usage to include timezone-aware datetime operations."
Loading

File-Level Changes

Change Details Files
Improved datetime handling with explicit timezone information
  • Replaced datetime.utcnow() with datetime.now(tz=timezone.utc)
  • Added timezone handling in SQLite adapter
  • Updated datetime conversion for file modification times
nxdrive/dao/base.py
nxdrive/dao/engine.py
nxdrive/client/local/base.py
nxdrive/engine/engine.py
nxdrive/engine/watcher/remote_watcher.py
Upgraded Python version from 3.9 to 3.12
  • Updated Python version in all CI workflow files
  • Updated documentation to reflect new Python version
  • Updated deployment scripts with new Python version
.github/workflows/functional_tests.yml
.github/workflows/unit_tests.yml
.github/workflows/dead_code.yml
.github/workflows/integration_tests.yml
docs/deployment.md
docs/gnu_linux.md
docs/support.md
tools/posix/deploy_ci_agent.sh
tools/windows/deploy_ci_agent.ps1
Updated dependencies and removed unused code
  • Updated pywin32 from version 301 to 306
  • Removed unused MockedPath class
  • Modified database cleanup approach in tests
tools/deps/requirements.txt
tests/unit/test_utils.py
tests/unit/conftest.py
tests/unit/test_proxy.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time. You can also use
    this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @gitofanindya - I've reviewed your changes - here's some feedback:

Overall Comments:

  • The sqlite datetime adapter implementation in AutoRetryCursor needs cleanup - remove commented code and avoid direct sqlite3 imports. Consider using the standard sqlite3 date adapters.
  • The removal of MockedPath test class reduces test coverage. Please either restore the test coverage or document why these tests are no longer needed.
  • Using os.system() for directory removal is unsafe and platform-dependent. Please use pathlib or shutil functions instead.
Here's what I looked at during the review
  • 🟡 General issues: 3 issues found
  • 🟢 Security: all looks good
  • 🟡 Testing: 2 issues found
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

class AutoRetryCursor(Cursor):
def adapt_datetime_iso(self, val):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: The strftime('%s') format specifier is not guaranteed to work on all platforms, particularly Windows

Comment on lines +39 to +47
con = sqlite3.connect(str(database))
with dump_file.open(mode="w", encoding="utf-8") as f:
for line in con.iterdump():
f.write(f"{line}\n")

# Force write of file to disk
f.flush()
fsync(f.fileno())
con.close()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Use context manager pattern for database connection to ensure proper cleanup

This ensures the connection is closed even if an exception occurs during the dump process.

Suggested change
con = sqlite3.connect(str(database))
with dump_file.open(mode="w", encoding="utf-8") as f:
for line in con.iterdump():
f.write(f"{line}\n")
# Force write of file to disk
f.flush()
fsync(f.fileno())
con.close()
with sqlite3.connect(str(database)) as con, dump_file.open(mode="w", encoding="utf-8") as f:
for line in con.iterdump():
f.write(f"{line}\n")
# Force write of file to disk
f.flush()
fsync(f.fileno())

nxdrive/utils.py Outdated
@@ -1315,7 +1315,7 @@ def get_current_locale() -> str:
l10n_code = NSLocale.currentLocale()
l10n = NSLocale.localeIdentifier(l10n_code)
else:
l10n = locale.getdefaultlocale()[0] or ""
l10n = locale.getlocale()[1] or ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Incorrect index used for locale code - should be [0] instead of [1]

Index [1] returns the encoding instead of the locale code. This will cause incorrect locale detection.

@@ -40,7 +40,8 @@ def __enter__(self):

def __exit__(self, exc_type, exc_val, exc_tb):
self.dispose()
self.db.unlink()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (testing): Using os.system() for test cleanup is unsafe

Using os.system() for file operations in tests is not recommended as it's platform-dependent and can be unsafe. Consider using pathlib.Path.unlink() or shutil.rmtree() for more reliable cleanup.

Comment on lines -52 to 53
db.unlink()
os.system(f'rmdir "{db}"')

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (testing): Same unsafe cleanup pattern in test_proxy.py

Similar to the issue in conftest.py, using os.system() for cleanup is not recommended. Use standard Python file operations instead.

@gitofanindya gitofanindya changed the base branch from master to wip-NXDRIVE-2929-upgrade-python-from-3.9.5-to-3.12.3 December 18, 2024 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant