Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[experiment] Backport migration workaround #6196

Draft
wants to merge 2 commits into
base: 3.49
Choose a base branch
from

Conversation

pedro-psb
Copy link
Member

The #6161 was not very good because it still added the field to the database.

This approaches adds a unique field to the branch which won't conflict with any future field.
It must be cleaned up later with an idempotent remove migration.

Still testing if it runs without a problem.

On a request for on-demand content in the content app, a corrupted Remote
that contains the wrong binary (for that content) prevented other Remotes
from being attempted on future requests.

Now the last failed Remotes are temporarily ignored and others may be picked.

[BACKPORT NOTES]
* removed the failed_at migration.
* on a different commit, a unique migration to this branch will be
  used as a replacement

Original issue pulp#5725
Backported from: pulp#6064
Original backport with migration already removed:
2f3b57b

This adds unique migration field and update the code to use it.
This will avoid conflicts with future fields.

Must be cleaned up on a future release with an idempotent remove migration.
@pedro-psb
Copy link
Member Author

pedro-psb commented Jan 10, 2025

My attempt to test the upgrade process:

  1. Start on 3.49 with a clean install
  2. checking out to main <-- errors [0]

I assume I'm missing some steps (update deps, ...?). Will continue on monday.

[0]:

[pulp]  | Traceback (most recent call last):
[pulp]  |   File "/usr/local/lib/python3.9/site-packages/gunicorn/arbiter.py", line 608, in spawn_worker
[pulp]  |     worker.init_process()
[pulp]  |   File "/src/pulpcore/pulpcore/app/entrypoint.py", line 47, in init_process
[pulp]  |     django.setup()
[pulp]  |   File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
[pulp]  |     apps.populate(settings.INSTALLED_APPS)
[pulp]  |   File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 124, in populate
[pulp]  |     app_config.ready()
[pulp]  |   File "/src/pulpcore/pulpcore/app/apps.py", line 258, in ready
[pulp]  |     super().ready()
[pulp]  |   File "/src/pulpcore/pulpcore/app/apps.py", line 134, in ready
[pulp]  |     self.import_viewsets()
[pulp]  |   File "/src/pulpcore/pulpcore/app/apps.py", line 171, in import_viewsets
[pulp]  |     from pulpcore.app.viewsets import NamedModelViewSet
[pulp]  |   File "/src/pulpcore/pulpcore/app/viewsets/__init__.py", line 1, in <module>
[pulp]  |     from .base import (
[pulp]  |   File "/src/pulpcore/pulpcore/app/viewsets/base.py", line 16, in <module>
[pulp]  |     from pulpcore.openapi import PulpAutoSchema
[pulp]  |   File "/src/pulpcore/pulpcore/openapi/__init__.py", line 13, in <module>
[pulp]  |     from drf_spectacular.plumbing import (
[pulp]  | ImportError: cannot import name 'process_webhooks' from 'drf_spectacular.plumbing' (/usr/local/lib/python3.9/site-packages/drf_spectacular/plumbing.py)

@pedro-psb
Copy link
Member Author

The upgrade did work. Here were the steps:

  1. Compose up oci_env with this branch code (based on 3.49 and with the failed_at_backport49 field)
  2. Checkout to pulpcore main
  3. run pip install src/pulpcore
  4. run pulpcore-manager migrate

The results

[root@4a043b9b8239 /]# pulpcore-manager migrate
Operations to perform:
  Apply all migrations: auth, certguard, contenttypes, core, file, sessions
Running migrations:
  Applying core.0118_task_core_task_unblock_2276a4_idx_and_more... OK
  Applying core.0119_grouprole_core_groupr_object__250e22_idx_and_more... OK
  Applying core.0120_get_url_removal... OK
  Applying core.0121_add_profile_artifacts_table... OK
  Applying core.0122_record_last_replication_timestamp... OK
  Applying core.0123_upstreampulp_q_select... OK
  Applying core.0124_task_deferred_task_immediate... OK
  Applying core.0125_openpgpdistribution_openpgpkeyring_openpgppublickey_and_more... OK
  Applying core.0126_remoteartifact_failed_at... OK
Access policy for artifacts created.
Access policy for domains updated.
Access policy for repositories/core/openpgp_keyring created.
Access policy for task-groups created.
Access policy for tasks updated.
Access policy for login created.
[root@4a043b9b8239 /]# su pulp
bash-5.1$ psql
psql (13.16)
Type "help" for help.

pulp=> \d core_remoteartifact;
                        Table "public.core_remoteartifact"
        Column        |           Type           | Collation | Nullable | Default
----------------------+--------------------------+-----------+----------+---------
 pulp_id              | uuid                     |           | not null |
 pulp_created         | timestamp with time zone |           | not null |
 pulp_last_updated    | timestamp with time zone |           |          |
 url                  | text                     |           | not null |
 size                 | bigint                   |           |          |
 md5                  | character varying(32)    |           |          |
 sha1                 | character varying(40)    |           |          |
 sha224               | character varying(56)    |           |          |
 sha256               | character varying(64)    |           |          |
 sha384               | character varying(96)    |           |          |
 sha512               | character varying(128)   |           |          |
 content_artifact_id  | uuid                     |           | not null |
 remote_id            | uuid                     |           | not null |
 pulp_domain_id       | uuid                     |           | not null |
 failed_at_backport49 | timestamp with time zone |           |          |
 failed_at            | timestamp with time zone |           |          |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant