[17.0][WIP][MIG] connector_jira: Migration to 17.0 #108
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
from
8967f55
to
880affa
Compare
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
2 times, most recently
from
e477bd2
to
affb16d
Compare
SilvioC2C
changed the title
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
2 times, most recently
from
095e311
to
c188178
Compare
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
3 times, most recently
from
158a241
to
47caa23
Compare
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
from
47caa23
to
07029ca
Compare
Add project binding Add export of projects Add Jira project tasks model Add external dependencies in manifest Add security accesses Add basic import of tasks Task import are triggered by Jira webhooks. A batch import must be added later to import the changes from missed hooks or for those not wanting to use the webhooks. Add a basic backend adapter Batch import of project tasks Remove markdown The fields are not in markdown, but in a wiki syntax (proper to JIRA yet be determined) Link users and import tasks assignees Import worklogs Use an adapter for the project export Import issue types Filter type of issues to synchronize And assign the worklog to the first parent we find that we synchronize. If the type of the worklog's issue is synchronized, the worklog is assigned on this issue. If the worklog's issue is a sub-task, and the parent task of the sub-task is of a synchronized type, it link it to it. If the task has an epic, and the epics are synchronized, it link it to it. In last resort, the worklog will be linked with the project but with no task. Configure the name of the Epic Link field from API Create webhooks in JIRA from Odoo Choose template on project creation Discard Epics from different projects JIRA allow to choose an epic for any project. Setting the Epic as the Odoo task would create inconsistencies between the project and the task. We discard such an Epic, likely the task will be empty. Delete worklogs deleted on JIRA (only with webhooks) Delete tasks deleted on JIRA (only with webhooks) Add JIRA issue type in tasks Add JIRA Issue Key Add core business project types Improve display of JIRA backend Use a dedicated button to activate the Epic Link. Link the Epic in a dedicated field Add JIRA Parent for subtasks Show JIRA key in name Create project from shared project Check JIRA project keys Remove the button to 'one-click-export' a project It can't be working because we have other fields to configure on the binding. Add help on issue type synchronizations Accept tasks without parent Do not use id builtin
add `jira_account_analytic_line` object
Replace openerp namespace by odoo namespace Change to new Binder's API Migrate connector_jira to new 10.0 API Make addons uninstallable Apply 2to3 for python3 Migrate to new Component API Add setup guide in readme Migrate to v11 and Py3 Allow to delete a backend if we have issue types
Fix project listener name typo Add links to JIRA issue on a.a.l Fix connector tab naming
text_field as attribute could not by validated by RNG, options is always valid Add base to support both export and link with projects Now linking a project with JIRA must be done through a wizard instead of the binding view. It must be either export (create the project on jira) or link (link with an existing project on jira). We keep this choice in the binding so we never export data to JIRA when we chose the link only.
Map projects by external_id + set of jira orgs Project bindings now can be assigned to one or more jira organizations. The binding for the project accept an additional argument for organizations. A task will be linked with the project having the exact same set of organizations that it has, or fallback to a project without organization. A constraint ensures that you cannot have several projects with the same set of organizations or 2 projects without organization. The link wizard has a new step to select the organization. The REST API for Serviced Desk is a different one. The former code was based on pycontribs/jira#388 which is closed and unmaintained. We only need to read the organizations from the servicedesk REST API and the local code is minimal. We can now use the normal jira library.
A normal user must be able to use the jira rest client under the hood. Update documentation Store original jira issues on analytic lines Instead of the issue they are linked to in Odoo. With the mechanism in place, if we don't import the task or bug issue types and we synchronize the epics, the worklogs will be attached to the epics of their tasks (of subtasks to tasks). The fields were showing the values of the Epic (or task for subtask), though it makes much more sense to keep the keys and URLs of the original issue and epic on the analytic lines. We still have the link to the task if we want to get the URL for the task they are currently linked to. Fix project binding unique constraint We should be able to have 2 bindings without external_id (not yet exported) Add wizard to link a task to JIRA Refactorize multistep wizards with a mixin Handle jira bindings in tasks merge wizard Import name of Epics instead of summary The Epic issues have a special (custom) field for their name. In fact they use both, the custom field AND the summary field. But I guess they are better identified by their epic name than their summary. Extract multi_step_wizard as an addon Set a timeout on requests to JIRA Fix typo in readme Fix backend selection on wizard to link a task The computed field was not called, due to the model being a TransientModel, replace it by default values. Add is_master on jira project bindings Allow to have one project binding per project type The unicity constraint (backend_id, odoo_id) on jira.backend.backend is relaxed: it now allows one binding of each type. The reason for this is: * supporting several projects of different types is a requirements (eg. 1 service desk and 1 software) * but if we implement new features like "if I create a task it is pushed to Jira", with different projects we would not know where to push them Using this constraint, we'll be able to focus new export features by project type. Move jira_key from project to jira project binding As we can have more than one project binding, we cannot store a single jira key. In case we have more than one jira key for a project, we compute a compound key joined by commas. Add method to handle jira API / connection errors Add migration for 11.0.1.1.0 Fix error when several odoo users match When we are importing a worklog for a user not yet linked and we found several candidate users in odoo, they will both be linked with the same jira user and make the unicity constraint fail. Properly raise an error in this case. Fix project_type as string Add fallback project for worklogs A new optional field on the backend allows to choose a fallback project for the worklogs. When a worklog doesn't match any project linked with Jira, they will be created there, allowing to find the misconfigurations and fix them. Add action to reimport worklogs from Jira This is meant mainly to be used when a worklog has been imported in the fallback project and we need to re-affect them to the correct project after we linked it. Add related action to open jira with import issues The 'related' button on jobs which import issues or worklog will now open jira directly on the issue. Skip import of tasks/worklogs before batch dates Jira may send webhooks for old records, for instance worklogs from 6 month ago because their task has been changed. This change ensures that we never import any record which has a last update date before the last date of batch import. The job methods now return the result of Importer.run() so the result is shown on the jobs in the UI. Extract import of dependencies Allowing to override partially the dependencies to import Fix import of subtasks When importing the parent of a subtask, the record in the 'parent' field (the task data) is incomplete, it contains only a few fields. Providing only the id to the dependency Importer will force it to read the whole record from Jira. Prevent duplicates with inactive projects The constraint did not look for inactive projects, which makes the import fail later because it finds several projects for the same task. Handle 404 errors when importing records When a record does not exist on Jira: * the job is done instead of failed * a result on the job tells about the missing record * the binding is deleted on Odoo * for worklogs, the analytic line is deleted as well Change xmlid for model_account_analytic_line account is not necessarily installed, while analytic is installed by the project module in any case Partial revert of 2fa7d39 Commit title was: Skip import of tasks/worklogs before batch dates The process is the following: * Read T from backend, T is the is the last time we ran the import * Create a batch job to import tasks/worklogs from T to now() and update the backend with now() The batch job is run asynchronously, it generates one job per task or worklog to import between T and now(). When the jobs are executed, the last batch date has already been updated to now(), so the jobs would never import any record. Fix error when force_reimport called on several records Prevent changing task's project on binding When the task Mapper binds a task to a task already existing in Odoo, we must not change the project_id of the task. This is not possible as long as we have invoiced timesheet lines, even if the id is the same than the current one. Add base for tests and a test for oauth Add test for check of connection With a fix in the method: getting the client only wasn't triggering any error, calling 'myself' raises a 401 when we are not authenticated. Add documentation for tests Add travis configuration Add tests on issue types and tasks Add first test for analytic account lines Make pylint-odoo happy Add a script to initiate the Oauth dance
Co-Authored-By: guewen <[email protected]>
When another user is already linked with the same jira user and matches. Without this check, the transaction fails with a unique constraint error. Do not change project_id on tasks updates When a task has invoiced timesheet lines, writing on 'project_id', even if the id is the same, would fail because of a constraint preventing to write project_id on timesheet lines. Refactorize aal batch importer with 'updating since' jira method This Jira method returns the list of ids updated since a timestamp, and the youngest timestamp, which means we can store this timestamp and reuse it the next time we ask the modified worklogs: this way we should never miss any update. To support this, a type of field which support milliseconds in datetime is required, otherwise lose this precision that Jira gives us. As of today, the "updated since" method exists on for worklogs on the Jira webservice, so the rest of the batch import still have to use the JQL filters (with minute-precision only). Use Datetime as main format for MilliDatetime field Store jira's 'updated_at' field to compare with binding We can compare the millisecond dates on both sides to see if any changes has been done on Jira and avoid useless writes. Previously, we were using the 'sync_date' field, which works too but will make more sync than necessary (because the sync happens after the last jira's updated_at value). Fortuitously, the updated_at is also very useful for debugging purposes. Add batch deleter for worklogs Force reimport of worklogs as admin Add tests for jira.backend.timestamp Add jira.backend tests Fix JiraTransactionCase → JiraSavepointCase 👀 Add tests for batch importer of tasks and worklogs Add tests for worklogs deletes Fix lint
Co-Authored-By: guewen <[email protected]>
Apply suggestions from @simahawk in code review Add scheduled action for importing deleted worklogs Fix fetch pagination of organizations The pagination specs of the servicedesk API are not the same than the Jira API (respectively isLastPage vs lastPage). So the lib's _fetch_pages does not fetch all the pages. Optimize the requests: we get all the data at the initial call, so the records are passed directly to "import_record" which will not do another request.
It seems that when updating other addons, even if we don't update connector_jira, it may already create the columns, then when we update connector_jira, it tries to create the columns when they already exist. Check before. Use the rendered task description as HTML The Jira REST API returns a rendered version of the description instead of the markup when we pass the 'expand=renderedFields' parameter. Update vrcpy cassettes as we read expanded issues With "expand=renderedFields"
Imported task on inactive project set task inactive too
Partial revert of 13ba6ec When a user records a tempo worklog, a Tempo Worklog is generated, which also generates a Jira Worklog. This Jira Worklog is the one we receive through the webhook. It is linked to the Tempo App user and not to the Jira user who recorded the worklog, and the description is a standard description. -> if we want things to work when Tempo is used, we need to only use the webhook as a notification, and not as a data source.
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
from
07029ca
to
1a6f0ce
Compare
|
Hi @SilvioC2C |
|
@MriV13 hello!
We are starting tests in the next few days, as soon as our Jira provider is able to prepare a Jira integration instance. We've been using it on v15, but the old module was connecting to a Jira Server instance, which is now deprecated in favor of Jira Cloud.
To be completely honest, I don't know. I guess it could be worth testing it. |
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
from
1a6f0ce
to
3a245fa
Compare
yankinmax
approved these changes
Jul 19, 2024
MriV13
suggested changes
Aug 14, 2024
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
from
3a245fa
to
912dde4
Compare
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
4 times, most recently
from
e2fcd4d
to
4496312
Compare
SilvioC2C
force-pushed
the
17.0-mig-connector_jira
branch
from
4496312
to
699a31f
Compare
MriV13
suggested changes
Aug 26, 2024
| _inherit = "jira.import.mapper" | ||
| _apply_on = ["jira.account.analytic.line"] | ||
|
|
||
| direct = [(whenempty("comment", _("missing description")), "name")] |
There was a problem hiding this comment.
I'm not sure, if this should prevent issue for receiving Time logs from Jira without description, but there is still an error in such cases:
Exception:?KeyError
Traceback (most recent call last):
File "/home/odoo/src/user/oca-queue/queue_job/controllers/main.py", line 104, in runjob
self._try_perform_job(env, job)
File "/home/odoo/src/user/oca-queue/queue_job/controllers/main.py", line 36, in _try_perform_job
job.perform()
File "/home/odoo/src/user/oca-queue/queue_job/job.py", line 517, in perform
self.result = self.func(*tuple(self.args), **self.kwargs)
File "/home/odoo/src/user/connector_jira/models/jira_account_analytic_line.py", line 88, in import_record
return importer.run(worklog_id, issue_id=issue_id, force=force)
File "/home/odoo/src/user/connector_jira/components/jira_analytic_line_importer.py", line 119, in run
return super().run(external_id, force=force, record=record, **kwargs)
File "/home/odoo/src/user/connector_jira/components/jira_importer.py", line 364, in run
self._import(binding, **kwargs)
File "/home/odoo/src/user/connector_jira/components/jira_analytic_line_importer.py", line 164, in _import
return super()._import(binding, **kwargs)
File "/home/odoo/src/user/connector_jira/components/jira_importer.py", line 379, in _import
record = self._create_data(map_record)
File "/home/odoo/src/user/connector_jira/components/jira_analytic_line_importer.py", line 93, in _create_data
return super()._create_data(
File "/home/odoo/src/user/connector_jira/components/jira_importer.py", line 155, in _create_data
return map_record.values(
File "/home/odoo/src/user/oca-connector/connector/components/mapper.py", line 1016, in values
values = self._mapper._apply(self, options=options)
File "/home/odoo/src/user/oca-connector/connector/components/mapper.py", line 786, in _apply
return self._apply_with_options(map_record)
File "/home/odoo/src/user/oca-connector/connector/components/mapper.py", line 813, in _apply_with_options
value = self._map_direct(map_record.source, from_attr, to_attr)
File "/home/odoo/src/user/oca-connector/connector/components/mapper.py", line 882, in _map_direct
return from_attr(self, record, to_attr)
File "/home/odoo/src/user/connector_jira/components/common.py", line 156, in modifier
value = record[field]
KeyError: 'comment'
in function whenempty() first arg is:
:param field: name of the source field in the record
which means, should be "name" not a comment, right?
There was a problem hiding this comment.
@SilvioC2C
There is actually nothing wrong in this line what i point out.
Issue is in function there:
https://github.com/camptocamp/connector-jira/blob/699a31f87cee5b0e53b5d6b919f2f6494779135a/connector_jira/components/common.py#L156
There should be used .get()
when WorkLog is created without description, then it's not present in the record at all.
|
This is based on: Which is in draft, looks unfinished and probably abandoned. Also, could you add the temporary test-requirements.txt so that the CI is able to run tests? That would help a lot with reviews 🙏🏻 Finally, if it's ready, please mark it as so |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Migration based on top of #106