Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[17.0][MIG] DMS : Migration to 17.0 #323

Merged
merged 141 commits into from
Jul 26, 2024
Merged

Conversation

tva-subteno-it
Copy link
Contributor

@tva-subteno-it tva-subteno-it commented Apr 11, 2024

Migration of the DMS module to v17.

Code review is in progress, so changes might still occur.
Closes #269

Done:

  • Reorganized manifest into security, actions, templates, data, views and wizard. Same for the assets, organized by styles, JS and XML
  • Removed inexistant assets mail.assets_messaging
  • Changed the onbaording flow, for since v17, new models onboarding.onboarding.step and onboarding.onboarding has been added. Corresponding data has been added in the data folder. Onboarding actions per model has been added. The action_close has been added to the onboarding.onboarding model as well as different open actions in onboarding.onboarding.step. Also updated res_company to reflect those changes
  • Refactoring of portal_my_dms_directory and portal_my_dms controller methods
  • Rename of the directoy and access_group filename to fit the model name
  • Fix context parsing in fields, by changing the context from a string to a dictionnary (cf this line)
  • Fixed unlink method of Base model causing infinite loop. See the comment for more info
  • Fixed copy method of access group, as well as the ones in directory and file in order to display (copy) or (1) when duplicated
  • Fixed _name_get being replace by _compute_display_name, attrs="{'invisible... replaced by invisible="...
  • In _compute_permissions removed the if self.env.su since odoo already does it in the _filter_access_rules called a few line below
  • Changed security.xml to better handle what is the user is allowed to see/write/create/unlink by calling _get_allowed_directory_ids doing the job of checking what the user can do according to its access groups.
  • Created two new icons to fit the new design of Odoo V17
  • Removed useless Javascript file (not in the manifest anymore) as well as restructured the files into a more consistant structure
  • Changed Javascript files displaying file's preview to use the new approach, with the useFileViewer hook
  • Fixed the patch of multiple class using the solution in the Odoo documentation
  • Fix of the kanban view for directory and files to the new architecture. This led to some refactoring in the SCSS, to remove useless class names, and clean up some duplicated code between file/directory styles sheets
  • Changed the res_settings view according to the Migration guide v16->v17
  • Translations (maybe after the PR is merged)
  • Correct code according to the code review
  • Add documentation, docstrings, comments
  • Refactor code, re-order fields by alphabetical order for more readability, follow the contributing guide for file naming. Follow the attribute model order stated here. Done : abstract_dms_mixin, base, dms_access_group, dms_directory
  • Fixing tests

@tva-subteno-it tva-subteno-it marked this pull request as draft April 11, 2024 08:11
@tva-subteno-it tva-subteno-it mentioned this pull request Apr 11, 2024
3 tasks
@pedrobaeza
Copy link
Member

Let's put here further conversation about this migration instead on the general migration issue.

That icon may serve. Do you have it on SVG?

@tva-subteno-it
Copy link
Contributor Author

Yes, they are both under static/description.

@pedrobaeza
Copy link
Member

Great, then other contributors may improve it in the future. For now seems enough.

@pedrobaeza
Copy link
Member

Please check red CIs.

dms/readme/CREDITS.md Outdated Show resolved Hide resolved
@tva-subteno-it
Copy link
Contributor Author

Thanks for taking the time of doing a little code review, but I don't have finished the migration yet. Things might change over time (I'm not quite used of contributing to projects, so I don't know if I'm doing things right).
About the CIs, I think I will correct them all at once once the migration is finished, as well as documenting the code, translations etc.

@tva-subteno-it
Copy link
Contributor Author

What should I do about Copyrights ? I'm migrating this module because a customer needs it. So at my job we copyright every file with this :

Copyright 2024 Subteno (https://www.subteno.com).
License LGPL-3.0 or later (https://www.gnu.org/licenses/lgpl).

which matches the current license used.
Should I add the Copyright 2024 Subteno (https://www.subteno.com). line on every file ? And the whole thing if the file doesn't already have one ?

@tva-subteno-it
Copy link
Contributor Author

I added another icon for the portal. Similarly to the previous one, I edited it in order to fit the others portal's icon.
portal_icon
image

It was a bit hard to find the correct one, and edited it order to stay "minimalist" and follow the color scheme of the portal, but it doesn't look too odd I think.
Let me know your thoughts.

@pedrobaeza
Copy link
Member

Yes, it seems good. Don't underestimate your design skills 😉

About the copyright in headers, you should only add it on the files you touch.

@tva-subteno-it
Copy link
Contributor Author

In the directory model, the file_ids field's copy attribute is set to False, but in the copy method of the directory, the files are manually duplicated :

def copy(self, default=None):
    # ...
    for record in self.file_ids:
        record.copy({"directory_id": new.id})

Should the files be duplicated with the directory or not ? I think it can be quite dangerous if you want to duplicate a folder, but you forgot there are many files, or big files in it.

@pedrobaeza
Copy link
Member

About the copy, there are 2 aspects:

  • Functional / business logic one: When you copy a folder in a file explorer, the contents are copied, so I think it's OK to copy contents as well when duplicating. If the storage is attachment, no real space duplication will happen.
  • Implementation aspect: not sure why it's copy=False and done manually. Maybe when this was done (v12), copy didn't work correctly, or it was a thing of permissions due to the overrides there. As there are no code comments for knowing the reasons, we can only speculate. You can try switching to copy=True to see if you find any issue.

@tva-subteno-it
Copy link
Contributor Author

Well, I set copy=True, as well as removing the manual copy, and it worked juste fine. There was the same process for child_directories_ids, so I did the same and it worked.
Maybe indeed it was some leftovers from the V12

@tva-subteno-it
Copy link
Contributor Author

Am I allowed to change the ecmaVersion in .eslintrc.yml ? I would like to change it to 2020, because when running pre-commit, I'm getting an error on props.record.data?.path_json, simply because it doesn't recognize ?.. (even though it works).

@tva-subteno-it
Copy link
Contributor Author

Same for ruff error : B023 Function definition does not bind loop variable inside the lambda of a filtered. I could just ignore this specific error, because these errors are juste in the case of .filtered, which is unavoidable.
It would be much more difficult and less readable trying to remove this error, knowing that it's done on purpose.

@pedrobaeza
Copy link
Member

Am I allowed to change the ecmaVersion in .eslintrc.yml ?

I would say that the official specification for v17 is still ECMA 2016. @ged-odoo can you please tell us?

@pedrobaeza
Copy link
Member

B023 Function definition does not bind loop variable

This linter in fact is correct. We are not doing something totally correct, and should be changed. How it should be done? From the existing code:

Before:

tags = record.tag_ids.filtered(lambda rec: not rec.category_id or rec.category_id == record.category_id)

After:

tags = record.tag_ids.filtered(lambda rec, record=record: not rec.category_id or rec.category_id == record.category_id)

Copy link
Member

@victoralmau victoralmau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you unify all change commits into 1 (v17 migration) for easier review?

@victoralmau
Copy link
Member

Please, cherry-pick #327 to commit history before migration v17 commit.

@tva-subteno-it tva-subteno-it force-pushed the 17.0-mig-dms branch 3 times, most recently from 3bbb33e to 4cf0da4 Compare April 25, 2024 09:54
@tva-subteno-it
Copy link
Contributor Author

@victoralmau Everything should be done. I rebased all my commits and cherry-picked the last change. I'm currently working on the tests. Let me know if you have any question/suggestion, or anything else.

Copy link
Member

@victoralmau victoralmau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job, some things:

  • It would be missing the pre-commit commit before the v17 migration commit ([IMP] dms: pre-commit auto fixes) https://github.com/OCA/maintainer-tools/wiki/Migration-to-version-17.0 I think that would help to "reduce" the changes in the migration commit.

  • Can you indicate all the changes you have done in the migration commit?

  • On the other hand, other improvements (adding the has_all_access field from dms.access.group or is_first_creation field in dms.directory) should go in another commit [IMP] after the migration commit OR do it in another PR after merging this (to make it easier to review this PR).

.eslintrc.yml Outdated Show resolved Hide resolved
dms/controllers/portal.py Outdated Show resolved Hide resolved
dms/models/dms_access_groups.py Outdated Show resolved Hide resolved
dms/models/dms_category.py Outdated Show resolved Hide resolved
dms/models/dms_directory.py Outdated Show resolved Hide resolved
dms/models/dms_directory.py Outdated Show resolved Hide resolved
dms/models/dms_security_mixin.py Outdated Show resolved Hide resolved
dms/models/res_users.py Outdated Show resolved Hide resolved
dms/pyproject.toml Show resolved Hide resolved
dms/views/menu.xml Outdated Show resolved Hide resolved
@victoralmau
Copy link
Member

Please, cherry-pick #318 to commit history (before migration v17 commit).

@tva-subteno-it tva-subteno-it force-pushed the 17.0-mig-dms branch 2 times, most recently from e839217 to cb30f22 Compare April 26, 2024 08:39
@tva-subteno-it
Copy link
Contributor Author

Please, cherry-pick #318 to commit history (before migration v17 commit).

Done. Also, I edited the comment of this PR to list what's still to be done, and what's already done.

@tva-subteno-it
Copy link
Contributor Author

Thank you, I will check it out soon.

Exactly, if you have access to a directory with access_token you have access to everything in it (including any level).

Okay, I fixed it.

Copy link
Member

@victoralmau victoralmau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code and functional review OK.

Ping @CarlosRoca13

Copy link

@CarlosRoca13 CarlosRoca13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code and functional review 👍 Thanks 😄

@OCA-git-bot
Copy link
Contributor

This PR has the approved label and has been created more than 5 days ago. It should therefore be ready to merge by a maintainer (or a PSC member if the concerned addon has no declared maintainer). 🤖

Copy link
Member

@pedrobaeza pedrobaeza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have started to review the migration commit, but it's barely impossible due to all the "style changes". Please refrain from doing them on the migration commit. You may heard that squashing commits should be done, but not at this level. The commit should be the unit to ease the review, so you must squash when you do a progressive migration refinement through several commits, not when you are doing a total move of every line of code for placing according a specific order. I agree that is good to have the code sorted according certain guidelines, but the first rule is to minimize the diff on any existing code. Please check the rest of my inline comments where I expand these thoughts.

dms/models/dms_access_groups.py Outdated Show resolved Hide resolved
auto_join=False,
copy=False,
copy=True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is changing the behavior. AFAIK, the problem here is that we can't assure the security consistency on the copied files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? Everything should be copied just fine. The copy method in dms_file.py was not doing much than what Odoo now already does.

dms/models/abstract_dms_mixin.py Outdated Show resolved Hide resolved
dms/models/abstract_dms_mixin.py Outdated Show resolved Hide resolved
dms/models/abstract_dms_mixin.py Outdated Show resolved Hide resolved
dms/models/dms_access_groups.py Outdated Show resolved Hide resolved
dms/models/dms_category.py Outdated Show resolved Hide resolved
domain="[('permission_create', '=', True)]",
context="{'dms_directory_show_path': True}",
context={"dms_directory_show_path": True},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to change this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To change what? If you compare with like the v16 or v15 I only removed the outer quotes according to the official doc for the context param to be a dict and not a string

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link doesn't work to the documentation you mention doesn't work. Can you correct it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah you're right, sorry. Here is the new new link. If you search for other places where they use the context in fields, you'll see that they don't use quotes round the dict.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the link. Well, the option to provide strings is also supported, and it's useful when you need to put a dynamic expression depending on a field value for example, but in this case, it's the same. It was only for reducing the diff of the migration.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean that I can leave as is?

dms/models/dms_file.py Outdated Show resolved Hide resolved
dms/models/dms_file.py Outdated Show resolved Hide resolved
@tva-subteno-it
Copy link
Contributor Author

@pedrobaeza it should be all good now. I think I've reversed all the reordering I did, and you can know more easily do the code review.

@tva-subteno-it tva-subteno-it force-pushed the 17.0-mig-dms branch 4 times, most recently from f428007 to cfe838f Compare July 25, 2024 12:12
Copy link
Member

@pedrobaeza pedrobaeza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the efforts and your patience. I think we are almost there. Just some final comments, and I don't nickpit more.

dms/models/base.py Outdated Show resolved Hide resolved
dms/models/base.py Show resolved Hide resolved
dms/models/directory.py Outdated Show resolved Hide resolved
dms/models/directory.py Outdated Show resolved Hide resolved
domain="[('permission_create', '=', True)]",
context="{'dms_directory_show_path': True}",
context={"dms_directory_show_path": True},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link doesn't work to the documentation you mention doesn't work. Can you correct it?

@tva-subteno-it
Copy link
Contributor Author

@pedrobaeza I fixed what you asked, and sent the right url this time (I hope).

Copy link
Member

@pedrobaeza pedrobaeza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, there are still some stylistic things (like empty lines inside methods, as I didn't mention each of them), but let's merge it for not disturbing you more. Thanks for the work.

/ocabot merge nobump

@OCA-git-bot
Copy link
Contributor

What a great day to merge this nice PR. Let's do it!
Prepared branch 17.0-ocabot-merge-pr-323-by-pedrobaeza-bump-nobump, awaiting test results.

@OCA-git-bot OCA-git-bot merged commit fa038ac into OCA:17.0 Jul 26, 2024
7 checks passed
@OCA-git-bot
Copy link
Contributor

Congratulations, your PR was merged at 6dbce74. Thanks a lot for contributing to OCA. ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.