Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HJ 181 - Datahub groundwork and UI #5666

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

HJ 181 - Datahub groundwork and UI #5666

wants to merge 9 commits into from

Conversation

andres-torres-marroquin
Copy link
Contributor

@andres-torres-marroquin andres-torres-marroquin commented Jan 13, 2025 •
edited
Loading

Closes #HJ-181

Description Of Changes

Datahub should be in fidesplus

Code Changes

  • Added Data Catalog tab to Integrations UI
  • Added Datahub to the Data Catalog tab
  • Added Data Catalog to the SystemType enum
  • Removed DatahubSchema.frequency, it is going to be added back on the new type of monitor
  • Updated tests to work without DatahubSchema.frequency

Steps to Confirm

  1. list any manual steps for reviewers to confirm the changes

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Followup issues:
    • Followup issues created (include link)
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

@vercel Vercel
Copy link

vercel bot commented Jan 13, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
fides-plus-nightly ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 14, 2025 7:46pm

@cypress Cypress
Copy link

cypress bot commented Jan 13, 2025 •
edited
Loading

fides    Run #11753

Run Properties:  status check passed Passed #11753  •  git commit ce33cce095 ℹ️: Merge 5a66ca1d0e98aecb466d728c627d63abc901bc8a into de209feb31ce2619c9db608d41fa...
Project fides
Branch Review refs/pull/5666/merge
Run status status check passed Passed #11753
Run duration 00m 38s
Commit git commit ce33cce095 ℹ️: Merge 5a66ca1d0e98aecb466d728c627d63abc901bc8a into de209feb31ce2619c9db608d41fa...
Committer Andres Torres
View all properties for this run ↗︎
Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
View all changes introduced in this branch ↗︎

@codecov Codecov
Copy link

codecov bot commented Jan 13, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 91.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 87.14%. Comparing base (de209fe) to head (5a66ca1).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/fides/api/models/connectionconfig.py 87.50% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #5666      +/-   ##
==========================================
- Coverage   87.14%   87.14%   -0.01%     
==========================================
  Files         388      388              
  Lines       24012    24013       +1     
  Branches     2593     2593              
==========================================
  Hits        20926    20926              
- Misses       2525     2526       +1     
  Partials      561      561

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@andres-torres-marroquin andres-torres-marroquin changed the title HJ 181 - Datahub should be in fidesplus HJ 181 - Datahub groundwork and UI Jan 14, 2025
erosselli
erosselli reviewed Jan 14, 2025
View reviewed changes
clients/admin-ui/src/features/integrations/integration-type-info/datahubInfo.tsx
Comment on lines +35 to +39
<ListItem>Database</ListItem>
<ListItem>SQL database</ListItem>
<ListItem>Storage system</ListItem>
<ListItem>Data detection</ListItem>
<ListItem>Data discovery</ListItem>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we mentioned these were going to be placeholders, but it's probably better to just leave an empty list right? or just one element that reads "Data catalog" for now. I think we can leave "smarter" placeholders than just copy-pasting it from some other monitor and causing potential confusion

clients/admin-ui/src/features/integrations/integration-type-info/datahubInfo.tsx
Comment on lines +43 to +55
<ListItem>CREATE ROLE my_monitor_role;</ListItem>
<ListItem>
GRANT USAGE ON DATABASE DATABASE_1 TO ROLE my_monitor_role;
</ListItem>
<ListItem>
GRANT USAGE ON SCHEMA DATABASE_1.TEST_SCHEMA TO ROLE my_monitor_role;
</ListItem>
<ListItem>
GRANT SELECT ON ALL TABLES IN SCHEMA DATABASE_1.TEST_SCHEMA TO ROLE
my_monitor_role;
</ListItem>
<ListItem>CREATE USER test_user PASSWORD=&apos;***&apos;;</ListItem>
<ListItem>GRANT ROLE my_monitor_role TO USER test_user;</ListItem>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, can we just leave an empty list?

clients/admin-ui/src/flags.json Outdated Show resolved Hide resolved
src/fides/api/models/connectionconfig.py Show resolved Hide resolved
src/fides/api/models/connectionconfig.py
ConnectionType.datahub.value: SystemType.data_catalog,
ConnectionType.dynamic_erasure_email.value: SystemType.email,
ConnectionType.dynamodb.value: SystemType.database,
ConnectionType.fides.value: SystemType.database, # What's the actualy SystemType here?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leaving a comment just so we don't forget to clarify the question

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@erosselli erosselli erosselli left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andres-torres-marroquin @erosselli