Add Organizational event logging feature #2868

BlackDex · 2022-10-26T17:50:02Z

This PR adds event/audit logging support for organizations.
By default this feature is disabled, since it does log a lot and adds
extra database transactions.

All events are touched except a few, since we do not support those
features (yet), like SSO for example.

This feature is tested with multiple clients and all database types.

Fixes #229

Checklist:

User ✔️

Cipher ✔️

Collection ✔️

CollectionCreated
CollectionUpdated
CollectionDeleted

Group ✔️

GroupCreated
GroupUpdated
GroupDeleted

Org User ✔️

OrganizationUserInvited
OrganizationUserConfirmed
OrganizationUserUpdated
OrganizationUserRemoved
OrganizationUserUpdatedGroups
~~OrganizationUserUnlinkedSso~~
~~OrganizationUserResetPasswordEnroll~~
~~OrganizationUserResetPasswordWithdraw~~
~~OrganizationUserAdminResetPassword~~
~~OrganizationUserResetSsoLink~~
~~OrganizationUserFirstSsoLogin~~
OrganizationUserRevoked
OrganizationUserRestored

Organization ✔️

OrganizationUpdated
OrganizationPurgedVault
OrganizationClientExportedVault
~~OrganizationVaultAccessed~~
~~OrganizationEnabledSso~~
~~OrganizationDisabledSso~~
~~OrganizationEnabledKeyConnector~~
~~OrganizationDisabledKeyConnector~~
~~OrganizationSponsorshipsSynced~~
PolicyUpdated

Misc:

Log Admin Interface user update/delete actions.
Config option to enable/disable this feature
Test MySQL and PostgreSQL
Cronjob to clean event logs with a retention of x days
Test all event types and links the web-vault generate
Test other clients, Desktop, Mobile, etc..
Use the continuation token and load a max of 30 items a time?

dani-garcia

Overall looks like a good start, I think we should provide a way to disable event logging for those who aren't interested in the functionality, though.

Other than that seems like we're going to have to add a lot of event logging calls for all the available EventTypes (all of them except UserClient* and CipherClient* which are collected and sent from the client it seems)

src/api/core/events.rs

src/api/identity.rs

src/db/models/event.rs

BlackDex · 2022-11-09T20:39:12Z

Ok, I have made some larger updates now. It would be really cool if some other longtime contributors also check this PR and give there comments where needed 😄 (hinting @dani-garcia & @jjlin mostly, but others are also welcome, the more the better).

Some items you can disregard upfront (Since i will remove these from this PR):

Library updates
Github Workflow adjustments
admin.rs clippy fixes

I also want to check if i can just have one log_event function instead of the current 3 (or maybe at least 2, log_event and log_user_event). But if you have any other thoughts about that, please let me know.

The items not yet checked above are still on the ToDo list, all other items should be there (and if you think otherwise, please point them out).

Thanks in advance!

BlackDex · 2022-11-18T17:00:50Z

Checking Bitwarden and the events (https://bitwarden.com/help/event-logs/#what-are-event-logs) it looks like they keep the events indefinitely. I still tend to add a job to clean-up. But maybe we need to disable it by default?

BlackDex · 2022-11-20T18:24:56Z

I Think this PR is ready for review.
I might make some small changes maybe, but this PR now is a fully working and tested feature.
All database types have been tested. Android, Linux Desktop, CLI and Browser interactions are tested.

Please provide your comments.

ilsalgo · 2022-11-23T09:30:26Z

Can I test this feature on my docker installation? How?
Thanks

BlackDex · 2022-11-23T09:46:27Z

Can I test this feature on my docker installation? How? Thanks

There is no image yet for this. If you really want I can build an image. But I would at least recommend to create a backup of the database before starting it.

If you want, let me know which image type, arch (x86_64, armv7) and base (Debian or Alpine).

ilsalgo · 2022-11-23T10:19:23Z

I have a fresh installation for my organisation and this feature would be great for me.
I have a docker installation on Arm64 device with Armbian aarch64 based on Ubuntu

BlackDex · 2022-11-23T12:29:12Z

I have a fresh installation for my organisation and this feature would be great for me. I have a docker installation on Arm64 device with Armbian aarch64 based on Ubuntu

If you dare, I have build the images locally and pushed them to my Docker Hub.
Just keep in mind that you need to switch the image once this PR has been merged :), i will probably not keep those images there for months.

See: https://hub.docker.com/r/blackdex/vaultwarden/tags
For Debian based: docker pull blackdex/vaultwarden:arm64
For Alpine based: docker pull blackdex/vaultwarden:arm64-alpine

ilsalgo · 2022-11-23T13:27:07Z

I risk it with enthusiasm, I try it right away.
You are a legend, for now thanks. 💯

ilsalgo · 2022-11-23T13:55:54Z

I have pull a new image and recreate the container but the log list dont appears, what am I doing wrong?

BlackDex · 2022-11-23T13:58:14Z

I have pull a new image and recreate the container but the log list dont appears, what am I doing wrong?

You need to enable it. It is disabled by default.

vaultwarden/.env.template

Lines 84 to 87 in 1e20761

    
           ## Controls whether event logging is enabled for organizations 
        
           ## This setting applies to organizations. 
        
           ## Default this is disabled. Also check the EVENT_CLEANUP_SCHEDULE and EVENTS_DAYS_RETAIN settings. 
        
           # ORG_EVENTS_ENABLED=false

ilsalgo · 2022-11-23T14:15:09Z

You are a man!! Works great! Thank you so much 👍

TwoTwenty · 2022-11-24T18:43:03Z

I'll test this as well with a clone of my production setup.

This PR adds event/audit logging support for organizations. By default this feature is disabled, since it does log a lot and adds extra database transactions. All events are touched except a few, since we do not support those features (yet), like SSO for example. This feature is tested with multiple clients and all database types. Fixes dani-garcia#229

ilsalgo · 2022-11-28T05:35:43Z

Hi, can I go back to the "latest" version without risk? Thank you

BlackDex · 2022-11-28T07:04:30Z

Hi, can I go back to the "latest" version without risk? Thank you

Yes you can, but as always, create a backup.

h00bi · 2023-01-02T13:14:46Z

ORG_EVENTS_ENABLED=true seems not to work fo me.
I run vaultwarden since 2020 with no env file. My config is completely based on the admin panel.
I added ORG_EVENTS_ENABLED with value true to ENV section in Portainer and re-deployed 1.27.0
Cannot find the event log in my organization.

BlackDex · 2023-01-02T13:49:12Z

ORG_EVENTS_ENABLED=true seems not to work fo me. I run vaultwarden since 2020 with no env file. My config is completely based on the admin panel. I added ORG_EVENTS_ENABLED with value true to ENV section in Portainer and re-deployed 1.27.0 Cannot find the event log in my organization.

It should work just fine. Those are read-only config items, which should work, and are not stored within the config.json.
So, setting that environment variable and stop/start should work just fine.

Check the admin setting under the read-only section.

BlackDex force-pushed the impl-events branch 2 times, most recently from a0e4ef6 to 090fb86 Compare October 26, 2022 20:14

dani-garcia reviewed Oct 26, 2022

View reviewed changes

src/api/core/events.rs Outdated Show resolved Hide resolved

src/api/identity.rs Outdated Show resolved Hide resolved

src/db/models/event.rs Outdated Show resolved Hide resolved

BlackDex force-pushed the impl-events branch 6 times, most recently from 57ddaf1 to 22b5dad Compare October 27, 2022 19:57

BlackDex force-pushed the impl-events branch 6 times, most recently from 838fc63 to dd6faa2 Compare November 9, 2022 20:27

BlackDex force-pushed the impl-events branch 7 times, most recently from 3ae78be to 963dbb7 Compare November 13, 2022 17:48

BlackDex mentioned this pull request Nov 14, 2022

[META] Feature Requests #246

Open

61 tasks

BlackDex force-pushed the impl-events branch 2 times, most recently from f3a8363 to 88167c4 Compare November 18, 2022 14:45

BlackDex force-pushed the impl-events branch 2 times, most recently from 3cd40a4 to 9aa6159 Compare November 20, 2022 16:58

BlackDex closed this Nov 20, 2022

BlackDex force-pushed the impl-events branch from 9aa6159 to bdc1cd1 Compare November 20, 2022 18:04

BlackDex changed the title ~~WIP: Event logging~~ Add Organizational event logging feature Nov 20, 2022

BlackDex reopened this Nov 20, 2022

BlackDex force-pushed the impl-events branch from 8ef4ce0 to 1e20761 Compare November 20, 2022 18:20

BlackDex marked this pull request as ready for review November 20, 2022 18:22

BlackDex force-pushed the impl-events branch from 1e20761 to 8305ea9 Compare November 27, 2022 09:53

dani-garcia approved these changes Nov 27, 2022

View reviewed changes

BlackDex force-pushed the impl-events branch from 8305ea9 to 2ea9b66 Compare November 27, 2022 22:37

dani-garcia merged commit ffa2044 into dani-garcia:main Dec 1, 2022

BlackDex deleted the impl-events branch December 2, 2022 09:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Organizational event logging feature #2868

Add Organizational event logging feature #2868

BlackDex commented Oct 26, 2022 •

edited

Loading

dani-garcia left a comment

BlackDex commented Nov 9, 2022 •

edited

Loading

BlackDex commented Nov 18, 2022

BlackDex commented Nov 20, 2022

ilsalgo commented Nov 23, 2022

BlackDex commented Nov 23, 2022

ilsalgo commented Nov 23, 2022

BlackDex commented Nov 23, 2022 •

edited

Loading

ilsalgo commented Nov 23, 2022

ilsalgo commented Nov 23, 2022

BlackDex commented Nov 23, 2022

ilsalgo commented Nov 23, 2022

TwoTwenty commented Nov 24, 2022

ilsalgo commented Nov 28, 2022

BlackDex commented Nov 28, 2022

h00bi commented Jan 2, 2023

BlackDex commented Jan 2, 2023

Add Organizational event logging feature #2868

Add Organizational event logging feature #2868

Conversation

BlackDex commented Oct 26, 2022 • edited Loading

dani-garcia left a comment

Choose a reason for hiding this comment

BlackDex commented Nov 9, 2022 • edited Loading

BlackDex commented Nov 18, 2022

BlackDex commented Nov 20, 2022

ilsalgo commented Nov 23, 2022

BlackDex commented Nov 23, 2022

ilsalgo commented Nov 23, 2022

BlackDex commented Nov 23, 2022 • edited Loading

ilsalgo commented Nov 23, 2022

ilsalgo commented Nov 23, 2022

BlackDex commented Nov 23, 2022

ilsalgo commented Nov 23, 2022

TwoTwenty commented Nov 24, 2022

ilsalgo commented Nov 28, 2022

BlackDex commented Nov 28, 2022

h00bi commented Jan 2, 2023

BlackDex commented Jan 2, 2023

BlackDex commented Oct 26, 2022 •

edited

Loading

BlackDex commented Nov 9, 2022 •

edited

Loading

BlackDex commented Nov 23, 2022 •

edited

Loading