Extend collections to include a soft delete feature for restoring deleting documents

[DanRibbens]

In other frameworks or ORMs there is usually a concept for enabling soft deleting an entity where a request to delete does not remove it from the database, but only marks it as deleted. This is useful for implementing trash can like functionality where documents can easily be restored or read later.

Typically this is done by adding a deletedAt timestamp to the document that can be nullable or undefined for non-deleted documents.

This would fall under the opt-in config pattern so that each collection could have a flag for retaining documents.
The default behavior when fetching (find, findByID and bulk update/delete) would not return documents that have deletedAt set unless a flag is specifically sent to include deleted, similar to how drafts: true currently works.

New endpoints could be added to restore documents, or we could simply rely on PATCH to set the deletedAt back to null.

This could be done at the database and API layer only and then in another feature we can add features to the UI to make it easier to view deleted documents and restore them.

#1968

[dkirchhof]

Just wanted to implement a custom "soft delete" feature by overriding the delete endpoints. Unfortunately the frontend will only show custom notifications on errors. I would like to show something like "Image was moved to trash" instead of "Image was deleted successfully".

So a start for custom solutions would be to change the response handler in the DeleteDocument component.

[BrianJM]

Unfortunately the frontend will only show custom notifications on errors. I would like to show something like "Image was moved to trash" instead of "Image was deleted successfully".

Check out the docs on i18n. Here's an example in (English):

i18n: {
    fallbackLng: 'en', // default
    debug: false, // default
    resources: {
      en: {
        // override existing translation keys
        // https://github.com/payloadcms/payload/blob/main/packages/payload/src/translations/en.json
        general: {
          titleDeleted: '{{label}} \"{{title}}\" successfully moved to trash.',
        },
      },
    },
  },

[dkirchhof]

@DanRibbens if I have a custom query param, which should filter for "trashed" items. But it will be removed somewhere in the list component where the fetch request will be triggered.

So, if I use something like this: where[or][0][and][0][deleteAt][equals]=null or where[or][0][and][0][deleteAt][notEquals]=null I will get an error from the query value sanitizer.

Adding a virtual field with type checkbox which is true, if the deleteAt field has a value, would break the filter as well (where[or][0][and][0][trash][equals]=true). So in a beforeOperation hook I have to remove this filter again.

Adding a "normal" checkbox field, which is in sync with the deleteAt field could work. But I don't like it.

Is it possible to search for date fields with null values?

[DanRibbens]

I can fix the query validation error. It won't be today, probably Monday.

[dkirchhof]

Sounds great :)
I'm out for 2 weeks, so I'm not pressed for time.

[dkirchhof]

I discarded my first idea and tried another approach.

1. For each collection we wan't to enable the soft delete functionality, we will create a trash collection
   1. Same fields as the original one
   2. Same slug with "_trash" suffix
   3. deleteAt field
2. When deleting documents of the original collection, they will be moved to the according trash collection. The deleteAt field will be populated with the date when this doc should be deleted (configurable)
3. Docs in the trash collection can be restored. Restoring means: Move them back to the original collection.
4. A cron job will run once a day and delete all docs with dateAt <= now of all trash collections.

[dkirchhof]

Don't know how it would handle versions since we use it only for collections without them

[cbratschi]

We implemented our own solution to keep track of deletions, otherwise users can remove content without leaving any traces and all revisions are gone. We store all revisions without autosaved ones in the same document. This works fine until the maximum size of a MongoDB document is reached. But is fine for our use case. Otherwise a second collection containing the revisions could be used.

Our stored data looks like this:

                                data: {
                                    relation: collection.slug,
                                    id: id.toString(),
                                    title,
                                    document: doc,
                                    //Note: skipping autosaved versions
                                    versions: versions.filter(ver => !ver.autosave).map(ver => <Exclude<Trash['versions'], null | undefined>[0]>{
                                        version: ver.version,
                                        createdAt: ver.createdAt,
                                        updatedAt: ver.updatedAt
                                    }),
                                    //Note: req.user failed
                                    user: req.user.id
                                }

We store the user who deleted the document, the document itself in a trash collection, all revisions and the title of the document.