Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: file conversion provider #49922

Merged
merged 2 commits into from
Jan 15, 2025
Merged

feat: file conversion provider #49922

merged 2 commits into from
Jan 15, 2025

Conversation

elzody
Copy link
Contributor

@elzody elzody commented Dec 18, 2024

Summary

This PR will introduce a file conversion API endpoint, which can be called to convert a file from one type to another. Apps can register their own conversion providers (which implement IConversionProvider), and the providers can define which MIME types they support for conversion via ConversionMimeTuples.

TODO

  • OpenAPI specs
  • Flesh out some more details around the IConversionProvider interface
    • What type should it really return? Stream, resource, File, file path?
  • What should happen when multiple apps register similar conversion providers?
  • How should we make the preferred list of conversion providers?

Checklist

@elzody elzody added enhancement 2. developing Work in progress feature: files feature: workflows php Pull requests that update Php code labels Dec 18, 2024
@elzody elzody added this to the Nextcloud 31 milestone Dec 18, 2024
@elzody elzody self-assigned this Dec 18, 2024
@elzody elzody force-pushed the feat/file-conversion-provider branch 2 times, most recently from 7e8ed84 to 3ab63cb Compare December 23, 2024 22:46
@elzody elzody force-pushed the feat/file-conversion-provider branch from b1d9670 to 1bceac6 Compare December 27, 2024 19:58
@elzody elzody force-pushed the feat/file-conversion-provider branch 2 times, most recently from 2cdd54b to e91c447 Compare January 7, 2025 20:24
@elzody elzody marked this pull request as ready for review January 9, 2025 15:37
@elzody elzody requested a review from provokateurin as a code owner January 9, 2025 15:37
@elzody elzody force-pushed the feat/file-conversion-provider branch 3 times, most recently from ac0843f to e8d3eac Compare January 13, 2025 22:37
@elzody elzody changed the title [WIP] feat: file conversion provider feat: file conversion provider Jan 13, 2025
Copy link
Member

@provokateurin provokateurin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! I only have a few smaller things to not, feel free to ignore the most nit picky ones 😅

apps/files/lib/Capabilities.php Outdated Show resolved Hide resolved
apps/files/lib/Capabilities.php Outdated Show resolved Hide resolved
lib/public/Files/Conversion/IConversionManager.php Outdated Show resolved Hide resolved
apps/files/lib/Controller/ConversionApiController.php Outdated Show resolved Hide resolved
apps/files/lib/Controller/ConversionApiController.php Outdated Show resolved Hide resolved
lib/public/Files/Conversion/ConversionMimeTuple.php Outdated Show resolved Hide resolved

// Operate in mebibytes
$fileSize = $file->getSize() / (1024 * 1024);
$threshold = $this->config->getValue('max_conversion_filesize', 100);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it make sense to also check that the threshold is a sensible value? Otherwise an admin might configure something like 100 GiB and break their instance. I'm not sure what a reasonable limit is, especially when it comes to high resolution photos or long (high resolution) videos.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a risk of breaking the instance here. If an admin configures long timeouts then it should be fine. At some point we may rather want to have background workers that do an async conversion so we don't run into timeouts (which is the main reason for adding this), but this was discussed out of scope due to the short-term scheduling with @AndyScherzinger.

lib/public/Files/Conversion/IConversionManager.php Outdated Show resolved Hide resolved
lib/public/Files/Conversion/IConversionProvider.php Outdated Show resolved Hide resolved
@skjnldsv skjnldsv mentioned this pull request Jan 14, 2025
@elzody elzody force-pushed the feat/file-conversion-provider branch from e8d3eac to 23e4606 Compare January 14, 2025 19:59
@skjnldsv
Copy link
Member

As a followup, it would be nice to add some integration tests.
Should be easy to test :

  • the api response on failure
  • the proper new png file after the api conversion endpoint have been reached

Signed-off-by: Elizabeth Danzberger <[email protected]>
@elzody elzody force-pushed the feat/file-conversion-provider branch from 1073cfd to fdfeb7f Compare January 15, 2025 21:39
Signed-off-by: Elizabeth Danzberger <[email protected]>
@elzody elzody merged commit 6ccd90b into master Jan 15, 2025
188 checks passed
@elzody elzody deleted the feat/file-conversion-provider branch January 15, 2025 23:02
@AndyScherzinger AndyScherzinger added 4. to release Ready to be released and/or waiting for tests to finish and removed 2. developing Work in progress labels Jan 15, 2025
@skjnldsv
Copy link
Member

🎉 🎉 🎉

try {
$convertedFile = $this->fileConversionManager->convert($file, $targetMimeType, $destination);
} catch (\Exception $e) {
throw new OCSException($e->getMessage());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should log the exception here and rather return a generic error message as the exception message may leak server internals. Also the trace is lost.

class ConversionMimeTuple implements JsonSerializable {
/**
* @param string $from The original MIME type of a file
* @param list<array{mime: string, name: string}> $to The desired MIME type for the file mapped to its translated name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elzody @skjnldsv I just noticed when implementing a simple pandoc provider, that we may also want to have the extension as a separate property for target formats. We list it in the name, but I assume for automatically generating a file with the new extension we should also have it directly accessible so it can be used to build the target file name on the frontend side?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow 🤔

Copy link
Member

@juliusknorr juliusknorr Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just FYI @elzody .. @skjnldsv had a quick call and noticed that the API only works with passing a destination, without we lack the ability to build the new destination filename. @skjnldsv will look into that as a follow up for the frontend.

We also had another discussion round on #49922 (comment) and decided to do a small change on the tuple drop the array to to have one tuple for each from-to mapping to be more strict on the api and not have the php array. Hope that makes sense

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4. to release Ready to be released and/or waiting for tests to finish enhancement feature: files feature: workflows php Pull requests that update Php code 🍀 2025-Spring
Projects
Status: ☑️ Done
Status: ☑️ Done
Development

Successfully merging this pull request may close these issues.

5 participants