NextJS API route - Unable to load plugin "sqip-plugin-primitive"

[Maxwell2022]

I'm just trying to create a simple route to test sqip which looks really really cool. I had issue with sharp that I fixed updating nextjs webpack config but now I'm facing another error:

  sqip Loading sqip-plugin-primitive +0ms
  sqip Loading sqip-plugin-data-uri +0ms
Error: Unable to load plugin "sqip-plugin-primitive". Try installing it via:

 npm install sqip-plugin-primitive

    at eval (webpack-internal:///(rsc)/./node_modules/.pnpm/[email protected]_@[email protected][email protected]_sqip-plugin-_l2j7ib26irsjvtky3rkbdcveiu/node_modules/sqip/dist/sqip.js:144:31)
    at Generator.throw (<anonymous>)
    at rejected (webpack-internal:///(rsc)/./node_modules/.pnpm/[email protected]_@[email protected][email protected]_sqip-plugin-_l2j7ib26irsjvtky3rkbdcveiu/node_modules/sqip/dist/sqip.js:50:40)

I'm using node v20.9.0 v18.18.2

Here is the app/sqip/route.ts file:

import { sqip } from 'sqip';
import axios from 'axios';
import { NextResponse } from 'next/server';

export async function GET() {
  try {
    const { data } = await axios({
      method: 'get',
      url: 'https://cdn.vox-cdn.com/thumbor/WR9hE8wvdM4hfHysXitls9_bCZI=/0x0:1192x795/1400x1400/filters:focal(596x398:597x399)/cdn.vox-cdn.com/uploads/chorus_asset/file/22312759/rickroll_4k.jpg,
      responseType: 'arraybuffer',
    });

    const result = await sqip({
      input: data,
      // Fix required "outputFileName" when using buffer
      outputFileName: 'whatever',
      plugins: [
        {
          name: 'sqip-plugin-primitive',
          options: {
            numberOfPrimitives: 8,
            mode: 0,
          },
        },
        'sqip-plugin-data-uri',
      ],
    });

    console.log(result);
    return Response.json({ ok: true });
  } catch (err) {
    console.error(err);
  }
}

package.json

{
    "sharp": "^0.32.6",
    "sqip": "1.0.0-alpha.41",
    "sqip-plugin-data-uri": "1.0.0-alpha.42",
    "sqip-plugin-primitive": "1.0.0-alpha.43",
    "sqip-plugin-svgo": "1.0.0-alpha.43"
}

and my next.config.js

module.exports = {
  webpack: (config) => {
    return {
      ...config,
      externals: {
        sharp: 'commonjs sharp',
      },
    };
  },
};

[axe312ger]

Did you try it with plain npm? Looks like you used pnpm?

The relevant line is here: https://github.com/axe312ger/sqip/blob/master/packages/sqip/src/sqip.ts#L142

Maybe output the actual error message before the throw and share it here? It might give details whats going wrong on your side?

[Maxwell2022]

I gave up spending more time trying to debug this error. It would have been a "nice to have" but not critical for what I needed. I just wanted to test this package when I found it.

But yes, I use pnpm and what I pasted was the full error message that happened when starting the dev server from memory.

Feel free to close this issue if there is not enough information for reproduction

[axe312ger]

Really weird. In the end we do a "regular await import" and if that fails, we throw. Not like this is super fancy or rare code. 🤔

I can try to reproduce, when I find time. I never used pnpm, as I am super happy as a yarn v1 fanboy :D

[tommyboylab]

I've also come across a similar issue while using a demo repository for PayloadJS 3.0. Previously I had managed to use SQIP within the media upload pipeline to generate placeholders using the primitive and svgo. Unfortunately updating to the new pipeline built on Next seems to have issues with finding the plugins within the repository when passing them within the 'onChange' hook as I used before.

I'm not sure if I could use an instantiated class of each plugin to pass to output rather than relying on 'await sqip...' to handle the plugins? I'm not super used to using SQIP outside of the provided documentation in the readme.

Easy way to reproduce would be to deploy this repository to vercel here: https://github.com/payloadcms/vercel-deploy-payload-postgres

Extend the config in Media collection with placeholder:

import type { CollectionConfig, PayloadRequest } from 'payload/types'
import { sqip } from 'sqip'
import path from 'path'


type GeneratePlaceholder = {
  req: PayloadRequest<any>,
  data: Partial<any>
}
async function generatePlaceholder({ req, data }: GeneratePlaceholder) {
  try {
    const imageFile = req.file;
    await sqip({
      input: Buffer.from(imageFile.data),
      output: path.resolve(__dirname, '../../media'),
      outputFileName: imageFile.name,
      plugins: [
       {
          name: 'sqip-plugin-primitive',
          options: {
            numberOfPrimitives: 8,
            mode: 1,
          },
        },
        'sqip-plugin-svgo'
      ]
    })

    return { ...data }

  }

  catch (err) {
    console.log('Failed to create placeholder with error', err)
    return data
  }

}

export const Media: CollectionConfig = {
  slug: 'media',
  access: {
    read: () => true,
  },
  hooks: {
    beforeChange: [async ({ req, data }) => generatePlaceholder({ req, data })],
  },
  upload: {
    staticDir: 'media',
    imageSizes: [
      {
        name: 'thumbnail',
        width: 480,
        height: undefined,
        position: 'centre',
      },
      {
        name: 'card',
        width: 768,
        height: undefined,
        position: 'centre',
      },
      {
        name: 'tablet',
        width: 1024,
        height: undefined,
        position: 'centre',
      },
    ],
    adminThumbnail: 'thumbnail',
    mimeTypes: ['image/*'],
  },
  fields: [
    {
      name: 'alt',
      type: 'text',
    },
  ],
};

Upload image and see log in console with unable to find modules and try installing as mentioned above in previous OP comment.

  FILE: {
    name: 'Screenshot 2024-05-14 105714.png',
    data: <Buffer 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 09 03 00 00 04 86 08 06 00 00 00 97 57 2a 42 00 00 00 01 73 52 47 42 00 ae ce 1c e9 00 00 00 04 ... 2792364 more byt
es>,
    mimetype: 'image/png',
    size: undefined
  }
}
Error: Cannot find module 'sqip-plugin-primitive'
    at /home/.next/server/app/(payload)/admin/[[...segments]]/page.js:25:11
    at async eval (webpack-internal:///(rsc)/./node_modules/sqip/dist/sqip.js:95:41)
    at async Promise.all (index 0) {
  code: 'MODULE_NOT_FOUND'
}
Error: Cannot find module 'sqip-plugin-svgo'
    at /home/.next/server/app/(payload)/admin/[[...segments]]/page.js:25:11
    at async eval (webpack-internal:///(rsc)/./node_modules/sqip/dist/sqip.js:95:41)
    at async Promise.all (index 1) {
  code: 'MODULE_NOT_FOUND'
}
Failed to create placeholder with error Error: Unable to load plugin "sqip-plugin-primitive". Try installing it via:

Just want to add that I've been using this plugin for years and really appreciate your work @axe312ger 🥇 might even notice my profile based on SQIP generation 🙇🏼

[circus2271]

same error after updating node from 16.20 to 20..

[axe312ger]

So strange and annoying that the module resolution is failing so badly.

I am happy about any help, I'll try to add official NextJS support later this year

For now.. no idea why for some its not working, sorry :/