Lots of inlined Scripts for Lexical and other component contexts causing inflated Admin Panel Size

[janus-reith]

Link to reproduction

No response

Payload Version

65

Node Version

20

Next.js Version

14 - 15 RC

Describe the Bug

I was just trying to debug why a Deployment to AWS Lambda would timeout for Admin Panel Routes while it works for API routes, and realized that the initial document is huge, almost 10 MB in my case, therefore probably hitting the 6MB Lambda limit.

At first I asssumed some incorrectly bundled dependency, however the build output shows rather acceptable bundle sizes:

Route (app)                              Size     First Load JS
┌ ○ /_not-found                          904 B          91.9 kB
├ ƒ /admin/[[...segments]]               160 B           572 kB
├ ƒ /api/[...slug]                       0 B                0 B
├ ƒ /api/graphql                         0 B                0 B
└ ƒ /api/graphql-playground              0 B                0 B
+ First Load JS shared by all            91 kB
  ├ chunks/8641-6744c8dcc8ee88a0.js      36.1 kB
  ├ chunks/d5ac473b-ff62f4efb4faca7b.js  51.7 kB
  └ other shared chunks (total)          3.15 kB

Loooking at the generated html that returned from these routes, I realized there are hundreds of inlined scripts that seem to belong to lexical. They all look like that:

 <script>
      self.__next_f.push([
        1,
        '60:[["lexical_internal_feature.link.fields.fields","$5f:0:1"],["features","$5f:1:1"]]\n61:[["lexical_internal_feature.link.fields.fields","$5f:0:1"],["features","$5f:1:1"]]\n62:[["lexical_internal_feature.link.fields.fields","$5f:0:1"],["features","$5f:1:1"]]\n',
      ]);
    </script>

That's one of the smallest, however some of them are huge like around 130 KB, which all sums up to around 10 MB on initial page load (On a Prod build).

I assume the way I use Lexical rather usual, around 28 Block Types and some nesting.

1. Something definitely seems off, there's no way 10 MB of JSON Data is needed for that. Especially since it doesnt contain the actual page data. I wonder if there's some duplication happening?

2. As a short-term relieve, I wonder if it's possible to prevent these scripts from being inlined to not hit the Lambda response limit?

Reproduction Steps

Hard to say. I will continue debugging, but don't have the impression that it's any specific block causing issues.

Adapters and Plugins

@payloadcms/richtext-lexical

[janus-reith]

I guess a start might also be skipping all these keys with $undefined as value, these are probably not necessary.

[janus-reith]

I started importing relevant parts of the richtext-lexical package into my payload app to tinker around with its getGenerateComponentMap util and see what I can improve there.

Leaving the generated componentMap out altogether had a significant effect - The initial size went from around 10 to 2,5 MB. But certainly that's not a solution since the Admin panel relies on it.

Next I tried removing undefined keys from the Component Map, which seems to work fine and shrunk the initial size from ~10 to ~7 MB.

The resulting HTML still contains more than 19999+ (That's where VSCode stops counting) cases of $undefined, so there seem to be several other places that need improvement.

[janus-reith]

Okay now thinks got confusing.
I took buildComponentMap from the UI Context the https://github.com/payloadcms/payload/blob/beta/packages/ui/src/providers/ComponentMap/buildComponentMap/index.tsx#L92 and also added logic there to purge all undefined values from componentMap before returning it.

That made the initial HTML grow to almost 20 MB, although now there's indeed only 400 occasions of "$undefined" variables left.
Sadly is hard to trace in that Output what actually changed and caused this, I wonder if it's some duplication, or if it might've affected some non-serializable data somehow.

However, I tried another approach to recursively remove undefined values which seems to have worked - only ~3k occasions of undefined left, but it didn't result in as much reduction of initial html as I had anticipated.

Looking at the leftover undefined values, I'm also starting to see that this won't really solve the core issue with these huge contexts - Many of these vars that are undefined in my case might aswell be props in use, like labels or fields props.

These contexts probably need to be split up more and need to be more atomic. A short-term relief might be to split these context from the initial html. But that wouldn't totally solve the core issue either - Since these are created dynamically, these contexts alone, even if it's all that's returned in one request, might go beyond the 6MB Lambda limit.

I wonder if Vercel is using some streaming implrementation since so far I didn't hit that limit there.

Examining this further I also found several other things in that bundle like translations for locales I'm not importing, but I'll create separate issues for that.

[janus-reith]

I also wonder if it’s most of these contexts even need to be created at runtime and can’t just be part of a static bundle created at build time. This could greatly reduce computing cost.

A reason to not have them static might be to not leak data model details to unauthorized users - That however is the case right now anyways, since all of these contexts are part of non-authorized pages aswell.

[janus-reith]

Updated the title to reflect that this issue isn’t specific to just the lexical plugin

[jmikrut]

Hey @janus-reith we're looking into this but I can't reproduce your issue. When loading up a Payload admin panel route with a relatively customized Lexical editor, my browser's inspector Network panel only shows transfers of 600-700kb of assets in total including the HTML, all scripts, images, Lexical, fonts, etc.

realized that the initial document is huge, almost 10 MB in my case

Where are you seeing 10mb? Is this in your Network panel in your browser's inspector?

Can you help us with a reproducible repo so that we can evaluate this further?

[janus-reith]

@jmikrut Ah yes that's an important detail I was missing:

When opening /admin/login in chrome, a document size of just 186 KB is shown in the network tab.
All the assets look rather small, and on the bottom of that list it says 1.2MB transferred, 21.MB Resources

Looking at the Response tab, it says: Can't be loaded: Request data was evicted from inspector cache instead of showing the html there as I would have expected.

I wonder if thats some kind of bug in the chrome dev tools.

I get the huge 10 MB file when I either run wget or curl for the /admin/login login (or any other admin route).

Can you reproduce that, or the, larger amount of data, 21 MB in my case, listed under Resources?

[janus-reith]

It might be that compression is helping there: That quick test I did was on deployment on Vercel where appareantly brotli reduced the size to 186 KB - Which is quite impressive.
I tested locally again with a prod, there I get 422 kb since only the nextjs-included gzip compression is used.

I think what's going on is, since I'm tryig to deploy with open-next, the internal nextjs compression is disabled (due to some bug they mentioned) and so on only brotli is used on Cloudfront.
Meaning the Lambda still has to serve the uncompressed 10 MB size to Cloudfront, and that's the point of failure.

[janus-reith]

I think the root cause still needs be solved, giventhat another v3 Deployment I have which so far only consist of very few collections and blocks creates less than <1 MB of uncompressed html.

[janus-reith]

I will try to create a reproducible repo later today

[lazidoca]

In my case, I use getPayload in the Server Component to use the Payload local API and it increase the first load JS from 92kB to 483kB.

[jmikrut]

@lazidoca that issue is actually known and we are working on it. See here for more info:

#6938

I will convert this to a discussion so that we can track it accordingly, but if this does indeed turn out to be an issue with Payload, I will re-open it as an issue!

[janus-reith]

Removal of the Component map in https://github.com/payloadcms/payload/releases/tag/v3.0.0-beta.79 solved this issues as expected :)