Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: use browser context to reduce cpu load on many files #138

Closed
chamabreu opened this issue Nov 3, 2022 · 3 comments · Fixed by #141
Closed

feature: use browser context to reduce cpu load on many files #138

chamabreu opened this issue Nov 3, 2022 · 3 comments · Fixed by #141
Assignees
Labels

Comments

@chamabreu
Copy link
Contributor

Problem:
If i start md2pdf with a lot of md files, I get some warnings:

> md2pdf ../mds/**/*.md
(node:40269) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
(node:40269) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:40269) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGTERM listeners added to [process]. Use emitter.setMaxListeners() to increase limit
(node:40269) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGHUP listeners added to [process]. Use emitter.setMaxListeners() to increase limit

For each file a seperate pupeteer.launch() function is called in generate-output.ts.
This registers a lot of system listeners.
const browser = await puppeteer.launch({ devtools: config.devtools, ...config.launch_options });

Solution:

Use one single browser instance and create for each convertToPdf just an browser context.
const listrContext = await browser.createIncognitoBrowserContext();

Pass this Context down the functions and use this, to create and manipulate the pages with puppeteer.

I just added this feature locally, but i can not push due to 403.

How can i push my Branch and open a PR for Review?

@simonhaenisch
Copy link
Owner

Hey thanks i've actually been wanting to do that for a while but never got around to it (i've personally never used it with multiple files 😅). Usually you'll have to fork the repo first and then you can make a PR from your fork's branch to this repo. But I'll just add you as a collaborator temporarily.

chamabreu added a commit that referenced this issue Nov 3, 2022
This feature uses puppeteer.BrowserContext for the convertion tasks.
Now, not for each task, an own browser instance is instantiated, but
only one. This one and only is used, and only browserContexts are passed
down to the convertion functions.
Reduces CPU Load, eliminates EventEmitter warnings and speed up
conversion of large amount of files by ~50%.
@chamabreu
Copy link
Contributor Author

or maybe not just temporarily, i think there will come more... #137 😄

@simonhaenisch
Copy link
Owner

Just ended up merging this now, will do a release so you can test it more easily (:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants