Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Membrane, after letting it run in a container for a while #562

Open
rrayst opened this issue Jun 1, 2023 · 1 comment
Open

Membrane, after letting it run in a container for a while #562

rrayst opened this issue Jun 1, 2023 · 1 comment
Milestone

Comments

@rrayst
Copy link
Contributor

rrayst commented Jun 1, 2023

New HTTP requests are not handled anymore, output:

[770158.591s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
[770158.591s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Thread-29598"
Exception in thread "RouterThread" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
	at java.base/java.lang.Thread.start0(Native Method)
	at java.base/java.lang.Thread.start(Unknown Source)
	at java.naming/com.sun.jndi.ldap.Connection.<init>(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapClient.<init>(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapClient.getInstance(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtx.connect(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtx.<init>(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxFromUrl(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(Unknown Source)
	at java.naming/com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(Unknown Source)
	at java.naming/javax.naming.spi.NamingManager.getInitialContext(Unknown Source)
	at java.naming/javax.naming.InitialContext.getDefaultInitCtx(Unknown Source)
	at java.naming/javax.naming.InitialContext.init(Unknown Source)
	at java.naming/javax.naming.InitialContext.<init>(Unknown Source)
	at java.naming/javax.naming.directory.InitialDirContext.<init>(Unknown Source)
	at com.predic8.membrane.core.interceptor.authentication.session.LDAPUserDataProvider.auth(LDAPUserDataProvider.java:212)
	at com.predic8.membrane.core.interceptor.authentication.session.LDAPUserDataProvider.verify(LDAPUserDataProvider.java:287)
	at com.predic8.membrane.core.interceptor.authentication.session.CachingUserDataProvider.verify(CachingUserDataProvider.java:53)
	at com.predic8.membrane.core.interceptor.authentication.BasicAuthenticationInterceptor.validUser(BasicAuthenticationInterceptor.java:61)
	at com.predic8.membrane.core.interceptor.authentication.BasicAuthenticationInterceptor.handleRequest(BasicAuthenticationInterceptor.java:52)
	at com.predic8.membrane.core.interceptor.InterceptorFlowController.invokeRequestHandlers(InterceptorFlowController.java:110)
	at com.predic8.membrane.core.interceptor.flow.ConditionalInterceptor.handleRequest(ConditionalInterceptor.java:85)
	at com.predic8.membrane.core.interceptor.InterceptorFlowController.invokeRequestHandlers(InterceptorFlowController.java:110)
	at com.predic8.membrane.core.interceptor.UserFeatureInterceptor.handleRequest(UserFeatureInterceptor.java:47)
	at com.predic8.membrane.core.interceptor.InterceptorFlowController.invokeRequestHandlers(InterceptorFlowController.java:110)
	at com.predic8.membrane.core.interceptor.InterceptorFlowController.invokeHandlers(InterceptorFlowController.java:75)
	at com.predic8.membrane.core.transport.http.AbstractHttpHandler.invokeHandlers(AbstractHttpHandler.java:61)
	at com.predic8.membrane.core.transport.http.HttpServerHandler.process(HttpServerHandler.java:223)
	at com.predic8.membrane.core.transport.http.HttpServerHandler.run(HttpServerHandler.java:127)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)```
@predic8
Copy link
Member

predic8 commented Aug 31, 2023

Ulimit is enough:

root@daa3658a483e:/opt/membrane# ulimit -a
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) 0
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 30977
max locked memory           (kbytes, -l) 82000
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1048576
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) unlimited
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited

And xmx xms are set according to your container limits to 1/4 of container memory:

❯ docker run --rm -ti --memory=1g temurin
openjdk version "20.0.2" 2023-07-18
OpenJDK Runtime Environment Temurin-20.0.2+9 (build 20.0.2+9)
OpenJDK 64-Bit Server VM Temurin-20.0.2+9 (build 20.0.2+9, mixed mode, sharing)
xms = 16
xmx = 247

@predic8 predic8 modified the milestones: 5.2.0, 5.3 Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants