Notes on choosing/optimising JRuby

[benlangfeld]

Opening this to start the discussion after another request for ideas on JRUBY_OPTS from @ggayan.

I'd like to expand this with:

• Specific explanation of why we recommend JRuby
• Explanation of each of the JRuby/JVM options

Several people have tackled this in the past, including @bklang, @JustinAiken, @polysics, @sfgeorge, @Jared-Prime and probably others. I'd appreciate input from any and all of you :)

[ggayan]

I just started using -J-Xmx2048m -J-Xms2048m -J-Xmn512m -J-XX:+CMSClassUnloadingEnabled -J-XX:+UseConcMarkSweepGC -J-XX:MaxMetaspaceSize=512M --server

I replaced the MaxPermSize -which no longer exists in java 8- with MaxMetaspaceSize

There is a good article that explains the difference: http://javaeesupportpatterns.blogspot.com/2013/02/java-8-from-permgen-to-metaspace.html

I wonder if it is a good idea to let the metaspace grow without limit? one of our adhearsion apps restarted automatically with the following error: There is insufficient memory for the Java Runtime Environment to continue

The other settings seem to be pretty standard on a jruby environment. What would be great is to have some coding tips to avoid memory leaks with jruby in an adhearsion app.

[Jared-Prime]

/cc @runningferret

[JustinAiken]

Most of what @ggayan posted is the same things I've found:

• --server is definitely good
• Bump up the memory: -J-Xmx2048m -J-Xms2048m -J-Xmn512m
• -J-XX:+UseConcMarkSweepGC was a less painful GC then the default
• Bumping up the perm size is a must: -J-XX:MaxPermSize=512M
• I can't remember what -J-XX:+CMSClassUnloadingEnabled did, but I put it in..

Other than command line stuff, beware of TLS connections over XMPP (never got them to work), and be sure to stress test the HTTP Client library you use - some of them have issues at load not seen with C-ruby.

And unless you hate yourself, just use MRI locally, and have a QA place to catch jruby-specific bugs. 😛

[benlangfeld]

On 27 October 2014 16:02, JustinAiken [email protected] wrote:

Most of what @ggayan https://github.com/ggayan posted is the same
things I've found:

• --server is definitely good
• Bump up the memory: -J-Xmx2048m -J-Xms2048m -J-Xmn512m
• -J-XX:+UseConcMarkSweepGC was a less painful GC then the default
• Bumping up the perm size is a must: -J-XX:MaxPermSize=512M
• I can't remember what -J-XX:+CMSClassUnloadingEnabled did, but I put
  it in..

Other than command line stuff, beware of TLS connections over XMPP (never
got them to work)

The problem there is EventMachine, which doesn't implement StartTLS on
JRuby.

, and be sure to stress test the HTTP Client library you use - some of
them have issues at load not seen with C-ruby.

And unless you hate yourself, just use MRI locally, and have a QA place to
catch jruby-specific bugs. [image: 😛]

—
Reply to this email directly or view it on GitHub
#32 (comment)
.

[runningferret]

-J-XX:+CMSClassUnloadingEnabled, along with -J-XX:+UseConcMarkSweepGC, allows for PermGen to be garbage collected. It's probably not strictly necessary with a 512m permgen size, though it is probably useful, and likely not harmful. (see http://stackoverflow.com/questions/3717937/cmspermgensweepingenabled-vs-cmsclassunloadingenabled/3720052#3720052 for a bit more info)

[ggayan]

I am also using -J-XX:ReservedCodeCacheSize=256M -J-XX:+UseCodeCacheFlushing now, because I ran into the same issue described here with one of our apps.

[sfgeorge]

You've all mentioned some excellent options for tuning, so I won't be repeat what's been said there.

The following settings that are generally considered for development environments, but I find them useful in Production as well

• -J-XX:+PrintCommandLineFlags - Prints the actual settings used by the JVM, which may be slightly different than what you specified in $JRUBY_OPTS. This also prints to stdout every time that a Java process is spawned - useful to noting and trying to reduce the number of times that Bundler and ahn fork a separate JRuby instance in order to boot up.
• -J-Xloggc:/var/log/java-ahn-gc.log - Writes garbage collection events to /var/log/java-ahn-gc.log, which can be easily viewed with GCViewer or a similar Java GC inspection tool. If you are hitting excessive <Adhearsion::Call::CommandTimeout>, <Celluloid::DeadActorError> or <Adhearsion::PunchblockPlugin::Initializer: Event received for inactive call> errors in Production, it might be because stop-the-world "Full GC" garbage collection events are too slow or too frequent for the system to be able to keep up with Asterisk events. This can be an indicator that you need to tune heap and GC settings via JRUBY_OPTS, or that you may need to scale-up your hardware.

Additionally, it's of course good to point people at the excellent Performance Tuning guide that the JRuby Team whipped together as well. An extensive read that is well worth it.

[bklang]

Some great advice here! Anyone feel like massaging it into a PR?