Subprocess to FFI: Memory, Performance, and Why You Shouldn't Shell Out

Code, references, and slides.

additional materials

Here's some additional reading you might find useful. Not too many people write about this stuff online.

• fork() in python: http://www.petercollingridge.co.uk/blog/running-multiple-processes-python
• https://github.com/oplatek/cffi-python-playground
• http://eli.thegreenplace.net/2013/03/09/python-ffi-with-ctypes-and-cffi/
• http://cffi.readthedocs.org/
• https://groups.google.com/forum/#!forum/python-cffi
• http://me.veekun.com/blog/2013/09/13/cython-versus-cffi/
• The Linux Programming Interface by Michael Kerrisk
• http://www.win.tue.nl/~aeb/linux/lk/lk-9.html # about kernel memory (this page is a bit dated but still has some useful background material)
• http://stackoverflow.com/questions/15443246/understanding-python-fork-and-memory-allocation-errors
• http://stackoverflow.com/questions/1367373/python-subprocess-popen-oserror-errno-12-cannot-allocate-memory/13329386#13329386
• http://stackoverflow.com/questions/12924048/fork-memory-allocation-behavior
• http://stackoverflow.com/questions/1216794/python-subprocess-popen-erroring-with-oserror-errno-12-cannot-allocate-memory
• http://opsmonkey.blogspot.com/2007/01/linux-memory-overcommit.html
• https://www.kernel.org/doc/
• http://www.quora.com/What-are-the-disadvantages-of-disabling-memory-overcommit-in-Linux

THANKS TO

Folks that helped me out with this talk, you rock:

Michael Grinich, Kavya Joshi, Erik Osheim, Hy Carrell