Debugging

Mapping a stack trace to source code

Typically a stack trace contains the function call stack, which allows you to easily see what was being executed when the stack trace was generated. But even if the stack trace contains only the addresses of the functions, it is still usually possible to figure the actual functions.

As an example, here is a stack trace showing only the addresses of the functions (and the modules were they reside):

2015-07-07 07:57:00   Fatal: MaxScale received fatal signal 11. Attempting backtrace.
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/bin/maxscale() [0x542a98]
2015-07-07 07:57:00     /lib64/libpthread.so.0(+0xf130) [0x7f105782f130]
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/modules/libMySQLBackend.so(+0x514b) [0x7f1038a1614b]
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/modules/libMySQLBackend.so(+0x318f) [0x7f1038a1418f]
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/bin/maxscale() [0x554a92]
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/bin/maxscale(poll_waitevents+0x616) [0x55435b]
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/bin/maxscale(main+0x1a10) [0x545563]
2015-07-07 07:57:00     /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f1056021af5]
2015-07-07 07:57:00     /usr/local/mariadb-maxscale/bin/maxscale() [0x5418dd]

Do as follows:

1. Download from https://mariadb.com/my_portal/download/maxscale the package that exactly corresponds to your OS and MaxScale versions.

   NOTE If you pick the wrong one, you will get a result, but it will not be correct.

2. Extract the files in the package.

3. Then, with addr2line, convert an address to file and line information.

    ...
    2015-07-07 07:57:00     /usr/local/mariadb-maxscale/bin/maxscale() [0x554a92]
    ...
   
    wikman@jwi-mariadb:bin $ addr2line -e ./maxscale 0x554a92
    /home/ec2-user/workspace/server/core/session.c:663
   

   Note that if the address is from a shared library, you need to use the address is the parenthesis (note the one in square brackets).

    ...
    2015-07-07 07:57:00     /usr/local/mariadb-maxscale/modules/libMySQLBackend.so(+0x318f) [0x7f1038a1418f]
    ...
   
    wikman@jwi-mariadb:bin $ addr2line -e ../modules/libMySQLBackend.so 0x318f
    /home/ec2-user/workspace/server/modules/protocol/mysql_backend.c:563
   

4. Then checkout the right tag in the git-repository.

    wikman@jwi-mariadb:MaxScale  (develop)$ git checkout 1.1.1
    Note: checking out '1.1.1'.
    ...
    HEAD is now at c1b956a... Fixed a non-static variable causing multiple declarations when FAKE_CODE was defined
    wikman@jwi-mariadb:MaxScale  ((1.1.1))$
   

5. Now you can check what was being done when the stack trace was generated.