Fails to graph with threaded calls

[mattiaslundberg]

This seems to be related to #386 and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=758868 .

When trying to graph in parallel (different threads) it ends up in an infinite loop generating the following output:

(process:1): GLib-GObject-CRITICAL **: g_object_get_qdata: assertion 'G_IS_OBJECT (object)' failed
(process:1): GLib-GObject-CRITICAL **: g_object_replace_qdata: assertion 'G_IS_OBJECT (object)' failed
(process:1): GLib-GObject-CRITICAL **: g_object_get_qdata: assertion 'G_IS_OBJECT (object)' failed
(process:1): GLib-GObject-CRITICAL **: g_object_replace_qdata: assertion 'G_IS_OBJECT (object)' failed
(process:1): GLib-GObject-CRITICAL **: g_object_get_qdata: assertion 'G_IS_OBJECT (object)' failed
(process:1): GLib-GObject-CRITICAL **: g_object_replace_qdata: assertion 'G_IS_OBJECT (object)' failed

I have been running the following test program to reproduce the problem. Replacing the last line with thr.run() works as expected.

import rrdtool
import time
import threading

rrdtool.create(
    '/tmp/myrrd',
    "--step", "1800", "--start", '0',
    "DS:metric1:GAUGE:2000:U:U",
    "DS:metric2:GAUGE:2000:U:U",
    "RRA:AVERAGE:0.5:1:600",
    "RRA:AVERAGE:0.5:6:700",
    "RRA:AVERAGE:0.5:24:775",
    "RRA:AVERAGE:0.5:288:797",
    "RRA:MAX:0.5:1:600",
    "RRA:MAX:0.5:6:700",
    "RRA:MAX:0.5:24:775",
    "RRA:MAX:0.5:444:797"
)

t = int(time.time())
for i in xrange(30):
    rrdtool.update('/tmp/myrrd', '%s:%s:%s' % (t + (i * 1800), i, 3))


class GraphThread(threading.Thread):
    def __init__(self, i, t, *args, **kwargs):
        self.i = i
        self.t = t
        super(GraphThread, self).__init__(*args, **kwargs)

    def run(self):
        rrdtool.graph(
            "myrrd%s.png" % self.i,
            "--start", "%s" % self.t, "--end", "%s" % (self.t + (30*1800)),
            "--vertical-label=Num",
            "-w 800",
            "DEF:m1_num=/tmp/myrrd:metric1:AVERAGE",
            "DEF:m2_num=/tmp/myrrd:metric2:AVERAGE",
            "LINE1:m1_num#0000FF:metric1\r",
            "LINE2:m2_num#00FF00:metric2\r",
        )

for i in xrange(10):
    thr = GraphThread(i, t)
    thr.start()

Environment:

• Debian jessie with RRDTool 1.4.8 (debian version)
• Debian jessie with RRDTool 1.4.9
• Debian jessie with RRDTool built from master

Dockerfile used for running with master:

# docker build --rm -t test .
# docker run -it test python /testfile.py
FROM debian:jessie
MAINTAINER Mattias Lundberg <[email protected]>

ENV LANG en_US.UTF-8

RUN apt-get update && \
        apt-get install -y git tcl-dev dh-autoreconf quilt libxml2-dev libcairo2-dev libpango1.0-dev  liblua5.1-0-dev groff libdbi0-dev python-all-dev python-all-dbg gem2deb lua5.1 libpcre3-dev && \
        apt-get clean

RUN git clone https://github.com/oetiker/rrdtool-1.x.git rrdtool
RUN cd rrdtool && ./configure && make && make install && make site-python-install

ADD ./testfile.py /testfile.py

Running the same on updated Arch Linux (RRDTool 1.4.9) gives the error (non-threaded works):

(process:2449): GLib-GObject-CRITICAL **: g_object_ref: assertion 'object->ref_count > 0' failed
(process:2449): Pango-CRITICAL **: pango_font_map_get_serial: assertion 'PANGO_IS_FONT_MAP (fontmap)' failed
(process:2449): Pango-CRITICAL **: pango_font_map_get_serial: assertion 'PANGO_IS_FONT_MAP (fontmap)' failed
(process:2449): Pango-CRITICAL **: pango_font_map_get_serial: assertion 'PANGO_IS_FONT_MAP (fontmap)' failed
zsh: segmentation fault (core dumped)  python2 rrd_tool_test.py

[oetiker]

Yes, this can not work since rrdtool graph uses a persistent pango_font_map structure (for performance reasons) I guess you would have to lock access to this structure in a multi_threaded scenario ...

If you want to create appropriate patches for rrd_graph.c ... I'll be glad to review them.

[JoakimSoderberg]

So I'm looking into making this thread safe. But from reading the documentation, simply using a static fontmap like this for performance will not work for multiple threads. Your suggestion about using a lock won't work. Both locking from the python example above, or in the C code doesn't help.

https://developer.gnome.org/pango/stable/pango-Cairo-Rendering.html#pango-cairo-font-map-get-default

Note that since Pango 1.32.6, the default fontmap is per-thread. Each thread gets its own default fontmap. In this way, PangoCairo can be used safely from multiple threads.

This is what is causing the assertions above. Using an older pango then this was no the case.

I tried a solution where instead use this API call: https://developer.gnome.org/pango/stable/pango-Cairo-Rendering.html#pango-cairo-font-map-new

Then I don't get the assertions.