syslog customization & logging leaves a bit to be desired

[he32]

With reference to issue #1121, this specific issue concentrates on the logging done by exabgp, perhaps first and foremost when it is run as a daemon.

It does appear that the following holds true:

• All the logging done via syslog by exabgp appears to be done with facility=user and priority=debug. I would have expected the log level for different log messages to vary across the exabgp code base, based at least partly on their "chattiness", and at least that some of them had a level of "notice" or at least "info".
• exabgp.env does not appear to offer a way to set the facility to be used for syslog messages (I would argue that it should).
• to my perhaps naive mind, exabgp falls more into the "system daemons" category than the "random user-level messages" category, so it could be argued that the default facility=user is wrong.
• The man page of exabgp does not mention that syslog messages are sent with facility=user. This is one of the important points of reference documentation: clearly document the external interface to your program. Yes, this includes "where do you send your syslog messages and how are they tagged".
• The default syslog.conf file on NetBSD does not handle facility=user log messages. Ref. the previous point.
• The healtcheck exabgp module's syslog messages are visible in /var/log/messages with a default NetBSD syslog.conf file, and it logs the "send announces for UP state to ExaBGP" messages with facility=daemon and priority=info. This contributed to my confusion as to where the log messages from exabgp itself ended up, as I would have expected "the same place".

How did I find this out? Looking at the system call trace, of course:

 15588  15588 python3.10 GIO   fd 3 wrote 64 bytes
       "<15>Created PIDfile /var/run/exabgp/exabgp.pid with value 15588\0"
...
 15588  15588 python3.10 GIO   fd 3 wrote 38 bytes
       "<15>initialising connection to peer-2\0"
...
 15588  15588 python3.10 GIO   fd 3 wrote 38 bytes
       "<15>initialising connection to peer-1\0"

etc. while the healthcheck module which runs as a separate process logs

 15063  15063 python3.10 GIO   fd 3 wrote 71 bytes
       "<30>healthcheck-resolver[15063]: send announces for UP state to ExaBGP\
        \0"

15 = 8+7, and 8 is 1x8 and 1 is "user", and 7 is "debug".
30 = 24+6, and 24 is 3x8 and 3 is "daemon", and 6 is "info".

[he32]

Oh, yes, I should have said: this testing & observations is done with ExaBGP version 4.2.21.

[he32]

And ... in the current exabgp code off github (substantially different from 4.2.21, at least in the logging department), I find in src/exabgp/logger/handler.py:

def _syslog(**kwargs):
    formating = kwargs.get('format', SHORT)
    handler = handlers.SysLogHandler(
        address=kwargs.get('address', '/dev/log'),
        facility=kwargs.get('facility', 'syslog'),
    )

Does that mean that the default facility is now syslog? That doesn't match well with the comment in <sys/syslog.h>:

#define LOG_SYSLOG      (5<<3)  /* messages generated internally by syslogd */

[thomas-mangin]

I am happy to change the way syslog works to be clearer and follow the expectation of users.

I believe the team uses stdout myself and it has been years since I have been a UNIX admin so I am a bit rusty.

If you have suggestions, I am happy to implement them.