Feature/java agent (#7)

* Refactor code to support different providers

* One file left behind

* Removed unused libraries and helper classes

* Refactor again, now implementing different clients instead of pollers

* MBeans client working

* Remove some redundant files and bugfix java agent

* WIP

* Fixed a few bugs

* Configuration and flatten refactoring

* Shading, configurations, fixes  - working version.

* Added changelog

* PR changes

* More PR fixes

* Change test application.conf to match the new format

* Final - updated log4j

README.md

@@ -2,7 +2,9 @@
 
 jmx2graphite is a one liner tool for polling JXM and writes into Graphite (every 30 seconds by default). You install & run it on every machine you want to poll its JMX.
 
-Currently it only reads JMX from a jolokia agent running on a JVM, since exposing JMX is the simplest and easiest through Jolokia agent (1 liner - see below).
+Currently it has two flavors:
+1. Docker image which reads JMX from a jolokia agent running on a JVM, since exposing JMX is the simplest and easiest through Jolokia agent (1 liner - see below).
+2. Run as a java agent, and get metrics directly from MBean Platform
 
 The reporting to graphite is done through the Pickle protocol, hence by default port 2004, since it's more efficient.
 
@@ -61,15 +63,22 @@ there are two ways to specify the host in the jolokia URL so this URL will be re
 5. Unzip it in any directory you'd like
 6. Move the directory jmx2graphite from ```opt/jmxgraphite``` to a location which fits you. Normally you would move it to ```/opt```
 7. Edit the configuration file at ```jmx2graphite/conf/application.conf```: The mandatory items are:
-   1. service/jolokiaUrl - Fill in the full URL to the JVM running Jolokia (It exposes your JMX as a REST service, normally under port 8778).
+   1. service/jolokiaFullUrl - Fill in the full URL to the JVM running Jolokia (It exposes your JMX as a REST service, normally under port 8778).
    2. service/name - The role name of the service.
    3. graphite/hostname  - Graphite host name the metrics will be sent to
 8. cd ```jmx2graphite/bin```
 9. run ```./jmx2graphite```. This runs interactively, so pressing ctrl-c will make it stop. 
 10. If you wish to run this as a service you need to create a service wrapper for it. Any pull requests for making it are welcome! If it's possible running it as docker making it simpler.
 
 
-
+## As Java Agent
+This lib can also get the metrics from MBean Platform instead of jolokia. In order to do so, we need to run inside the JVM.
+- First, get the java agent jar from the releases page
+- Modify your app JVM arguments and add the following:  java -javaagent:/path/to/jmx2graphite-1.1.0-javaagent.jar=GRAPHITE_HOSTNAME=graphite.host;SERVICE_NAME=Myservice ...
+- The parameters are key-value pairs, in the format of key=value;key=value;...
+- The parameters names and functions are exactly as described in Environment Variables section. (Except no need to specify JOLOKIA_URL of course)
+- The javaagent.jar is an "Uber-Jar" that shades all of its dependencies inside, to prevent class collisions
+- For example: java -javaagent:/opt/jmx2graphite-1.1.0-javaagent.jar=GRAPHITE_HOSTNAME=graphite.example.com;SERVICE_NAME=PROD.MyAwesomeCategory example.jar
 
 
 # How to expose JMX Metrics using Jolokia Agent
@@ -211,6 +220,10 @@ We welcome any contribution! You can help in the following way:
 ./gradlew build
 docker build -t logzio/jmx2graphite .
 ```
+Build Java Agent
+```
+./gradlew build javaAgent
+```
 
 # Deploy
 ```
@@ -220,7 +233,10 @@ docker push logzio/jmx2graphite
 
 
 # Changelog
-
+- v1.1.0
+  - Major refactoring - jmx2graphite now comes in two flavors: standalone using docker as it was in 1.0.x, and as a Java Agent running alongside you app. This is useful if your app is running inside Docker on Mesos and coupling it with another container just to read its metrics contradicts the Mesos paradigm.
+  - Added java agent capabilities, through MBeans Platform
+  - Changed logback to log4j
 - v1.0.8
   - First migration step to Kotlin language
 - v1.0.7

build.gradle

@@ -3,28 +3,30 @@ description 'JMX 2 Graphite. Dumps JMX beans into Graphite'
 apply plugin:'java'
 apply plugin: 'kotlin'
 apply plugin:'application'
+apply plugin: "com.github.johnrengelman.shadow"
 
-mainClassName = 'io.logz.jmx2graphite.Jmx2Graphite'
+mainClassName = 'io.logz.jmx2graphite.Jmx2GraphiteJolokia'
 
 group = "io.logz"
-version = "1.0.8"
+version = "1.1.0"
 
 repositories {
     mavenCentral()
 }
 
 dependencies {
     compile 'org.slf4j:slf4j-api:1.7.12'
-    compile 'ch.qos.logback:logback-core:1.1.3'
-    compile 'ch.qos.logback:logback-classic:1.1.3'
+    compile group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.6.1'
+    compile group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.6.1'
+    compile group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: '2.6.1'
     compile 'org.apache.httpcomponents:fluent-hc:4.5.1'
     compile 'com.fasterxml.jackson.core:jackson-databind:2.6.3'
     compile 'io.dropwizard.metrics:metrics-graphite:3.1.2'
-    compile 'org.quartz-scheduler:quartz:2.2.2'
-    compile 'com.google.inject:guice:4.0'
     compile 'com.typesafe:config:1.3.0'
     compile 'com.google.guava:guava:18.0'
     compile 'com.nurkiewicz.asyncretry:asyncretry:0.0.7'
+    compile group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.7.4'
+    compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.7.4'
     testCompile 'io.netty:netty-all:4.0.33.Final'
     testCompile 'junit:junit:4.12'
     compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
@@ -41,6 +43,28 @@ distributions {
     }
 }
 
+jar {
+    manifest {
+        attributes ("Main-Class" : "io.logz.jmx2graphite.Jmx2GraphiteJolokia",
+                    "Premain-Class": "io.logz.jmx2graphite.Jmx2GraphiteJavaAgent")
+    }
+}
+
+task javaAgent(dependsOn: [shadowJar]) {
+}
+
+shadowJar {
+    manifest.from jar.manifest
+    classifier = 'javaagent'
+    relocate 'org.apache', 'io.logz.org.apache'
+    relocate 'com.fasterxml.jackson', 'io.logz.com.fasterxml.jackson'
+    relocate 'io.dropwizard.metrics', 'io.logz.ch.io.dropwizard.metrics'
+    relocate 'com.typesafe', 'io.logz.com.typesafe'
+    relocate 'com.google', 'io.logz.com.google'
+    relocate 'com.nurkiewicz.asyncretry', 'io.logz.com.nurkiewicz.asyncretry'
+    relocate 'org.jetbrains.kotlin', 'io.logz.org.jetbrains.kotlin'
+}
+
 /*
    Hack to get the conf directory into the classpath
  */
@@ -61,9 +85,13 @@ buildscript {
     ext.kotlin_version = '1.0.1-2'
     repositories {
         mavenCentral()
+        maven {
+            url "https://plugins.gradle.org/m2/"
+        }
     }
     dependencies {
         classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
+        classpath "com.github.jengelman.gradle.plugins:shadow:1.2.3"
     }
 }
 sourceSets {

src/dist/conf/application.conf

@@ -1,22 +1,25 @@
 # The service to poll the JMX metrics from
 service {
-  # Full URL to jolokia.
-  # Example: http://172.31.63.154:11001/jolokia/
-  jolokiaUrl = ${JOLOKIA_URL}
 
   # Metrics prefix when sent to Graphite is: [service-name].[service-host].
   # This makes it easier to browse metrics by service name (i.e. Log Receiver) and then by machine host/IP
 
   # Service name
   name = ${SERVICE_NAME}
 
-  # Hostname. If not supplied, taking hostname from jolokiaUrl
+  # Hostname. If not supplied, taking hostname from jolokiaFullUrl
   host = ${?SERVICE_HOST}
+
+  poller {
+     jolokia {
+        jolokiaFullUrl = ${JOLOKIA_URL}
+     }
+  }
 }
 
 graphite {
   hostname = ${GRAPHITE_HOST}
   port = ${?GRAPHITE_PORT}
 }
 
-intervalInSeconds = ${?INTERVAL_IN_SEC}
+metricsPollingIntervalInSeconds = ${?INTERVAL_IN_SEC}

src/dist/conf/log4j2.xml

@@ -0,0 +1,30 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Configuration monitorInterval="30">
+<Appenders>
+    <RollingFile name="default.file" fileName="/var/log/jmx2graphite/jmx2graphite.log" append="true"
+                 filePattern="/var/log/jmx2graphite/jmx2graphite.%i.log.gz">
+        <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n"/>
+        <Policies>
+            <SizeBasedTriggeringPolicy size="10 MB"/>
+        </Policies>
+        <DefaultRolloverStrategy max="7"/>
+    </RollingFile>
+
+    <Console name="default.console" target="SYSTEM_OUT">
+        <PatternLayout pattern="%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n"/>
+    </Console>
+</Appenders>
+
+<Loggers>
+
+    <!-- Enable this line to get a list of metric sent to Graphite -->
+    <!--<Logger name="io.logz.jmx2graphite" level="trace">
+        <AppenderRef ref="default.console"/>
+    </Logger>-->
+
+    <Root level="info">
+        <AppenderRef ref="default.console"/>
+        <AppenderRef ref="default.file"/>
+    </Root>
+</Loggers>
+</Configuration>

src/main/java/io/logz/jmx2graphite/GraphiteClient.java

@@ -19,7 +19,6 @@ public class GraphiteClient implements Closeable {
     private static final Logger logger = LoggerFactory.getLogger(GraphiteClient.class);
     private GraphiteSender graphite;
     private String metricsPrefix;
-    private ExecutorService executorService = Executors.newSingleThreadExecutor();
     private int failuresAtLastWrite = 0;
 
     public GraphiteClient(String serviceHost, String serviceName, String graphiteHostname, int graphitePort,
@@ -37,7 +36,7 @@ public GraphiteClient(String serviceHost, String serviceName, String graphiteHos
             metricsPrefix = "";
         }
 
-        logger.info("Graphite metrics prefix: "+metricsPrefix);
+        logger.info("Graphite metrics prefix: {}", metricsPrefix);
         logger.info("Graphite Client: using writeTimeoutMs of {} [ms]. Establishing connection..." ,writeTimeoutMs);
 
         SocketFactory socketFactory = new SocketFactoryWithTimeouts(connectTimeout, socketTimeout);
@@ -64,6 +63,8 @@ public static String sanitizeMetricName(String s, boolean keepDot) {
                 sb.append('_');
             } else if (c == '"') {
                 // Removing it
+            } else if (c == ' ') {
+                sb.append('-');
             } else {
                 sb.append(c);
             }