Rollbar Daily News

Your source for runnable rollbar reporting changes!

Friday February 5th, 2016

API

custom.err rule was too loose

• Fixed the custom.err grouping rule to instead check for custom.err.message. It was causing some errors that did have a custom.err to strangely report if that error had an empty message.

Changed Rules

[
  {
    "condition": {
      "contains": "message", 
      "path": "custom.err"
    }, 
    "fingerprint": "{{ custom.err.message }}", 
    "title": "{{ custom.err.message }}"
  }
]

Thursday February 4th, 2016

Shiva / Metis

• Shiva and metis were being grouped together (since the astral project itself only had a single rollbar key). Split these into two separate projects in rollbar and redeployed metis with the new key.
• Added "by queue" grouping rules for both projects (see below)

New Rules

[
  {
    "condition": {
      "all": [
        {
          "path": "custom",
          "contains": "queue"
        },
        {
          "path": "body.trace.exception",
          "contains": "message"
        }
      ]
    },
    "fingerprint": "{{ custom.queue }}{{ body.trace.exception.message }}",
    "title": "{{ custom.queue }} - {{ body.trace.exception.message }}"
  }
]

Wednesday February 3rd, 2016

Charon

#2 Error: Invalid Ip Return by API

• runnable-api-client was not reporting when errors occurred and was instead returning a null result when charon performed the DNS lookup. Internally, charon was applying a blanket validation rule to the result expecting errors to be thrown as opposed to encountering null or empty data, and was thus incorrectly reporting a "ip is malformed" error.
• Fixed api-client to return meaningful errors
• Fixed charon to not report when the ip is empty (which can happen)

Tuesday February 2nd, 2016

API

#12443 Worker create-instance-container: Create container failed: unable to find a node that satisfies org==2828361

• Runnable just didn't have enough docks, this has been rectified.

#12329 Error: unknown

• This was, partially, a super-grouping being applied by rollbar to general errors that were wrapping specific errors (which can be found in the custom data section)
• These are now grouped using the "custom.err" object with realistic titles (based on custom.err.message)

New Rules

[
  {
    "condition": {
      "path": "custom",
      "contains": "err"
    },
    "fingerprint": "{{ custom.err.message }}",
    "title": "{{ custom.err.message }}"
  }
]

#10886 Error: {"message":"Not Found",...}

• This is a super grouping of all 404 errors as reported by the API's express server
• Created a custom rule to group 404s by route
• Added specific rule for the webhook internal route, as it seems to be reported in large batches for some reason (root cause unknown)

New Rules

[
  {
    "condition": {
      "eq": "/contexts/:contextId/versions/:id/actions/build", 
      "path": "context"
    }, 
    "fingerprint": "webhook-404s", 
    "title": "Build 404 When Processing Webhook"
  },
  {
    "condition": {
      "eq": "Not Found",
      "path": "custom.error"
    },
    "fingerprint": "404-{{ context }}",
    "title": "404 Not Found: {{ context }}"
  }
]

#12496 Error: Not running

• This appears to be triggered when hermes.close is called and the server is not running
• Should be safe to ignore, going to look into the error handling code in the API to see about doing that
• Only happens in staging (thanks RQL!), so added additional logic to bypass reporting of the error if process.env.NODE_ENV === 'staging'
• PR: https://github.com/CodeNow/api/pull/1325