Heimdall Tools has been deprecated. See the new SAF CLI

Heimdall Tools

HeimdallTools supplies several methods to convert output from various tools to "Heimdall Data Format"(HDF) format to be viewable in Heimdall. The current converters are:

1. aws_config_mapper - assess, audit, and evaluate AWS resources
2. burpsuite_mapper - commercial dynamic analysis tool
3. dbprotect_mapper - database vulnerability scanner
4. fortify_mapper - commercial static code analysis tool
5. jfrog_xray_mapper - package vulnerability scanner
6. nessus_mapper - commercial security scanner (supports compliance and vulnerability scans from Tenable.sc and Tenable.io)
7. netsparker_mapper - web application security scanner
8. nikto_mapper - open-source web server scanner
9. sarif_mapper - static analysis results interchange format
10. scoutsuite_mapper - multi-cloud security auditing tool
11. snyk_mapper - commercial package vulnerability scanner
12. sonarqube_mapper - open-source static code analysis tool
13. xccdf_results_mapper - extensible configuration checklist description results format
14. scc_mapper - scap compliance checker format
15. zap_mapper - OWASP ZAP - open-source dynamic code analysis tool

Want to recommend a mapper for another tool? Please use these steps:

1. Create an issue, and email [email protected] citing the issue link so we can help
2. Provide a sample output, preferably the most detailed the tool can provide, and also preferably in a machine-readable format, such as xml, json, or csv - whichever is natively available. If it is sensitive we'll work that in #3. (If it's an API only, we'll also just talk about it in #3)
3. Let's arrange a time to take a close look at the data it provides to get an idea of all it has to offer. We'll suggest an initial mapping of the HDF core elements.
4. Note: if the tool doesn't provide a NIST SP 800-53 reference, we've worked on mappings to other references such as CWE or OWASP Top 10.
5. If the tool doesn't provide something for #4, or another core element such as impact, we'll help you identify a custom mapping approach.
6. We'll help you decide how to preserve any other information (non-core elements) the tool provides to ensure that all of the original tool's intent comes through for the user when the data is viewed in Heimdall.
7. Finally, We'll provide final peer review and support merging your pull request. We appreciate your contributions, but we're here to help!

How to Install Heimdall Tools:

Ruby 2.4 or higher (check using "ruby -v")

If installation of Ruby is required, perform these steps:

Linux Installation of Ruby

Installation on RHEL-based systems

To install Ruby using RVM (Ruby Version Manager):

sudo yum install curl gpg gcc gcc-c++ make patch autoconf automake bison libffi-devel libtool patch readline-devel sqlite-devel zlib-devel openssl-devel

sudo gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

curl -sSL https://get.rvm.io | bash -s stable

source ~/.rvm/scripts/rvm

Finally, install the latest version of Ruby (stable), currently 2.6.x:

rvm install 2.6

rvm use 2.6 --default

Verify the installed version number:

ruby -v

ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]

Installation on Ubuntu-based systems

Installation of Heimdall Tools:

gem install heimdall_tools

Command line Usage

On the Command Line, heimdall_tools help will print a listing of all the command with a short description. For detailed help on any command, run heimdall_tools help [COMMAND]. Help can also be called with the -h, --help flags after any command, like heimdall_tools fortify_mapper -h.

For Docker usage, replace the heimdall_tools command with the correct Docker command below for your operating system:

• On Linux and Mac: docker run -it -v$(pwd):/share mitre/heimdall_tools
• On Windows CMD: docker run -it -v%cd%:/share mitre/heimdall_tools

Note that all of the above Docker commands will mount your current directory on the Docker container. Ensure that you have navigated to the directory you intend to convert files in before executing the command.

aws_config_mapper

aws_config_mapper pulls Ruby AWS SDK data to translate AWS Config Rule results into HDF format json to be viewable in Heimdall

AWS Config Rule Mapping:

The mapping of AWS Config Rules to 800-53 Controls was sourced from this link.

Authentication with AWS:

Developer Guide for configuring Ruby AWS SDK for authentication

USAGE: heimdall_tools aws_config_mapper [OPTIONS] -o

FLAGS:
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools aws_config_mapper -o aws_config_results_hdf.json

burpsuite_mapper

burpsuite_mapper translates an BurpSuite Pro exported XML results file into HDF format json to be viewable in Heimdall

USAGE: heimdall_tools burpsuite_mapper [OPTIONS] -x  -o

FLAGS:
    -x                : path to BurpSuitePro exported XML results file.
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools burpsuite_mapper -x burpsuite_results.xml -o scan_results.json

dbprotect_mapper

dbprotect_mapper translates DBProtect report in Check Results Details format XML to HDF format JSON be viewed on Heimdall.

USAGE: heimdall_tools dbprotect_mapper [OPTIONS] -x  -o

FLAGS:
    -x            : path to DBProtect report XML file.
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools dbprotect_mapper -x check_results_details_report.xml -o db_protect_hdf.json

fortify_mapper

fortify_mapper translates an Fortify results FVDL file into HDF format json to be viewable in Heimdall

USAGE: heimdall_tools fortify_mapper [OPTIONS] -f  -o

FLAGS:
	-f --fvdl          : path to Fortify Scan FVDL file.
	-o --output        : path to output scan-results json.
	-V --verbose                     : verbose run [optional].

example: heimdall_tools fortify_mapper -f audit.fvdl -o scan_results.json

jfrog_xray_mapper

jfrog_xray_mapper translates an JFrog Xray results JSON file into HDF format JSON to be viewable in Heimdall

USAGE: heimdall_tools jfrog_xray_mapper [OPTIONS] -j  -o

FLAGS:
    -j            : path to xray results JSON file.
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools jfrog_xray_mapper -j xray_results.json -o xray_results_hdf.json

nessus_mapper

nessus_mapper translates a Nessus-exported XML results file into HDF format json to be viewable in Heimdall. Supports compliance and vulnerability scans from Tenable.sc and Tenable.io.

Note: A separate HDF JSON file is generated for each host reported in the Nessus Report.

USAGE: heimdall_tools nessus_mapper [OPTIONS] -x  -o

FLAGS:
    -x           : path to Nessus-exported XML results file.
    -o --output_prefix       : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools nessus_mapper -x nessus-results.xml -o test-env

netsparker_mapper

netsparker_mapper translates an Netsparker XML results file into HDF format JSON to be viewable in Heimdall.

The current iteration only works with Netsparker Enterprise Vulnerabilities Scan.

USAGE: heimdall_tools netsparker_mapper [OPTIONS] -x  -o

FLAGS:
    -x       : path to netsparker results XML file.
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools netsparker_mapper -x netsparker_results.xml -o netsparker_hdf.json

nikto_mapper

nikto_mapper translates an Nikto results JSON file into HDF format JSON to be viewable in Heimdall

Note: Current this mapper only support single target Nikto Scans.

USAGE: heimdall_tools nikto_mapper [OPTIONS] -x  -o

FLAGS:
    -j           : path to Nikto results JSON file.
    -o --output_prefix       : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools nikto_mapper -j nikto_results.json -o nikto_results.json

sarif_mapper

sarif_mapper translates a SARIF JSON file into HDF format JSON to be viewable in Heimdall

USAGE: heimdall_tools sarif_mapper [OPTIONS] -j  -o

FLAGS:
    -j           : path to SARIF results JSON file.
    -o --output_prefix       : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools sarif_mapper -j sarif_results.json -o sarif_results_hdf.json

scoutsuite_mapper

scoutsuite_mapper translates Scout Suite results from Javascript to HDF-formatted JSON so as to be viewable on Heimdall

Note: Currently this mapper only supports AWS.

USAGE: heimdall_tools scoutsuite_mapper -i  -o

FLAGS:
    -i --input -j --javascript  : path to Scout Suite results Javascript file.
    -o --output                 : path to output scan-results json.

example: heimdall_tools scoutsuite_mapper -i scoutsuite_results.js -o scoutsuite_hdf.json

snyk_mapper

snyk_mapper translates an Snyk results JSON file into HDF format json to be viewable in Heimdall

Note: A separate HDF JSON is generated for each project reported in the Snyk Report.

USAGE: heimdall_tools snyk_mapper [OPTIONS] -x  -o

FLAGS:
    -j           : path to Snyk results JSON file.
    -o --output_prefix       : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools snyk_mapper -j snyk_results.json -o output-file-prefix

sonarqube_mapper

sonarqube_mapper pulls SonarQube results, for the specified project, from the API and outputs in HDF format Json to be viewed on Heimdall

USAGE: heimdall_tools sonarqube_mapper [OPTIONS] -n  -u  -o

FLAGS:
    -n --name          : Project Key of the project in SonarQube
    -u --api_url            : url of the SonarQube Server API. Typically ends with /api.
    --auth               : username:password or token [optional].
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example:

heimdall_tools sonarqube_mapper -n sonar_project_key -u http://sonar:9000/api -o scan_results.json

heimdall_tools sonarqube_mapper -n sonar_project_key -u http://sonar:9000/api --auth admin:admin -o scan_results.json

xccdf_results_mapper

Note: SCC outputs scan results in XCCDF-Results format.

xccdf_results_mapper translates an XCCDF_Results XML scan into HDF format json to be viewable in Heimdall.

USAGE: heimdall_tools xccdf_results_mapper [OPTIONS] -x  -o

FLAGS:
    -x            : path to XCCDF-Results XML file.
    -o --output        : path to output scan-results json.

example: heimdall_tools xccdf_results_mapper -x xccdf_results.xml -o scan_results.json

zap_mapper

zap_mapper translates OWASP ZAP results Json to HDF format Json be viewed on Heimdall

USAGE: heimdall_tools zap_mapper [OPTIONS] -j  -n  -o

FLAGS:
    -j --json              : path to OWASP ZAP results JSON file.
    -n --name             : URL of the site being evaluated.
    -o --output        : path to output scan-results json.
    -V --verbose                     : verbose run [optional].

example: heimdall_tools zap_mapper -j zap_results.json -n site_name -o scan_results.json

version

Prints out the gem version

USAGE: heimdall_tools version

Development

Submitting a PR

A complete PR should include 7 core elements:

• A signed PR ( aka git commit -a -s )
• Code for the new functionality
• Updates to the CLI
• New unit tests for the functionality
• Updates to the docs and examples in README.md and ./docs/*
• (if needed) Example / Template files ( metadata.yml,example.yml, etc )
  • Scripts / Scaffolding code for the Example / Template files ( generate_map is an example )
• Example Output of the new functionality if it produces an artifact

Overview of our PR process

1. open an issue on the main inspec_tools website noting the issues your PR will address
2. fork the repo
3. checkout your repo
4. cd to the repo
5. git co -b ``
6. bundle install
7. hack as you will
8. test via rake
9. ensure unit tests still function and add unit tests for your new feature
10. add new docs to the README.md and to ./docs/examples
11. update the CLI as needed and add in usage example
12. (if needed) create and document any example or templates
13. (if needed) create any supporing scripts
14. (opt) gem build inspec_tools.gemspec
15. (opt) gem install inspec_tools
16. (opt) test via the installed gem
17. git commit -a -s ``
18. Open a PRs aginst the MITRE inspec_tools repo

Publishing a Release

If you are a maintainer, it is very easy to cut a release of this gem:

1. Click on "Releases" and there should be a draft pending.
2. Make sure the Tag version and Release title match!
3. Add any additional notes can be added in the Description box.
4. Click "Publish release".
5. Release notes will be posted and a new gem will be pushed to Rubygems & Github Packages with the version you specified on step 2.

Testing

This gem was developed using the CLI Template, a generator tool that builds a starter CLI project.

There are a set of unit tests. Run rake test to run the tests.

To release a new version, update the version number in version.rb according to the Semantic Versioning Policy. Then, run bundle exec rake release which will create a git tag for the specified version, push git commits and tags, and push the .gem file to github.com.

License and Author

Authors

• Author:: Rony Xavier rx294
• Author:: Dan Mirsky mirskiy

NOTICE

© 2018 The MITRE Corporation.

Approved for Public Release; Distribution Unlimited. Case Number 18-3678.

NOTICE

MITRE hereby grants express written permission to use, reproduce, distribute, modify, and otherwise leverage this software to the extent permitted by the licensed terms provided in the LICENSE.md file included with this project.

NOTICE

This software was produced for the U. S. Government under Contract Number HHSM-500-2012-00008I, and is subject to Federal Acquisition Regulation Clause 52.227-14, Rights in Data-General.

No other use other than that granted to the U. S. Government, or to those acting on behalf of the U. S. Government under that Clause is authorized without the express written permission of The MITRE Corporation.

For further information, please contact The MITRE Corporation, Contracts Management Office, 7515 Colshire Drive, McLean, VA 22102-7539, (703) 983-6000.