container image scanning (#318)

* container image scan feature * Temp commit * Temp commit * Temp commit * Temp commit
ShiftLeftSecurity · May 25, 2021 · 620d26f · 620d26f
1 parent 9ce72a6
commit 620d26f
Show file tree

Hide file tree

Showing 16 changed files with 207 additions and 61 deletions.
diff --git a/.github/workflows/dockertests.yml b/.github/workflows/dockertests.yml
@@ -0,0 +1,27 @@
+name: docker tests
+
+on: [push]
+
+jobs:
+  build:
+    runs-on: ubuntu-20.04
+    strategy:
+      matrix:
+        os: [ubuntu-20.04, macos-latest, windows-latest]
+    steps:
+    - uses: actions/checkout@v1
+    - name: Test container images
+      run: |
+        docker pull shiftleft/scan-slim:latest
+        docker pull shiftleft/scan:latest
+        docker save -o scanslim.tar shiftleft/scan-slim:latest
+        docker save -o scan.tar shiftleft/scan:latest
+        docker run --rm -e "WORKSPACE=${PWD}" -v $PWD:/app shiftleft/scan:docker scan --src /app/scanslim.tar -o /app/reports --type docker
+        docker run --rm -e "WORKSPACE=${PWD}" -e "FETCH_LICENSE=true" -e "ENABLE_OSS_RISK=true" -v $PWD:/app shiftleft/scan:docker scan --src /app/scan.tar -o /app/reports --type docker
+      env:
+        PYTHONPATH: "."
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+    - uses: actions/upload-artifact@v1
+      with:
+        name: reports
+        path: reports
diff --git a/Dockerfile b/Dockerfile
@@ -128,7 +128,6 @@ USER root
 RUN microdnf install python38-devel && pip3 install --no-cache-dir wheel \
     && python3 -m pip install --upgrade pip \
     && python3 -m pip install --no-cache-dir -r /usr/local/src/requirements.txt \
-    && mv /usr/local/bin/scan /usr/local/bin/depscan \
     && npm install --no-audit --progress=false --only=production -g @appthreat/cdxgen @microsoft/rush --unsafe-perm \
     && mkdir -p /opt/sl-cli /opt/phpsast && cd /opt/phpsast && composer require --quiet --no-cache --dev vimeo/psalm \
     && composer require --quiet --no-cache --dev phpstan/phpstan \

diff --git a/README.md b/README.md
@@ -9,7 +9,7 @@
 ╚══════╝ ╚═════╝╚═╝  ╚═╝╚═╝  ╚═══╝
 ```
 
-[Scan](https://slscan.io) is a free open-source security tool for modern DevOps teams. With an integrated multi-scanner based design, Scan can detect various kinds of security flaws in your application and infrastructure code in a single fast scan without the need for any _remote server_. Scan is purpose built for workflow integration with nifty features such as automatic build breaker, results baseline and PR summary comments. Scan products are open-source under a GNU GPL 3.0 or later (GPL-3.0-or-later) license.
+[Scan](https://slscan.io) is a free open-source security tool for modern DevOps teams. With an integrated multi-scanner based design, Scan can detect various kinds of security flaws in your application, and infrastructure code in a single fast scan without the need for any _remote server_. Scan is purpose built for workflow integration with nifty features such as automatic build breaker, results baseline and PR summary comments. Scan products are open-source under a GNU GPL 3.0 or later (GPL-3.0-or-later) license.
 
 [![Build Status](https://dev.azure.com/shiftleftsecurity/sl-appthreat/_apis/build/status/ShiftLeftSecurity.sast-scan?branchName=master)](https://dev.azure.com/shiftleftsecurity/sl-appthreat/_build/latest?definitionId=11&branchName=master)
 
@@ -20,38 +20,40 @@
 
 ## Bundled tools
 
-| Programming Language | Tools                              |
-| -------------------- | ---------------------------------- |
-| ansible              | ansible-lint                       |
-| apex                 | pmd                                |
-| arm                  | checkov                            |
-| aws                  | checkov                            |
-| bash                 | shellcheck                         |
-| bom                  | cdxgen                             |
-| credscan             | gitleaks                           |
-| depscan              | dep-scan                           |
-| go                   | gosec, staticcheck                 |
-| groovy               | find-sec-bugs                      |
-| java                 | cdxgen, gradle, find-sec-bugs, pmd |
-| jsp                  | pmd, find-sec-bugs                 |
-| json                 | jq, jsondiff, jsonschema           |
-| kotlin               | detekt, find-sec-bugs              |
-| scala                | find-sec-bugs                      |
-| kubernetes           | checkov, kubesec, kube-score       |
-| nodejs               | cdxgen, yarn, rush                 |
-| php                  | psalm, phpstan (ide only)          |
-| plsql                | pmd                                |
-| python               | cfg-scan (1), bandit, cdxgen       |
-| ruby                 | brakeman (2), dep-scan             |
-| rust                 | cdxgen                             |
-| serverless           | checkov                            |
-| terraform            | checkov, tfsec                     |
-| Visual Force (vf)    | pmd                                |
-| Apache Velocity (vm) | pmd                                |
-| yaml                 | yamllint                           |
-
-(1) - Deep analyzer for Python is a built-in feature
-(2) - Brakeman is not bundled with scan. Use brakeman with an appropriate license and export the report in json format using `-o reports/source-ruby-report.json`
+| Programming Language   | Tools                              |
+| ---------------------- | ---------------------------------- |
+| ansible                | ansible-lint                       |
+| apex                   | pmd                                |
+| arm                    | checkov                            |
+| aws                    | checkov                            |
+| bash                   | shellcheck                         |
+| bom                    | cdxgen                             |
+| credscan               | gitleaks                           |
+| depscan                | dep-scan                           |
+| dockerfile             | checkov                            |
+| go                     | gosec, staticcheck                 |
+| groovy                 | find-sec-bugs                      |
+| java                   | cdxgen, gradle, find-sec-bugs, pmd |
+| jsp                    | pmd, find-sec-bugs                 |
+| json                   | jq, jsondiff, jsonschema           |
+| kotlin                 | detekt, find-sec-bugs              |
+| scala                  | find-sec-bugs                      |
+| kubernetes             | checkov, kubesec, kube-score       |
+| nodejs                 | cdxgen, yarn, rush                 |
+| php                    | psalm, phpstan (ide only)          |
+| plsql                  | pmd                                |
+| python                 | cfg-scan (1), bandit, cdxgen       |
+| ruby                   | brakeman (2), dep-scan             |
+| rust                   | cdxgen                             |
+| serverless             | checkov                            |
+| terraform              | checkov, tfsec                     |
+| Visual Force (vf)      | pmd                                |
+| Apache Velocity (vm)   | pmd                                |
+| yaml                   | yamllint                           |
+| docker/container image | dep-scan                           |
+
+- (1) - Deep analyzer for Python is a built-in feature
+- (2) - Brakeman is not bundled with scan. Use brakeman with an appropriate license and export the report in json format using `-o reports/source-ruby-report.json`
 
 ## Bundled languages/runtime
 
@@ -131,6 +133,47 @@ docker run --rm -e "WORKSPACE=${PWD}" -v ~/.gradle:/.gradle -v <source path>:/ap
 
 Feel free to skip `--type` to enable auto-detection. Or pass comma-separated values if the project has multiple types.
 
+### Scanning container images
+
+Scanning container images is now possible with slscan. The recommended approach is to export the container image using docker or podman save command first followed by an invocation of scan with the .tar file.
+
+```bash
+docker pull shiftleft/scan-slim:latest
+docker save -o scanslim.tar shiftleft/scan-slim:latest
+# podman save --format oci-archive -o scanslim.tar shiftleft/scan-slim:latest
+docker run --rm -e "WORKSPACE=${PWD}" -v $PWD:/app shiftleft/scan scan --src /app/scanslim.tar -o /app/reports --type docker
+```
+
+Alternatively, it is possible to let scan pull the container image before analysis. However, it requires exposing your docker or podman daemon socket and therefore **not recommended**. You can try it if you are feeling adventurous by passing the below parameters to the docker run command.
+
+```bash
+-e "DOCKER_HOST=unix:/var/run/docker.sock:" -v "/var/run/docker.sock:/var/run/docker.sock"
+```
+
+Example: To scan the container image `shiftleft/scan-slim`:
+
+```bash
+docker run --rm -e "WORKSPACE=$(pwd)" -e "DOCKER_HOST=unix:/var/run/docker.sock:" \
+    -v "/var/run/docker.sock:/var/run/docker.sock" \
+    -v "$(pwd):/app" shiftleft/scan scan -t docker -i shiftleft/scan-slim
+```
+
+Example: To scan the container image `redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e`:
+
+```bash
+docker run --rm -e "WORKSPACE=$(pwd)" -e "DOCKER_HOST=unix:/var/run/docker.sock:" \
+    -v "/var/run/docker.sock:/var/run/docker.sock" \
+    -v "$(pwd):/app" shiftleft/scan scan -t docker -i redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e
+```
+
+Same example with podman
+
+```bash
+podman run --rm -e "WORKSPACE=$(pwd)" -e "DOCKER_HOST=unix:/run/user/1000/podman/podman.sock:" \
+    -v "/run/user/1000:/run/user/1000" \
+    -v "$(pwd):/app" shiftleft/scan scan -t docker -i redmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e
+```
+
 ## Viewing reports
 
 Reports would be produced in the directory specified for `--out_dir`. In the above examples, it is set to `reports` which will be a directory under the source code root directory.

diff --git a/appimage-builder.yml b/appimage-builder.yml
@@ -12,7 +12,6 @@ script:
   - cp tools_config/io.shiftleft.scan.appdata.xml AppDir/usr/share/metainfo/
   # Install application dependencies
   - python3 -m pip install --no-cache-dir --ignore-installed --prefix=/usr --root=AppDir -r ./requirements.txt --no-warn-script-location
-  - mv AppDir/usr/bin/scan AppDir/usr/bin/depscan
   - chmod +x AppDir/usr/src/appimage-reqs.sh && AppDir/usr/src/appimage-reqs.sh AppDir
   - npm install --no-audit --progress=false --only=production --no-save --prefix AppDir/usr/local/lib yarn @appthreat/cdxgen @microsoft/rush
   - mkdir -p AppDir/opt/phpsast && cd AppDir/opt/phpsast && composer init --name shiftleft/scan --description scan --quiet && composer require --quiet --no-cache -n --no-ansi --dev vimeo/psalm:^4.1

diff --git a/ci/Dockerfile-csharp b/ci/Dockerfile-csharp
@@ -88,7 +88,6 @@ USER root
 RUN python3 -m pip install --upgrade pip \
     && pip3 install --no-cache-dir wheel \
     && pip3 install --no-cache-dir -r /usr/local/src/requirements.txt \
-    && mv /usr/local/bin/scan /usr/local/bin/depscan \
     && npm install --no-audit --progress=false --only=production -g @appthreat/cdxgen @microsoft/rush --unsafe-perm \
     && microdnf clean all
 

diff --git a/ci/Dockerfile-dynamic-lang b/ci/Dockerfile-dynamic-lang
@@ -52,7 +52,6 @@ RUN microdnf install -y python38 nodejs git-core which \
     && python3 -m pip install --upgrade pip \
     && pip3 install --no-cache-dir wheel \
     && python3 -m pip install --no-cache-dir -r /usr/local/src/requirements.txt \
-    && mv /usr/local/bin/scan /usr/local/bin/depscan \
     && npm install --no-audit --progress=false --only=production -g @appthreat/cdxgen @microsoft/rush --unsafe-perm \
     && microdnf clean all
 

diff --git a/ci/Dockerfile-java b/ci/Dockerfile-java
@@ -137,7 +137,6 @@ USER root
 RUN python3 -m pip install --upgrade pip \
     && pip3 install --no-cache-dir wheel \
     && pip3 install --no-cache-dir -r /usr/local/src/requirements.txt \
-    && mv /usr/local/bin/scan /usr/local/bin/depscan \
     && npm install --no-audit --progress=false --only=production -g @appthreat/cdxgen @microsoft/rush --unsafe-perm \
     && microdnf clean all
 

diff --git a/ci/Dockerfile-oss b/ci/Dockerfile-oss
@@ -126,7 +126,6 @@ RUN microdnf install python38-devel && pip3 install --no-cache-dir wheel \
     && python3 -m pip install --upgrade pip \
     && pip3 install --no-cache-dir -r /usr/local/src/requirements.txt \
     && pip3 install --no-cache-dir njsscan \
-    && mv /usr/local/bin/scan /usr/local/bin/depscan \
     && npm install --no-audit --progress=false --only=production -g @appthreat/cdxgen @microsoft/rush --unsafe-perm \
     && mkdir -p /opt/phpsast && cd /opt/phpsast && composer require --quiet --no-cache --dev vimeo/psalm \
     && composer require --quiet --no-cache --dev phpstan/phpstan \

diff --git a/dev-build.sh b/dev-build.sh
@@ -3,9 +3,9 @@ DOCKER_CMD=docker
 if command -v podman >/dev/null 2>&1; then
     DOCKER_CMD=podman
 fi
-python3 -m black .
-python3 -m black scan
 isort **/*.py
+python3 -m black lib
+python3 -m black scan
 flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
 flake8 . --count --exit-zero --statistics
 

diff --git a/lib/config.py b/lib/config.py
@@ -374,7 +374,60 @@ def set(configName, value):
         "%(report_fname_prefix)s.json",
     ],
     "docker": {
-        "source-docker": [
+        "image-docker": [
+            get("DEPSCAN_CMD"),
+            "--no-banner",
+            "--suggest",
+            "-t",
+            "docker",
+            "--src",
+            "%(src)s",
+            "--report_file",
+            "%(report_fname_prefix)s.json",
+        ]
+    },
+    "podman": {
+        "image-podman": [
+            get("DEPSCAN_CMD"),
+            "--no-banner",
+            "--suggest",
+            "-t",
+            "docker",
+            "--src",
+            "%(src)s",
+            "--report_file",
+            "%(report_fname_prefix)s.json",
+        ]
+    },
+    "container": {
+        "image-container": [
+            get("DEPSCAN_CMD"),
+            "--no-banner",
+            "--suggest",
+            "-t",
+            "docker",
+            "--src",
+            "%(src)s",
+            "--report_file",
+            "%(report_fname_prefix)s.json",
+        ]
+    },
+    "dockerfile": {
+        "source-dockerfile": [
+            "checkov",
+            "-s",
+            "--framework",
+            "dockerfile",
+            "--quiet",
+            "--no-guide",
+            "-o",
+            "json",
+            "-d",
+            "%(src)s",
+        ]
+    },
+    "containerfile": {
+        "source-containerfile": [
             "checkov",
             "-s",
             "--framework",
@@ -805,7 +858,11 @@ def set(configName, value):
     "checkov": "Security Audit for Infrastructure",
     "source-aws": "Security Audit for AWS",
     "source-arm": "Security Audit for Azure Resource Manager",
-    "source-docker": "Dockerfile Security Audit",
+    "source-containerfile": "Containerfile Security Audit",
+    "source-dockerfile": "Dockerfile Security Audit",
+    "image-container": "Container Image Audit",
+    "image-docker": "Container Image Audit",
+    "image-podman": "Container Image Audit",
     "source-k8s": "Kubernetes Security Audit",
     "source-kt": "Kotlin Static Analysis",
     "audit-kt": "Kotlin Security Audit",

diff --git a/lib/executor.py b/lib/executor.py
@@ -235,7 +235,12 @@ def execute_default_cmd(  # scan:ignore
     # Suppress psalm output
     if should_suppress_output(type_str, cmd_with_args[0]):
         stdout = subprocess.DEVNULL
-    exec_tool(tool_name, cmd_with_args, cwd=src, stdout=stdout)
+    exec_tool(
+        tool_name,
+        cmd_with_args,
+        cwd=os.getcwd() if "image" in tool_name else src,
+        stdout=stdout,
+    )
     # Should we attempt to convert the report to sarif format
     if should_convert(convert, tool_name, cmd_with_args[0], report_fname):
         crep_fname = utils.get_report_file(

diff --git a/lib/pyt/core/ast_helper.py b/lib/pyt/core/ast_helper.py
@@ -8,6 +8,7 @@
 from functools import lru_cache
 
 from _ast import AST
+
 from lib.pyt.core.astsearch import ASTPatternFinder, prepare_pattern
 from lib.pyt.core.transformer import PytTransformer
 

diff --git a/lib/pyt/vulnerabilities/insights.py b/lib/pyt/vulnerabilities/insights.py
@@ -1161,7 +1161,7 @@ def _check_flask_common_misconfig(ast_tree, path):
                             },
                             path,
                         )
-                        if kw_arg == "verify" and kw_arg_value == False:
+                        if kw_arg == "verify" and not kw_arg_value:
                             violations.append(
                                 Insight(
                                     f"""Security Misconfiguration with the config `{kw_arg}` not set to the recommended value `True` for production use""",

diff --git a/lib/utils.py b/lib/utils.py
@@ -220,6 +220,15 @@ def detect_project_type(src_dir, scan_mode):
     else:
         project_types.append("credscan")
     depscan_supported = False
+    if (
+        "docker.io" in src_dir
+        or "quay.io" in src_dir
+        or ":latest" in src_dir
+        or "@sha256" in src_dir
+        or src_dir.endswith(".tar")
+        or src_dir.endswith(".tar.gz")
+    ):
+        project_types.append("docker")
     if find_files(src_dir, ".cls", False, True):
         project_types.append("apex")
     if find_python_reqfiles(src_dir) or find_files(src_dir, ".py", False, True):
@@ -285,7 +294,7 @@ def detect_project_type(src_dir, scan_mode):
     if find_files(src_dir, "serverless.yml", False, True):
         project_types.append("serverless")
     if find_files(src_dir, "Dockerfile", True, True):
-        project_types.append("docker")
+        project_types.append("dockerfile")
     if find_files(src_dir, "deploy.json", False, True) or find_files(
         src_dir, "parameters.json", False, True
     ):