4.1.9 (#2449)

* Anti-analysis bypass - JADX fallback to DEX files on APK decompilation failure - apktool fallback to androguard for AndroidManifest.xml extraction - apksigner.jar fallback to apksigtool/androguard for signature version extraction - Graceful erorrs for failures instead of exceptions
MobSF · Nov 11, 2024 · 3cf21ce · 3cf21ce
1 parent 70e243d
commit 3cf21ce
Show file tree

Hide file tree

Showing 15 changed files with 193 additions and 113 deletions.
diff --git a/README.md b/README.md
@@ -1,18 +1,15 @@
 # Mobile Security Framework (MobSF)
-Version: v4.1
 
 ![](https://cloud.githubusercontent.com/assets/4301109/20019521/cc61f7fc-a2f2-11e6-95f3-407030d9fdde.png)
 
 Mobile Security Framework (MobSF) is a security research platform for mobile applications in Android, iOS and Windows Mobile. MobSF can be used for a variety of use cases such as mobile application security, penetration testing, malware analysis, and privacy analysis. The Static Analyzer supports popular mobile app binaries like APK, IPA, APPX and source code. Meanwhile, the Dynamic Analyzer supports both Android and iOS applications and offers a platform for interactive instrumented testing, runtime data and network traffic analysis. MobSF seamlessly integrates with your DevSecOps or CI/CD pipeline, facilitated by REST APIs and CLI tools, enhancing your security workflow with ease.
 
 Made with ![Love](https://cloud.githubusercontent.com/assets/4301109/16754758/82e3a63c-4813-11e6-9430-6015d98aeaab.png) in India
 
-[![python](https://img.shields.io/badge/python-3.10+-blue.svg?logo=python&labelColor=yellow)](https://www.python.org/downloads/)
+[![Docker Pulls](https://img.shields.io/docker/pulls/opensecurity/mobile-security-framework-mobsf?style=social)](https://hub.docker.com/r/opensecurity/mobile-security-framework-mobsf/) [![python](https://img.shields.io/badge/python-3.10+-blue.svg?logo=python&labelColor=yellow)](https://www.python.org/downloads/)
 [![PyPI version](https://badge.fury.io/py/mobsf.svg)](https://badge.fury.io/py/mobsf)
 [![platform](https://img.shields.io/badge/platform-osx%2Flinux%2Fwindows-green.svg)](https://github.com/MobSF/Mobile-Security-Framework-MobSF/)
 [![License](https://img.shields.io/:license-GPL--3.0--only-blue.svg)](https://www.gnu.org/licenses/gpl-3.0.html)
-[![Docker Pulls](https://img.shields.io/docker/pulls/opensecurity/mobile-security-framework-mobsf?style=social)](https://hub.docker.com/r/opensecurity/mobile-security-framework-mobsf/)
-
 [![MobSF tests](https://github.com/MobSF/Mobile-Security-Framework-MobSF/workflows/MobSF%20tests/badge.svg?branch=master)](https://github.com/MobSF/Mobile-Security-Framework-MobSF/actions)
 [![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=MobSF_Mobile-Security-Framework-MobSF&metric=alert_status)](https://sonarcloud.io/dashboard?id=MobSF_Mobile-Security-Framework-MobSF)
 ![GitHub closed issues](https://img.shields.io/github/issues-closed/MobSF/Mobile-Security-Framework-MobSF)
@@ -32,23 +29,21 @@ MobSF is also bundled with [Android Tamer](https://tamerplatform.com), [BlackArc
 
 [![Donate to MobSF](https://user-images.githubusercontent.com/4301109/117404264-7aab5480-aebe-11eb-9cbd-da82d7346bb3.png)](https://opensecurity.in/donate)
 
-If you liked MobSF and find it useful, please consider donating.
 
-*It's easy to build open source, maintaining one is a different story. Long live open source!*
+> Has MobSF made a difference for you? Show your support and help us innovate with a donation. It's easy to build open source, maintaining one is a different story. 
+
+*Long live open source!*
 
 ## Documentation
 
-Quick setup
+Quick setup with docker
 
 ```
 docker pull opensecurity/mobile-security-framework-mobsf:latest
 docker run -it --rm -p 8000:8000 opensecurity/mobile-security-framework-mobsf:latest
 ```
 
 [![See MobSF Documentation](https://user-images.githubusercontent.com/4301109/70686099-3855f780-1c79-11ea-8141-899e39459da2.png)](https://mobsf.github.io/docs)
-[![See MobSF Documentation in Chinese](https://user-images.githubusercontent.com/4301109/117404947-b09d0880-aebf-11eb-9db8-3d7360f47914.png)](https://mobsf.github.io/docs/#/zh-cn/)
-[![See MobSF Documentation in Japanese](https://user-images.githubusercontent.com/4301109/148662149-7ee671b4-66a2-4232-9522-276b8e88d924.png)](https://mobsf.github.io/docs/#/ja-jp/)
-[![See MobSF Documentation in Español](https://user-images.githubusercontent.com/4301109/173221657-ac1f7221-6ae9-44d8-bf6b-8732d84bf120.png)](https://mobsf.github.io/docs/#/es/)
 
 * Try MobSF Static Analyzer Online: [mobsf.live](https://mobsf.live)
 * MobSF in CI/CD: [mobsfscan](https://github.com/MobSF/mobsfscan)
@@ -101,7 +96,7 @@ docker run -it --rm -p 8000:8000 opensecurity/mobile-security-framework-mobsf:la
 
 * [Dominik Schlecht](https://github.com/sn0b4ll) ![germany](https://user-images.githubusercontent.com/4301109/37564176-743238ba-2ab6-11e8-9666-5d98f0a1d127.png)
 
-## Honorable Contributors
+## Honorable Contributors & Shoutouts
 
 * Amrutha VC - For the new MobSF logo
 * Dominik Schlecht - For the awesome work on adding Windows Phone App Static Analysis to MobSF
@@ -111,9 +106,6 @@ docker run -it --rm -p 8000:8000 opensecurity/mobile-security-framework-mobsf:la
 * Abhinav Saxena - (@xandfury) - For Travis CI and Logging integration
 * ![netguru](https://user-images.githubusercontent.com/4301109/76340877-a3dc4f00-62d2-11ea-8631-b4cc8d9e42ed.png) [Netguru](https://www.netguru.com/) (@karolpiateknet, @mtbrzeski) - For iOS Swift support, Rule contributions and SAST refactoring.
 * Maxime Fawe - (@Arenash13) - For Matching Strategy implementation of SAST pattern matching algorithms.
-
-## Shoutouts
-
 * Abhinav Sejpal (@Abhinav_Sejpal) - For poking me with bugs, feature requests, and UI & UX suggestions
 * Anant Srivastava (@anantshri) - For Activity Tester Idea
 * Anto Joseph (@antojoseph) - For the help with SuperSU

diff --git a/docker/nginx.conf b/docker/nginx.conf
@@ -2,33 +2,41 @@ user  nginx;
 events {
     worker_connections   1000;
 }
+
 http {
-        server {
-              listen 4000;
-              location / {
-                proxy_set_header        X-Forwarded-Host    $host;
-                proxy_set_header        X-Forwarded-Port    443;
-                proxy_set_header        X-Forwarded-For     $proxy_add_x_forwarded_for;
+    client_max_body_size 256M;
+    upstream mobsf_upstream {
+        server mobsf:8000;
+        server mobsf:1337;
+        keepalive 16;
+    }
+
+    map $server_port $forwarded_port {
+        4000 443;
+        4001 443;
+    }
+
+    proxy_set_header  X-Forwarded-Host    $host;
+    proxy_set_header  X-Forwarded-Port    $forwarded_port;
+    proxy_set_header  X-Forwarded-For     $proxy_add_x_forwarded_for;
+    proxy_redirect    off;
+    proxy_buffering   on;
 
-                proxy_pass http://mobsf:8000;
-                proxy_redirect         off;
-                proxy_read_timeout     900;
-                proxy_buffering        on;
-              }
-              client_max_body_size 256M;
+    server {
+        listen 4000;
+        location / {
+            proxy_pass          http://mobsf:8000;
+            proxy_read_timeout  900;
+            client_max_body_size 256M;
         }
-        server {
-              listen 4001;
-              location / {
-                proxy_set_header        X-Forwarded-Host    $host;
-                proxy_set_header        X-Forwarded-Port    443;
-                proxy_set_header        X-Forwarded-For     $proxy_add_x_forwarded_for;
+    }
 
-                proxy_pass http://mobsf:1337;
-                proxy_redirect         off;
-                proxy_read_timeout     120;
-                proxy_buffering        on;
-              }
-              client_max_body_size 10M;
+    server {
+        listen 4001;
+        location / {
+            proxy_pass          http://mobsf:1337;
+            proxy_read_timeout  120;
+            client_max_body_size 10M;
         }
+    }
 }
diff --git a/mobsf/MalwareAnalyzer/views/android/permissions.py b/mobsf/MalwareAnalyzer/views/android/permissions.py
@@ -32,6 +32,7 @@
     'android.permission.WRITE_EXTERNAL_STORAGE',
     'android.permission.READ_EXTERNAL_STORAGE',
     'android.permission.VIBRATE',
+    'android.permission.REQUEST_INSTALL_PACKAGES',
 ]
 OTHER_PERMISSIONS = [
     'android.permission.ACCESS_BACKGROUND_LOCATION',
@@ -71,7 +72,6 @@
     'android.permission.READ_CALENDAR',
     'android.permission.PACKAGE_USAGE_STATS',
     'android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS',
-    'android.permission.REQUEST_INSTALL_PACKAGES',
     'android.permission.WRITE_CONTACTS',
     'android.permission.WRITE_SMS',
     'com.android.launcher.permission.INSTALL_SHORTCUT',

diff --git a/mobsf/MobSF/init.py b/mobsf/MobSF/init.py
@@ -18,7 +18,7 @@
 
 logger = logging.getLogger(__name__)
 
-VERSION = '4.1.8'
+VERSION = '4.1.9'
 BANNER = r"""
   __  __       _    ____  _____       _  _    _ 
  |  \/  | ___ | |__/ ___||  ___|_   _| || |  / |

diff --git a/mobsf/StaticAnalyzer/views/android/cert_analysis.py b/mobsf/StaticAnalyzer/views/android/cert_analysis.py
@@ -152,8 +152,8 @@ def get_signature_versions(checksum, app_path, tools_dir, signed):
         if re.findall(r'\(APK Signature Scheme v4\): true', out):
             v4 = True
     except Exception as exp:
-        msg = 'Failed to get signature versions'
-        logger.exception(msg)
+        msg = 'Failed to get signature versions with apksigner'
+        logger.error(msg)
         append_scan_status(checksum, msg, repr(exp))
     return v1, v2, v3, v4
 
@@ -166,18 +166,30 @@ def apksigtool_cert(checksum, apk_path, tools_dir):
     signed = False
     certs_no = 0
     min_sdk = None
+    av1, av2, av3, av4 = None, None, None, None
     try:
         from apksigtool import (
             APKSignatureSchemeBlock,
             extract_v2_sig,
             parse_apk_signing_block,
         )
+        from apksigcopier import (
+            extract_meta,
+        )
+        meta = extract_meta(apk_path)
+        sig_files = [x.filename for x, _ in meta]
+        if sig_files:
+            av1 = True
+        else:
+            av1 = False
         _, sig_block = extract_v2_sig(apk_path)
         for pair in parse_apk_signing_block(sig_block).pairs:
             b = pair.value
             if isinstance(b, APKSignatureSchemeBlock):
                 signed = True
                 for signer in b.signers:
+                    av2 = b.is_v2()
+                    av3 = b.is_v3()
                     if b.is_v3():
                         min_sdk = signer.min_sdk
                     certs_no = len(signer.signed_data.certificates)
@@ -200,6 +212,10 @@ def apksigtool_cert(checksum, apk_path, tools_dir):
             apk_path,
             tools_dir,
             signed)
+        if signed and not (v1 or v2 or v3 or v4):
+            # apksigner.jar failed to get signature versions
+            logger.info('Fetching signature versions with apksigtool')
+            v1, v2, v3, v4 = av1, av2, av3, av4
         certlist.append(f'v1 signature: {v1}')
         certlist.append(f'v2 signature: {v2}')
         certlist.append(f'v3 signature: {v3}')
@@ -238,6 +254,10 @@ def get_cert_data(checksum, a, app_path, tools_dir):
         app_path,
         tools_dir,
         signed)
+    if signed and not (v1 or v2 or v3 or v4):
+        # apksigner.jar failed to get signature versions
+        logger.info('Fetching signature versions with androguard')
+        v1, v2, v3, v4 = a.is_signed_v1(), a.is_signed_v2(), a.is_signed_v3(), None
     certlist.append(f'v1 signature: {v1}')
     certlist.append(f'v2 signature: {v2}')
     certlist.append(f'v3 signature: {v3}')

diff --git a/mobsf/StaticAnalyzer/views/android/code_analysis.py b/mobsf/StaticAnalyzer/views/android/code_analysis.py
@@ -104,6 +104,9 @@ def code_analysis(checksum, app_dir, typ, manifest_file, android_permissions):
             {'.java', '.kt'},
             [src],
             skp)
+        msg = 'Android API Analysis Completed'
+        logger.info(msg)
+        append_scan_status(checksum, msg)
         # Permission Mapping
         rule_file = get_perm_rules(checksum, perm_rules, android_permissions)
         if rule_file:

diff --git a/mobsf/StaticAnalyzer/views/android/converter.py b/mobsf/StaticAnalyzer/views/android/converter.py
@@ -9,6 +9,7 @@
 import subprocess
 import threading
 import stat
+from pathlib import Path
 
 from django.conf import settings
 
@@ -71,51 +72,84 @@ def dex_2_smali(checksum, app_dir, tools_dir):
 
 
 def apk_2_java(checksum, app_path, app_dir, dwd_tools_dir):
-    """Run jadx."""
+    """Run JADX to decompile APK or all DEX files to Java source code."""
     try:
         jadx_version = '1.5.0'
-        jadx_path = f'jadx/jadx-{jadx_version}/bin/'
-        msg = 'Decompiling APK to Java with jadx'
+        jadx_base_path = Path(dwd_tools_dir) / 'jadx' / f'jadx-{jadx_version}' / 'bin'
+        output_dir = Path(app_dir) / 'java_source'
+
+        msg = 'Decompiling APK to Java with JADX'
         logger.info(msg)
         append_scan_status(checksum, msg)
-        args = []
-        output = os.path.join(app_dir, 'java_source/')
 
-        if os.path.exists(output):
-            # ignore WinError3 in Windows
-            shutil.rmtree(output, ignore_errors=True)
+        # Clean output directory if it exists
+        if output_dir.exists():
+            shutil.rmtree(output_dir, ignore_errors=True)
 
+        # Determine JADX executable path
         if (len(settings.JADX_BINARY) > 0
                 and is_file_exists(settings.JADX_BINARY)):
-            jadx = settings.JADX_BINARY
+            jadx = Path(settings.JADX_BINARY)
         elif platform.system() == 'Windows':
-            jadx = os.path.join(
-                dwd_tools_dir, f'{jadx_path}jadx.bat')
+            jadx = jadx_base_path / 'jadx.bat'
         else:
-            jadx = os.path.join(
-                dwd_tools_dir, f'{jadx_path}jadx')
-        # Set execute permission, if JADX is not executable
-        if not os.access(jadx, os.X_OK):
-            os.chmod(jadx, stat.S_IEXEC)
+            jadx = jadx_base_path / 'jadx'
+
+        # Ensure JADX has execute permissions
+        if not os.access(str(jadx), os.X_OK):
+            os.chmod(str(jadx), stat.S_IEXEC)
+
+        # Prepare the base arguments for JADX
+        def run_jadx(arguments):
+            """Run JADX command with the specified arguments."""
+            with open(os.devnull, 'w') as fnull:
+                return subprocess.run(
+                    arguments,
+                    stdout=fnull,
+                    stderr=subprocess.STDOUT,
+                    timeout=settings.JADX_TIMEOUT)
+
+        # First attempt to decompile APK
         args = [
-            jadx,
-            '-ds',
-            output,
-            '-q',
-            '-r',
-            '--show-bad-code',
-            app_path,
-        ]
-        fnull = open(os.devnull, 'w')
-        subprocess.run(args,
-                       stdout=fnull,
-                       stderr=subprocess.STDOUT,
-                       timeout=settings.JADX_TIMEOUT)
+            str(jadx), '-ds', str(output_dir),
+            '-q', '-r', '--show-bad-code', app_path]
+        result = run_jadx(args)
+        if result.returncode == 0:
+            return  # Success
+
+        # If APK decompilation fails, attempt to decompile all DEX files recursively
+        msg = 'Decompiling with JADX failed, attempting on all DEX files'
+        logger.warning(msg)
+        append_scan_status(checksum, msg)
+
+        dex_files = Path(app_path).parent.rglob('*.dex')
+        decompile_failed = False
+
+        for dex_file in dex_files:
+            msg = f'Decompiling {dex_file.name} with JADX'
+            logger.info(msg)
+            append_scan_status(checksum, msg)
+
+            # Update argument to point to the current DEX file
+            args[-1] = str(dex_file)
+            result_dex = run_jadx(args)
+
+            if result_dex.returncode != 0:
+                decompile_failed = True
+                msg = f'Decompiling with JADX failed for {dex_file.name}'
+                logger.error(msg)
+                append_scan_status(checksum, msg)
+
+        if decompile_failed:
+            msg = 'Some DEX files failed to decompile'
+            logger.error(msg)
+            append_scan_status(checksum, msg)
+
     except subprocess.TimeoutExpired as exp:
-        msg = 'Decompiling with jadx timed out'
+        msg = 'Decompiling with JADX timed out'
         logger.warning(msg)
         append_scan_status(checksum, msg, repr(exp))
     except Exception as exp:
-        msg = 'Decompiling with jadx failed'
+        msg = 'Decompiling with JADX failed'
         logger.exception(msg)
         append_scan_status(checksum, msg, repr(exp))
diff --git a/mobsf/StaticAnalyzer/views/android/jar_aar.py b/mobsf/StaticAnalyzer/views/android/jar_aar.py
@@ -114,6 +114,7 @@ def common_analysis(request, app_dic, rescan, api, analysis_type):
                 app_dic['app_dir'],
                 app_dic['tools_dir'],
                 'aar',
+                apk,
             )
             app_dic['manifest_file'] = mani_file
             app_dic['ns'] = ns