Malcolm v23.05.0 #249

mmguero · 2023-05-01T19:23:31Z

Malcolm v23.05.0 is a major release with new features, enhancements, component version updates and bug fixes.

IMPORTANT NOTE: Malcolm v23.05.0 has completely changed the way it manages its settings: rather than using environment variables found at the top of the docker-compose.yml file, it uses environment variables in .env files inside of the config directory. The locations of a number of configuration files have also changed. It's not recommended to update to Malcolm v23.05.0 from a previous version of Malcolm. Instead, shut down Malcolm, rename your old Malcolm installation directory to something else, and reconfigure Malcolm using ./scripts/configure and ./scripts/auth_setup.

v23.04.0...v23.05.0

New features
- integrate ICSNPP-Synchrophasor parser (integrate synchrophasor parser idaholab/Malcolm#190)
- End-to-end Malcolm and Hedgehog Linux ISO Installation document (end-to-end ISO install documentation for Malcolm and Hedgehog idaholab/Malcolm#181)
- support Malcolm deployment with Kubernetes (support Malcolm deployment with Kubernetes idaholab/Malcolm#149)
  - see Deploying Malcolm with Kubernetes
  - This could be considered a "beta" release for Malcolm deployment with Kubernetes, as there is still some work to be done in this area. Please let us know what issues or suggestions you have via the issue tracker or via email to [email protected].
  - contributing issues:
    - inotify issue (kubernetes deployment - inotify issue idaholab/Malcolm#168)
    - htadmin/nginx and htpasswd (kubernetes deployment - htadmin/nginx and htpasswd idaholab/Malcolm#169)
    - opensearch (kubernetes deployment - opensearch idaholab/Malcolm#170)
    - uploading large PCAP files (kubernetes deployment - uploading large PCAP files idaholab/Malcolm#171)
    - script consolidation (kubernetes deployment - script consolidation idaholab/Malcolm#172)
    - documentation (kubernetes deployment - documentation idaholab/Malcolm#173)
    - user-defined persistent volumes (kubernetes deployment - user-defined persistent volumes idaholab/Malcolm#174)
    - opensearch keystore (kubernetes deployment - opensearch keystore idaholab/Malcolm#176)
    - expose other TCP services (kubernetes deployment - expose other TCP services idaholab/Malcolm#183)
    - provide with filebeat access to nginx access and error logs (kubernetes deployment - provide with filebeat access to nginx access and error logs idaholab/Malcolm#186)
    - use Secrets for some environment variables instead of ConfigMaps (kubernetes - use Secrets for some environment variables instead of ConfigMaps idaholab/Malcolm#189)
Enhancements and fixes
- remove name-map-ui container (remove name-map-ui container idaholab/Malcolm#165) in favor of using NetBox for asset identification
- Python script refactoring, consolidation and cleanup
- standardization of Docker container entrypoints
- create ./scripts/configure alias for ./scripts/install.py --configure
Component version updates
- Arkime to v4.3.0
- Capa to v5.1.0
- Fluent Bit to v2.1.2
- NetBox to v3.5.0
- NGINX to v1.22.1
- Supercronic to v0.2.24
- Suricata to v6.0.10
- Yara to v4.3.0
- Zeek to v5.2.1

…in nginx

… containers

…anyway and it's not included in older rereleases

… of docker-compose yml file (install.py and documentation not done yet)

…into kubernetes

…. the docker-compose.yml files

…into kubernetes

…oard

…5bdc109be7a4dd8ee2c2838673980b5f/CHANGELOG#L33-L72)

…like the configmap directory on container entrypoint

… instead of ConfigMaps

…d configure-interfaces

…into v23.05.0_merge_cisagov

SG and others added 30 commits March 20, 2023 14:33

work in progress with services

8aa25e0

Use slightly smarter fallback HTTP server

1e407f5

work in progress

273cd89

for idaholab#165, remove name-map-ui container

44ff108

allow API to be reached without speacifying trailing slash delimiter …

8408951

…in nginx

getting upload to work

95792cc

upload wip

ea26049

example working claim

5e06916

remove trailing slashes

f36d08f

work in progress

0837203

work in progress, added opensearch-curlrc and ca-trust volumes across…

8794485

… containers

remove --mkpath from rsync command, as we're only creating one level …

25577d5

…anyway and it's not included in older rereleases

tweaks to images and env. variables

6533e57

give volume names uniqueness across the board

20bc6b8

added rsync to arkime

33ab78e

fix opensearch curlrc files

097af99

fix opensearch curlrc files

4ee6793

work in progress moving all environment variables into config instead…

b9dc58e

… of docker-compose yml file (install.py and documentation not done yet)

fix LS_JAVA_OPTS env var

ad2322c

rewording documentation

07389fb

Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …

98f0af1

…into kubernetes

work in progress on using environment variables for storing config vs…

592182f

…. the docker-compose.yml files

Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …

81acc26

…into kubernetes

tweak dockerignore/gitignore

3e635ad

fix file-upload start

c67ebc5

set all netbox-related variables

8266cd9

Fix logstash->netbox connection

700746f

arkime should default to WISE on

4cb2747

don't prompt for both netsniff and tcpdump

1654dc1

Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …

de07af5

…into kubernetes

mmguero added 25 commits April 25, 2023 17:48

point upstream for zeek packages

62fc4ab

bump fluent bit to v2.1.2 (https://fluentbit.io/announcements/v2.1.2/)

3c7a4ef

idaholab#190, just about everything for synchrophasor minus the dashb…

9a948df

…oard

idaholab#190, a little normalization and dashboards

efcc7f1

idaholab#190, a little normalization and dashboards

25dbbb1

idaholab#190, allow overriding synchrophasor ports

c0a13cf

fix version in kubernetes manifests

b789f0f

bump arkime to v4.3.0 (https://github.com/arkime/arkime/blob/fa0db241…

ec7e518

…5bdc109be7a4dd8ee2c2838673980b5f/CHANGELOG#L33-L72)

minor arkime ini tweaks

5dfbf68

for idaholab#189, have a special secretmap directory that is treated …

2abc485

…like the configmap directory on container entrypoint

idaholab#189; kubernetes - use Secrets for some environment variables…

15d9365

… instead of ConfigMaps

change image references for kubernetes (only in my development branch)

b590f34

ensure created .env files get correct permissions/ownership

b92bdb3

put CSRF_TRUSTED_ORIGINS in correct .env file

7796a60

Added a missing script into the ISOs

314084e

Bump version for v23.05.0 release

29100c3

buildalo

b6c2136

fix missing import

69fb3a7

use run_subprocess instead of run_process in ISO configure-capture an…

e83244f

…d configure-interfaces

fixes for aggregator ISO init

7640a4d

fixes for aggregator ISO init

5dc1987

fix sensor build

4d8c309

fix mkdir for zeek_intel_setup.sh

cb8229f

fix sensor build

418456d

Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …

3a22b1e

…into v23.05.0_merge_cisagov

mmguero self-assigned this May 1, 2023

mmguero added 2 commits May 1, 2023 14:53

update sha256 sums

c3cda80

Merge branch 'development' of https://github.com/mmguero-dev/Malcolm …

22d0338

…into v23.05.0_merge_cisagov

mmguero merged commit c02a6e9 into main May 1, 2023

mmguero deleted the v23.05.0_merge_cisagov branch May 1, 2023 21:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Malcolm v23.05.0 #249

Malcolm v23.05.0 #249

mmguero commented May 1, 2023 •

edited

Loading

Malcolm v23.05.0 #249

Malcolm v23.05.0 #249

Conversation

mmguero commented May 1, 2023 • edited Loading

mmguero commented May 1, 2023 •

edited

Loading