From c86b3d035c2149cd4197a3181bfe80f7866347bb Mon Sep 17 00:00:00 2001 From: Sebastian Wagner Date: Mon, 8 Aug 2022 11:39:31 +0200 Subject: [PATCH 1/4] doc: update supported operating systems --- docs/user/installation.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/user/installation.rst b/docs/user/installation.rst index 8de1ff39c..6fc8bb13c 100644 --- a/docs/user/installation.rst +++ b/docs/user/installation.rst @@ -32,11 +32,11 @@ Supported and recommended operating systems are: * CentOS 7 and 8 * Debian 10 Buster and 11 Bullseye -* openSUSE Leap 15.2, 15.13 and Tumbleweed +* openSUSE Leap 15.3, 15.4 and Tumbleweed * Ubuntu: 18.04, 20.04 * For the Docker-installation: Docker Engine: 18.x and higher -Other distributions which are (most probably) supported include RHEL, Fedora and FreeBSD 12. +Other distributions which are (most probably) supported include AlmaLinux, Fedora, FreeBSD 12, RHEL and RockyLinux. A short guide on hardware requirements can be found on the page :doc:`hardware-requirements`. @@ -50,8 +50,10 @@ These are the operating systems which are currently supported by packages: * **CentOS 8** (run ``dnf install epel-release`` first) * **Debian 10** Buster * **Debian 11** Bullseye +* **Fedora 34** * **openSUSE Leap 15.2** * **openSUSE Leap 15.3** (make sure the ``openSUSE:Backports:SLE-15-SP3`` repository is enabled) +* **openSUSE Leap 15.4** * **openSUSE Tumbleweed** * **Ubuntu 18.04** Bionic Beaver (enable the universe repositories by appending ``universe`` in ``/etc/apt/sources.list`` to ``deb http://[...].archive.ubuntu.com/ubuntu/ bionic main`` first) * **Ubuntu 20.04** Focal Fossa (enable the universe repositories by appending ``universe`` in ``/etc/apt/sources.list`` to ``deb http://[...].archive.ubuntu.com/ubuntu/ focal main`` first) From 64fe142ff7592c6ed72dea4ed0ca3ed9d67b3cfb Mon Sep 17 00:00:00 2001 From: Sebastian Wagner Date: Mon, 8 Aug 2022 11:58:04 +0200 Subject: [PATCH 2/4] TST: open ctip azure test data as UTF8 failed on systems with default encoding = ASCII or others --- intelmq/tests/bots/parsers/microsoft/test_parser_ctip_azure.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/intelmq/tests/bots/parsers/microsoft/test_parser_ctip_azure.py b/intelmq/tests/bots/parsers/microsoft/test_parser_ctip_azure.py index a199a194e..72616b99a 100644 --- a/intelmq/tests/bots/parsers/microsoft/test_parser_ctip_azure.py +++ b/intelmq/tests/bots/parsers/microsoft/test_parser_ctip_azure.py @@ -3,6 +3,7 @@ # SPDX-License-Identifier: AGPL-3.0-or-later # -*- coding: utf-8 -*- +import codecs import json import os import unittest @@ -12,7 +13,7 @@ from intelmq.lib.utils import base64_encode -with open(os.path.join(os.path.dirname(__file__), 'ctip_azure.txt')) as handle: +with codecs.open(os.path.join(os.path.dirname(__file__), 'ctip_azure.txt'), encoding='utf8') as handle: EXAMPLE_DATA = handle.read() EXAMPLE_LINES = EXAMPLE_DATA.splitlines() EXAMPLE_PARSED = [json.loads(EXAMPLE_LINES[0]), json.loads(EXAMPLE_LINES[1])] From de604cd95dbf6126aa5ec657ff655d830bc48783 Mon Sep 17 00:00:00 2001 From: Sebastian Wagner Date: Mon, 8 Aug 2022 11:39:47 +0200 Subject: [PATCH 3/4] doc: known issues in changelog, clean up NEWS --- CHANGELOG.md | 17 +++++++++++++++++ NEWS.md | 28 +++------------------------- 2 files changed, 20 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 690fd7e01..c6504e68e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -132,6 +132,23 @@ CHANGELOG - logrotate: Move compress and ownership rules to the IntelMQ-blocks to prevent that they apply to other files (PR#2111 by Sebastian Wagner, fixes #2110). ### Known issues +This is short list of the most important known issues. The full list can be retrieved from [GitHub](https://github.com/certtools/intelmq/labels/bug?page=2&q=is%3Aopen+label%3Abug). +- intelmq_psql_initdb does not work for SQLite (#2202). +- SyntaxError in bots causes intelmqctl check to crash (#2177). +- intelmqctl create log file before dropping privileges (#2176). +- intelmqsetup: should install a default state file (#2175). +- Misp Expert - Crash if misp event already exist (#2170). +- Turris greylist has been updated (#2167). +- Spamhaus CERT parser uses wrong field (#2165). +- Custom headers ignored in HTTPCollectorBot (#2150). +- Missing commas in SQL query for separate Events table (#2125). +- intelmqctl log: parsing syslog does not work (#2097). +- Bash completion scripts depend on old JSON-based configuration files (#2094). +- Bot configuration examples use JSON instead of YAML (#2066). +- intelmqdump: logging_path parameter not honoured (#1605). +- CSV line recovery forces Windows line endings (#1597). +- Bots started with IntelMQ-API/Manager stop when the webserver is restarted (#952). +- Corrupt dump files when interrupted during writing (#870). 3.0.2 (2021-09-10) diff --git a/NEWS.md b/NEWS.md index 780a8e52b..e29125a5d 100644 --- a/NEWS.md +++ b/NEWS.md @@ -7,33 +7,20 @@ NEWS ==== This file lists all changes which have an affect on the administration of IntelMQ and contains steps that you need to be aware off for the upgrade. -Please refer to the changelog for a full list of changes. +Please refer to the change log for a full list of changes. 3.1.0 Feature release (unreleased) ---------------------------------- -### Requirements - ### Bots #### ShadowServer Reports API collector The misleading `country` parameter has been depreciated and a `reports` parameter has been added. The backwards-compatibility will be removed in IntelMQ version 4.0.0. See the [Shadowserver Reports API bot's documentation](https://intelmq.readthedocs.io/en/latest/user/bots.html#shadowserver-reports-api). - -3.1.0 Feature release (unreleased) ----------------------------------- - -### Requirements - -### Bots -#### ShadowServer Reports API collector -The misleading `country` parameter has been depreciated and a `reports` parameter has been added. -The backwards-compatibility will be removed in IntelMQ version 4.0.0. -See the [Shadowserver Reports API bot's documentation](https://intelmq.readthedocs.io/en/latest/user/bots.html#shadowserver-reports-api). - -### Tools +#### GitHub Collector +GitHub removed the basic `Username/Password` Authentication in favor of personal access tokens. So the GitHub Collector uses an Personal Access Token for authentication [Github Documentation: Generate a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) ### Data Format #### Field name checks @@ -53,8 +40,6 @@ sudo find /var/log/ -user intelmq ! -path \*intelmq\* #### Threshold Expert The parameter `timeout` has been merged into `redis_cache_ttl`. -### Libraries - ### Postgres databases The following statements optionally update existing data for the harmonization classification changes: ```sql @@ -142,13 +127,6 @@ UPDATE events ``` -### Bots - -#### Github Collector -GitHub removed the basic `Username/Password` Authentication in favor of personal access tokens. So the GitHub Collector uses an Personal Access Token for authentication [Github Documentation: Generate a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) -======= - - 3.0.2 Maintenance release (2021-09-10) -------------------------------------- Two performance issues were fixed. One affected all collectors which processed high volumes of data and the other issue affected some bots which used threading. From 91c5cc33ff734bbeb595f3a0645e3334e7af0319 Mon Sep 17 00:00:00 2001 From: Sebastian Wagner Date: Mon, 8 Aug 2022 11:42:24 +0200 Subject: [PATCH 4/4] REL: 3.1.0 Release candidate 1 --- debian/changelog | 6 +++--- intelmq/version.py | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/debian/changelog b/debian/changelog index 1c78d09cd..d9b2a8387 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,8 +1,8 @@ -intelmq (3.1.0~alpha1-1) UNRELEASED; urgency=medium +intelmq (3.1.0~rc1-1) stable; urgency=medium - * Start 3.1.0 development. + * 3.1.0 Release candidate 1. - -- Sebastian Wagner Fri, 13 Aug 2021 11:09:58 +0200 + -- Sebastian Wagner Mon, 08 Aug 2022 11:41:15 +0200 intelmq (3.0.2-1) stable; urgency=medium diff --git a/intelmq/version.py b/intelmq/version.py index dd62ffa2c..a028ab811 100644 --- a/intelmq/version.py +++ b/intelmq/version.py @@ -1,6 +1,6 @@ -# SPDX-FileCopyrightText: 2016-2021 Sebastian Wagner +# SPDX-FileCopyrightText: 2016-2022 Sebastian Wagner # # SPDX-License-Identifier: AGPL-3.0-or-later -__version_info__ = (3, 1, 0, 'alpha1') +__version_info__ = (3, 1, 0, 'rc1') __version__ = '.'.join(map(str, __version_info__))