Merge pull request #2230 from certtools/310rc1

3.1.0 Release candidate 1

CHANGELOG.md

@@ -132,6 +132,23 @@ CHANGELOG
 - logrotate: Move compress and ownership rules to the IntelMQ-blocks to prevent that they apply to other files (PR#2111 by Sebastian Wagner, fixes #2110).
 
 ### Known issues
+This is short list of the most important known issues. The full list can be retrieved from [GitHub](https://github.com/certtools/intelmq/labels/bug?page=2&q=is%3Aopen+label%3Abug).
+- intelmq_psql_initdb does not work for SQLite (#2202).
+- SyntaxError in bots causes intelmqctl check to crash (#2177).
+- intelmqctl create log file before dropping privileges (#2176).
+- intelmqsetup: should install a default state file (#2175).
+- Misp Expert - Crash if misp event already exist (#2170).
+- Turris greylist has been updated (#2167).
+- Spamhaus CERT parser uses wrong field (#2165).
+- Custom headers ignored in HTTPCollectorBot (#2150).
+- Missing commas in SQL query for separate Events table (#2125).
+- intelmqctl log: parsing syslog does not work (#2097).
+- Bash completion scripts depend on old JSON-based configuration files (#2094).
+- Bot configuration examples use JSON instead of YAML (#2066).
+- intelmqdump: logging_path parameter not honoured (#1605).
+- CSV line recovery forces Windows line endings (#1597).
+- Bots started with IntelMQ-API/Manager stop when the webserver is restarted (#952).
+- Corrupt dump files when interrupted during writing (#870).
 
 
 3.0.2 (2021-09-10)

NEWS.md

@@ -7,33 +7,20 @@ NEWS
 ====
 
 This file lists all changes which have an affect on the administration of IntelMQ and contains steps that you need to be aware off for the upgrade.
-Please refer to the changelog for a full list of changes.
+Please refer to the change log for a full list of changes.
 
 
 3.1.0 Feature release (unreleased)
 ----------------------------------
 
-### Requirements
-
 ### Bots
 #### ShadowServer Reports API collector
 The misleading `country` parameter has been depreciated and a `reports` parameter has been added.
 The backwards-compatibility will be removed in IntelMQ version 4.0.0.
 See the [Shadowserver Reports API bot's documentation](https://intelmq.readthedocs.io/en/latest/user/bots.html#shadowserver-reports-api).
 
-
-3.1.0 Feature release (unreleased)
-----------------------------------
-
-### Requirements
-
-### Bots
-#### ShadowServer Reports API collector
-The misleading `country` parameter has been depreciated and a `reports` parameter has been added.
-The backwards-compatibility will be removed in IntelMQ version 4.0.0.
-See the [Shadowserver Reports API bot's documentation](https://intelmq.readthedocs.io/en/latest/user/bots.html#shadowserver-reports-api).
-
-### Tools
+#### GitHub Collector
+GitHub removed the basic `Username/Password` Authentication in favor of personal access tokens. So the GitHub Collector uses an Personal Access Token for authentication [Github Documentation: Generate a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
 
 ### Data Format
 #### Field name checks
@@ -53,8 +40,6 @@ sudo find /var/log/ -user intelmq ! -path \*intelmq\*
 #### Threshold Expert
 The parameter `timeout` has been merged into `redis_cache_ttl`.
 
-### Libraries
-
 ### Postgres databases
 The following statements optionally update existing data for the harmonization classification changes:
 ```sql
@@ -142,13 +127,6 @@ UPDATE events
 ```
 
 
-### Bots
-
-#### Github Collector
-GitHub removed the basic `Username/Password` Authentication in favor of personal access tokens. So the GitHub Collector uses an Personal Access Token for authentication [Github Documentation: Generate a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
-=======
-
-
 3.0.2 Maintenance release (2021-09-10)
 --------------------------------------
 Two performance issues were fixed. One affected all collectors which processed high volumes of data and the other issue affected some bots which used threading.

debian/changelog

@@ -1,8 +1,8 @@
-intelmq (3.1.0~alpha1-1) UNRELEASED; urgency=medium
+intelmq (3.1.0~rc1-1) stable; urgency=medium
 
-  * Start 3.1.0 development.
+  * 3.1.0 Release candidate 1.
 
- -- Sebastian Wagner <[email protected]>  Fri, 13 Aug 2021 11:09:58 +0200
+ -- Sebastian Wagner <[email protected]>  Mon, 08 Aug 2022 11:41:15 +0200
 
 intelmq (3.0.2-1) stable; urgency=medium
 

docs/user/installation.rst

@@ -32,11 +32,11 @@ Supported and recommended operating systems are:
 
 * CentOS 7 and 8
 * Debian 10 Buster and 11 Bullseye
-* openSUSE Leap 15.2, 15.13 and Tumbleweed
+* openSUSE Leap 15.3, 15.4 and Tumbleweed
 * Ubuntu: 18.04, 20.04
 * For the Docker-installation: Docker Engine: 18.x and higher
 
-Other distributions which are (most probably) supported include RHEL, Fedora and FreeBSD 12.
+Other distributions which are (most probably) supported include AlmaLinux, Fedora, FreeBSD 12, RHEL and RockyLinux.
 
 A short guide on hardware requirements can be found on the page :doc:`hardware-requirements`.
 
@@ -50,8 +50,10 @@ These are the operating systems which are currently supported by packages:
 * **CentOS 8** (run ``dnf install epel-release`` first)
 * **Debian 10** Buster
 * **Debian 11** Bullseye
+* **Fedora 34**
 * **openSUSE Leap 15.2**
 * **openSUSE Leap 15.3** (make sure the ``openSUSE:Backports:SLE-15-SP3`` repository is enabled)
+* **openSUSE Leap 15.4**
 * **openSUSE Tumbleweed**
 * **Ubuntu 18.04** Bionic Beaver (enable the universe repositories by appending ``universe`` in ``/etc/apt/sources.list`` to ``deb http://[...].archive.ubuntu.com/ubuntu/ bionic main`` first)
 * **Ubuntu 20.04** Focal Fossa (enable the universe repositories by appending ``universe`` in ``/etc/apt/sources.list`` to ``deb http://[...].archive.ubuntu.com/ubuntu/ focal main`` first)

intelmq/tests/bots/parsers/microsoft/test_parser_ctip_azure.py

@@ -3,6 +3,7 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 # -*- coding: utf-8 -*-
+import codecs
 import json
 import os
 import unittest
@@ -12,7 +13,7 @@
 from intelmq.lib.utils import base64_encode
 
 
-with open(os.path.join(os.path.dirname(__file__), 'ctip_azure.txt')) as handle:
+with codecs.open(os.path.join(os.path.dirname(__file__), 'ctip_azure.txt'), encoding='utf8') as handle:
     EXAMPLE_DATA = handle.read()
 EXAMPLE_LINES = EXAMPLE_DATA.splitlines()
 EXAMPLE_PARSED = [json.loads(EXAMPLE_LINES[0]), json.loads(EXAMPLE_LINES[1])]

intelmq/version.py

@@ -1,6 +1,6 @@
-# SPDX-FileCopyrightText: 2016-2021 Sebastian Wagner
+# SPDX-FileCopyrightText: 2016-2022 Sebastian Wagner
 #
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
-__version_info__ = (3, 1, 0, 'alpha1')
+__version_info__ = (3, 1, 0, 'rc1')
 __version__ = '.'.join(map(str, __version_info__))