[CE] An NPE is thrown if port 1883 is in use when HiveMQ CE is started #424
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tgracchus
changed the title
mchernyakov
reviewed
Oct 4, 2023
mchernyakov
approved these changes
Oct 4, 2023
hlohse
reviewed
Oct 5, 2023
Co-authored-by: Henning Lohse <[email protected]>
hlohse
approved these changes
Oct 5, 2023
cmkrafft
pushed a commit
to com2m/hivemq-community-edition
that referenced
this pull request
Jan 16, 2024
* update version to 2022.2-SNAPSHOT * Update dependency versions (hivemq#346) * update Netty and Jackson version * update dependency check plugin * update dependencies (hivemq#347) * update not critical dependencies * refactored FutureUtils * Refactor DiagnosticModeTest to use mock(...) instead of @mock. * Add SystemProperties.DIAGNOSTIC_MODE replacing diagnosticMode * Fix SIGSEGV during shutdown with enabled diagnostics mode This was caused by DiagnosticMode's ConsoleReporter which periodically writes metrics to disk and must not access any RocksDB instance after it has been closed already. Therefore, stop DiagnosticMode and its ConsoleReporter's executore first via ShutdownHooks. * fixed extension priority comparator for multiple entries with same priorities. * Reduce Garbage Collection during incoming subscribe by removing wrapper classes like ClientToken around ClientConnection. * fix duplicate extension starts (hivemq#361) * fix naming of extension task executor (hivemq#362) * add XSD for broker config.xml * add missing license header * copy xsd file to conf folder on packaging * move defaults to documentation to prevent confusing empty values * Refactor SslContextStore * Fix not sending will on clean start during re-connect If a client re-connects with a clean start, the specification defines that this "ends" and existing sessions. Whenever a session ends, the specification defines that the ending session's will must be sent, if available and still pending due to a session expiry interval or a will delay interval that has not been reached yet. This is achieved by sending any still pending will between disconnecting and (re-)connecting. Also improved method names of PendingWillMessages. * Send any pending wills on session expiry Don't rely on the CheckWillsTask and ClientSessionCleanUpTask to be scheduled well enough that there can never be a situation where there is still a will send pending while the session, which contains the will data, has already been cleaned up. * Log a warning if a will can't be sent because the session is gone For testing purposes, assert that this never happens because it is unexpected: We must guarantee that a will is sent before the session is gone/cleaned up. To detect unknown race conditions in production, log a warning and don't throw a NullPointerException, as this might block client connections and the sending of other wills. * Fix will count metrics on client re-connect or takeover with cleanStart=true * Fix will message disk leak on client re-connect or takeover Added commit hooks to update will message references whenever a client session is removed or overwritten. * Reduce used message IDs memory footprint Switched from HashSet<Integer> to Eclipse Collections' ShortHashSet to reduce the memory footprint. We can use ShortHashSet instead of IntHashSet because the range of IDs, 1..65535, is unsigned short anyway, and we can shift those int values into signed short range. * Removed obsolete @author comment * Fix leak of CONNECT in ClientConnection if no plugin initializers exist * [CE] Refactoring of the topic tree structures and a performance improvement (hivemq#373) * Refactoring of the topic tree and performance improvement for mapping PUBLISHes to the shared subscriptions. Co-authored-by: A-Imal <[email protected]> * Choose shared subscriber via randomly-ordered iteration Avoid shuffling a copy of the collection to reduce CPU load. * Remove desiredExtensionClass parameter from extension loading * Catch errors thrown by ExtensionMain classes during extension loading * Reduce BucketLock contention by RemoveEntryTask Added a guard so that concurrently running RemoveEntryTasks don't try to process entries of the shared removablePayloads queue. Since this is more of a workaround, the structure should be improved in the scope of the ticket #11000. * Remove TODO Ticket number is enough. * Add TRACE logs for 'no matching subscriber' case * Refactoring of the utils (hivemq#378) Removed utils: ChannelAttributes ChannelUtils ClientSessions DefaultSslEngineUtil MacAddressFormatter PhysicalMemoryUtil PublishUtil ResourceUtils TemporaryFileUtils UcaughtExceptionHandler Validators * Remove payloadCache Leave the caching to RocksDB. * Refactor code, remove warnings * Fix and extend Exception handling test cases * renamed a class (hivemq#382) * Code refactoring, formatting * Catch Throwable to guarantee re-scheduling of CleanUpTask * Add timeout to CleanUpTask, schedule next once timed out Avoids situations where the cleanup seems stuck in case any persistence cleanup takes a long time or is actually stuck because of a bug. * Debug-log CancellationException of timeouts * fix reason string in event log * Version to 2023.1 * Version to 2023.1 in README.adoc * Update third party licenses * Remove sdk checkout from publish workflow (use release sdk version) * Remove redundant addition, use preconditions * Remove warnings * Fix a race of polling duplicate messages, overwriting futures Might happen if previous sends finish concurrently and decrement the in-flight counter to 0. This would trigger message polling which overwrites existing futures of previous polling(s). * Share UnsubscribeHandler between channels/client connections * Remove warnings, reformat code * refactoring * fixing circular dependency and broken unit test * Fix shields.io build badge * Update README.adoc Co-authored-by: Silvio Giebl <[email protected]> * Introduced and applied .editorconfig. * Optimize imports. * Remove connectionAttributesMutex which can be avoided by making the set method synchronized from the ClientConnection. * Pass-through the cleanStart flag directly to EventLog. * Introduced ClientConnectionContext which is used for client entry points like Netty handlers where the state of the client can be undefined (connecting). * Avoid creating multiple Netty attributes for ClientConnectionContext. * Annotate fields in ClientConnection with NotNull when we are sure that they are always available. * Remove unused methods in UndefinedClientConnection. * Formatting. * Remove unused CONNACKs from ConnackMessages. * Move the builder of CONNACK out of the class for a better overview and isolation of the classes. * Use for any instance creation of CONNACK the builder class. Removes all other constructors. * Move the precondition check of CONNACK to the build method. * Fix failing tests. * Formatting. * Version to 2023.2 * Fix race where a QoS 0 payload gets deleted concurrently during polling (hivemq#394) Previously, the payload reference counter was decremented immediately after polling the QoS 0 PUBLISH from its queue, but before assigning the payload to the PUBLISH again. This introduced a race condition with the payload cleanup which was then permitted to delete the payload concurrently. Therefore, assign the payload during lookup to fix the race. Additionally, drop messages without payload immediately during iteration; don't schedule another single writer task. * LocalTopicTree > Avoid iterating over the root wildcards when there is nothing to remove. * Version to 2023.3 * [CE] Feature-id 14176: TLS log messages port (hivemq#400) * added information to tls logs * fix license header * remove wrong since * Refactoring: Remove warnings * Fix IntelliJ typo warnings * Use exclusive and readonly transactions - Exclusive transactions can be used by these methods because all calls are serialized by the single writer - Some transactions are only read-only * Move metrics, payload and event log updates to transaction hooks This makes changes only visible after the transaction succeeds. Avoids intermediate state during e.g. loops in a transaction. * Refactoring: Remove warnings * Fix IntelliJ typo warnings * Make removeWithTimestamp only visible for testing There is no public calling/use case. * Use single Runnable commit hook where possible * Update session counters via accumulated delta * Fix wrong session count on start with persistent session * Version to 2023.4 * Update dependencies (hivemq#404) * Update gradle to 7.4.2 * Update io.github.sgtsilvio.gradle.utf8 -> defaults * Cleanup test logging * Replace license-third-party-tool with buildSrc plugin * Update third party licenses * CVE-2023-2976, Guava update to recommended version 32.0.1-jre * CVE: CVE-2023-34462, Netty update to recommended version 4.1.94.Final * reauth > fix missing client state update * Update version (hivemq#413) * Formatting and cleanup for better readability. * RocksDBLocalPersistence > Reduce number of returns. * Fix memTable typo. * RetainedMessagePersistence > Remove unnecessary try-catch guards. * ClientQueueLocalPersistence > Removed unused qos0Size method. * reference counting > Remove unnecessary delta arguments. * RetainedMessageLocalPersistence > Cleanup. * Fix resource leakage. * reference counting > Add and decrement payloads for queue entries in the ClientQueuePersistence. * reference counting > Remove workarounds for retained message payloads which required before some retries until the payloads were persisted on the disk. Currently, we are persisting the payloads in the same single writer tasks to the persistence which guarantees us to find the payload when we can find the retained message itself. * PublishPayloadXodusLocalPersistence > Use exclusive transaction for write operations. * Remove unused PayloadCacheRemovalListener. * PayloadPersistence > Cleanup and formatting. * PayloadPersistence > Make methods in PublishPayloadXodusSerializer static. * PublishPayloadPersistence > Remove test only method from the interface. * PublishPayloadPersistence > Rework the payload cleanup job: Threads are assigned to buckets for which they are responsible. * reference counting > Remove usage of PayloadPersistenceException when payload was not found. * reference counting > Removed any calls to the payload persistence for retained messages, all adds and removes of payloads are only performed in the RetainedMessageLocalPersistence. * Fix RetainedMessageRocksDBLocalPersistence not adding the payload when replacing an existing retained message. * Fix EmbeddedHiveMQ leakage in test code. * Update gradle to 7.6.2 * Update io.github.sgtsilvio.gradle plugins * Update bouncycastle jdk15on -> jdk18on * Bugfix for expired inflight PUBREL not getting removed from the queue (hivemq#417) Expired inflight PUBREL gets default values for publish timestamp and message expiry to avoid stuck message queue; added inflight PUBREL expiry config. * Version to 2023.6 * Update base image to eclipse temurin (hivemq#419) * Version to 2023.7 * [CE] An NPE is thrown if port 1883 is in use when HiveMQ CE is started (hivemq#424) Co-authored-by: Henning Lohse <[email protected]> * Version to 2023.8 * Use version catalog (hivemq#425) * Ranged IDs implementation for packet IDs (hivemq#426) * Ranged IDs implementation for assigning packet IDs * [1429] A failed CONNACK should be sent, when a CONNECT packet exceed the max packet size version 2 (hivemq#423) Co-authored-by: Mikhail Chernyakov <[email protected]> Co-authored-by: Vladimir Podolskiy <[email protected]> * NoMessageIdAvailableException is thrown even if there are free ids available in the pool (hivemq#428) * FreePacketIdRanges > Making class thread-safe and fixing an edge case when an empty id range could become a root range. * Update version to 2023.9 (hivemq#430) * Update third party licenses (hivemq#431) * Logging > The current TurboFilterList implementation of logback-classic which calls the TurboFilters is a CopyOnWriteArrayList. This means every time any logger method is called the array of TurboFilters is copied. As we are using the logger extensively this small copy is visible in the flamegraph. A call to isXLevelEnabled check does not avoid this path. * make JMX port configurable * JMX_PORT -> HIVEMQ_JMX_PORT * Update logback, update apache-commmons-compress * Version to 2023.10 * Logging > Register our TurboFilter only once. During the HiveMQ start we load our logback.xml which will reset the current context and call the reset which will register the TurboFilter a second time. * Logging > corrected and added logged statements about logging configuration * LoggingBootstrap > formatting * LoggingBootstrap > formatting * SingleWriterService > Handle all exceptions that can be thrown during the task execution. This allows all futures to finish and issues being visible. * fixed missing empty string handling for MQTT3 Publish topic and MQTT5 Publish response topic * added empty string handling for MQTT5 will response topic and improved logging and reason strings. * Version to 2024.1 * Fixed mac build * Updated version --------- Co-authored-by: Daniel Krueger <[email protected]> Co-authored-by: Daniel Krüger <[email protected]> Co-authored-by: Lukas Brandl <[email protected]> Co-authored-by: Henning Lohse <[email protected]> Co-authored-by: flimpoec <[email protected]> Co-authored-by: A-Imal <[email protected]> Co-authored-by: Christoph Schaebel <[email protected]> Co-authored-by: Mario Schwede <[email protected]> Co-authored-by: Vladimir Podolskiy <[email protected]> Co-authored-by: David Sondermann <[email protected]> Co-authored-by: Michael Walter <[email protected]> Co-authored-by: gitseti <[email protected]> Co-authored-by: Silvio Giebl <[email protected]> Co-authored-by: Peter Giacomo Lombardo <[email protected]> Co-authored-by: Yonathan Negussie Mengesha <[email protected]> Co-authored-by: Antonio Fernandez Alhambra <[email protected]> Co-authored-by: tgracchus <[email protected]> Co-authored-by: Mikhail Chernyakov <[email protected]> Co-authored-by: yannick <[email protected]> Co-authored-by: Christoph Kempfer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Resolves #403
and
https://hivemq.kanbanize.com/ctrl_board/42/cards/16936/details/
Changes