KAFKA-18472: Remove MetadataSupport #18483

m1a2st · 2025-01-10T16:55:56Z

Jira: https://issues.apache.org/jira/browse/KAFKA-18472

Committer Checklist (excluded from commit message)

Verify design and implementation
Verify test coverage and CI build status
Verify documentation (including upgrade notes)

# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

chia7712 · 2025-01-15T10:55:17Z

@m1a2st please fix conflicts

# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

ijuma · 2025-01-15T14:14:03Z

core/src/main/scala/kafka/server/KafkaApis.scala

-            new DescribeClientQuotasResponse(result)
-          })
-      }
+      val result = metadataCache.asInstanceOf[KRaftMetadataCache]describeClientQuotas(describeClientQuotasRequest.data())


Do we have a JIRA for removing this cast? There should be no reason to do casts anymore since the only implementation we have is this one.

There is a Jira to remove zkMetadataCache, I think we should also addressed it in this Jira

Sounds good.

chia7712 · 2025-01-15T15:37:28Z

@m1a2st please fix the conflicts

# Conflicts: # core/src/test/scala/unit/kafka/server/KafkaApisTest.scala

chia7712 · 2025-01-15T17:34:51Z

I believe we can eliminate MetadataSupport entirely, as KafkaApis can directly utilize forwardingManager. This would streamline the codebase.

m1a2st · 2025-01-15T17:37:30Z

It make sense to me, I will change to remove all MetadataSupport

chia7712 · 2025-01-15T18:31:24Z

core/src/main/scala/kafka/server/KafkaApis.scala

@@ -2665,7 +2632,6 @@ class KafkaApis(val requestChannel: RequestChannel,
            requestHelper.sendMaybeThrottle(request, subscriptionRequest.getErrorResponse(Errors.INVALID_REQUEST.exception))
        }
      case None =>
-        info("Received get telemetry client request for zookeeper based cluster")


Could you please revert this change as it is included by #18550?

chia7712 · 2025-01-15T18:31:27Z

core/src/main/scala/kafka/server/KafkaApis.scala

@@ -2682,7 +2648,6 @@ class KafkaApis(val requestChannel: RequestChannel,
            requestHelper.sendMaybeThrottle(request, pushTelemetryRequest.getErrorResponse(Errors.INVALID_REQUEST.exception))
        }
      case None =>
-        info("Received push telemetry client request for zookeeper based cluster")


chia7712 · 2025-01-15T18:31:32Z

core/src/main/scala/kafka/server/KafkaApis.scala

          // but we should handle it gracefully.
-          info("Received list client metrics resources request for zookeeper based cluster")


chia7712 · 2025-01-15T18:31:38Z

core/src/main/scala/kafka/server/KafkaApis.scala

@@ -3120,7 +3083,6 @@ class KafkaApis(val requestChannel: RequestChannel,
      case Some(manager) => manager
      case None =>
        // The API is not supported when the SharePartitionManager is not defined on the broker
-        info("Received share acknowledge request for zookeeper based cluster")


chia7712 · 2025-01-15T18:31:42Z

core/src/main/scala/kafka/server/KafkaApis.scala

@@ -2800,7 +2764,6 @@ class KafkaApis(val requestChannel: RequestChannel,
      case Some(manager) => manager
      case None =>
        // The API is not supported when the SharePartitionManager is not defined on the broker
-        info("Received share fetch request for zookeeper based cluster")


# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

ijuma

LGTM, thanks.

ijuma · 2025-01-16T03:37:34Z

core/src/main/scala/kafka/server/KafkaApis.scala

-            new DescribeClientQuotasResponse(result)
-          })
-      }
+      val result = metadataCache.asInstanceOf[KRaftMetadataCache].describeClientQuotas(describeClientQuotasRequest.data())


A note for someone who sees this - there is a plan to address these casts separately. My suggestion elsewhere is to simply move the relevant methods to the base interface and then the cast is no longer needed.

Thanks for @ijuma reminder, cc @FrankYang0529

Reviewers: Ismael Juma <[email protected]>, Chia-Ping Tsai <[email protected]>

clean up MetadataSupport

822f9ee

github-actions bot added triage PRs from the community core Kafka Broker labels Jan 10, 2025

m1a2st changed the title ~~KAFKA-18472: Remove ZkSupport and cleanup MetadataSupport~~ [WIP]KAFKA-18472: Remove ZkSupport and cleanup MetadataSupport Jan 11, 2025

chia7712 mentioned this pull request Jan 11, 2025

KAFKA-18399: Remove ZooKeeper from KafkaApis (3/N): USER_SCRAM_CREDENTIALS #18456

Merged

3 tasks

m1a2st added 3 commits January 12, 2025 02:06

Merge branch 'trunk' into KAFKA-18472

3fa8ef5

# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

Merge branch 'trunk' into KAFKA-18472

5301e43

# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

rebase code

7ef516f

chia7712 added the ci-approved label Jan 15, 2025

m1a2st added 3 commits January 15, 2025 19:33

Merge branch 'trunk' into KAFKA-18472

ebd91a2

# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

resolve the conflict

a4f02e6

fix the error test

e0ac001

m1a2st changed the title ~~[WIP]KAFKA-18472: Remove ZkSupport and cleanup MetadataSupport~~ KAFKA-18472: Remove ZkSupport and cleanup MetadataSupport Jan 15, 2025

m1a2st mentioned this pull request Jan 15, 2025

KAFKA-18533: Remove KafkaConfig zookeeper related logic #18547

Merged

3 tasks

ijuma reviewed Jan 15, 2025

View reviewed changes

m1a2st added 2 commits January 15, 2025 22:20

Merge branch 'trunk' into KAFKA-18472

6cb3743

remove tests in KafkaApisTest

18b2574

Merge branch 'trunk' into KAFKA-18472

85ea75a

# Conflicts: # core/src/test/scala/unit/kafka/server/KafkaApisTest.scala

github-actions bot added the small Small PRs label Jan 15, 2025

m1a2st mentioned this pull request Jan 15, 2025

KAFKA-18406 Remove ZkBrokerEpochManager #18561

Merged

3 tasks

m1a2st changed the title ~~KAFKA-18472: Remove ZkSupport and cleanup MetadataSupport~~ KAFKA-18472: Remove MetadataSupport Jan 15, 2025

addressed comments

0b62e36

github-actions bot added performance and removed small Small PRs labels Jan 15, 2025

m1a2st added 2 commits January 16, 2025 02:03

addressed comments

615eff1

remove zookeeper log

c4642fc

chia7712 reviewed Jan 15, 2025

View reviewed changes

Merge branch 'trunk' into KAFKA-18472

eb41034

# Conflicts: # core/src/main/scala/kafka/server/KafkaApis.scala

github-actions bot removed the triage PRs from the community label Jan 16, 2025

ijuma approved these changes Jan 16, 2025

View reviewed changes

ijuma reviewed Jan 16, 2025

View reviewed changes

ijuma merged commit b9ccab4 into apache:trunk Jan 16, 2025
9 checks passed

ijuma pushed a commit that referenced this pull request Jan 16, 2025

KAFKA-18472: Remove MetadataSupport (#18483)

77018f0

Reviewers: Ismael Juma <[email protected]>, Chia-Ping Tsai <[email protected]>

askew pushed a commit to askew/kafka that referenced this pull request Jan 23, 2025

KAFKA-18472: Remove MetadataSupport (apache#18483)

f8d3ffe

Reviewers: Ismael Juma <[email protected]>, Chia-Ping Tsai <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KAFKA-18472: Remove MetadataSupport #18483

KAFKA-18472: Remove MetadataSupport #18483

m1a2st commented Jan 10, 2025 •

edited by ijuma

Loading

chia7712 commented Jan 15, 2025

ijuma Jan 15, 2025 •

edited

Loading

m1a2st Jan 15, 2025

ijuma Jan 15, 2025

chia7712 commented Jan 15, 2025

chia7712 commented Jan 15, 2025

m1a2st commented Jan 15, 2025

chia7712 Jan 15, 2025

chia7712 Jan 15, 2025

chia7712 Jan 15, 2025

chia7712 Jan 15, 2025

chia7712 Jan 15, 2025

ijuma left a comment

ijuma Jan 16, 2025

m1a2st Jan 16, 2025

		// but we should handle it gracefully.
		info("Received list client metrics resources request for zookeeper based cluster")

KAFKA-18472: Remove MetadataSupport #18483

KAFKA-18472: Remove MetadataSupport #18483

Conversation

m1a2st commented Jan 10, 2025 • edited by ijuma Loading

Committer Checklist (excluded from commit message)

chia7712 commented Jan 15, 2025

ijuma Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chia7712 commented Jan 15, 2025

chia7712 commented Jan 15, 2025

m1a2st commented Jan 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ijuma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m1a2st commented Jan 10, 2025 •

edited by ijuma

Loading

ijuma Jan 15, 2025 •

edited

Loading