Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify proto options and do not generate unused fields #1275

Merged
merged 4 commits into from
Nov 13, 2024

Conversation

overvenus
Copy link
Member

@overvenus overvenus commented Nov 5, 2024

Unify proto options and do not generate unused fields by setting following options.

option (gogoproto.goproto_unkeyed_all) = false;
option (gogoproto.goproto_unrecognized_all) = false;
option (gogoproto.goproto_sizecache_all) = false;

Tests:

# pwd: kvproto
cd ../kvproto
git diff
diff --git a/go.mod b/go.mod
index 5dedb0a4f1..7671cd8042 100644
--- a/go.mod
+++ b/go.mod
@@ -326,4 +326,6 @@ replace (
        // Please remove these dependencies.
        sourcegraph.com/sourcegraph/appdash => github.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0
        sourcegraph.com/sourcegraph/appdash-data => github.com/sourcegraph/appdash-data v0.0.0-20151005221446-73f23eafcf67
+
+        github.com/pingcap/kvproto => ../kvproto
 )
make       
CGO_ENABLED=1  GO111MODULE=on go build -tags codes  -ldflags '-X "github.com/pingcap/tidb/pkg/parser/mysql.TiDBReleaseVersion=v8.5.0-alpha-41-g4cfe22cfa0-dirty" -X "github.com/pingcap/tidb/pkg/util/versioninfo.TiDBBuildTS=2024-11-05 07:39:00" -X "github.com/pingcap/tidb/pkg/util/versioninfo.TiDBGitHash=4cfe22cfa02490ed8e574f86a54d366404e111e1" -X "github.com/pingcap/tidb/pkg/util/versioninfo.TiDBGitBranch=master" -X "github.com/pingcap/tidb/pkg/util/versioninfo.TiDBEdition=Community" ' -o bin/tidb-server ./cmd/tidb-server
Build TiDB Server successfully!

Cc pingcap/tipb#347 a complement PR for tipb.

Close #1278

Signed-off-by: Neil Shen <[email protected]>
* Do not generate XXX_ fields for Go protobuf structs
* Enable lite runtime for Rust protobuf files

Signed-off-by: Neil Shen <[email protected]>
Signed-off-by: Neil Shen <[email protected]>
@ti-chi-bot ti-chi-bot bot added the size/XXL label Nov 5, 2024
@ti-chi-bot ti-chi-bot bot requested a review from TszKitLo40 November 5, 2024 07:43
Signed-off-by: Neil Shen <[email protected]>
Copy link
Contributor

@cfzjywxk cfzjywxk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

ti-chi-bot bot commented Nov 5, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-11-05 09:21:14.388885656 +0000 UTC m=+946387.228041203: ☑️ agreed by cfzjywxk.
  • 2024-11-05 09:22:56.178159794 +0000 UTC m=+946489.017315339: ☑️ agreed by BornChanger.

@overvenus
Copy link
Member Author

/approve

@zhangjinpeng87
Copy link
Contributor

What will happen during upgrading if new version add a new filed in the proto, and old version received the new message? @overvenus

@overvenus
Copy link
Member Author

What will happen during upgrading if new version add a new filed in the proto, and old version received the new message?

If the old version receives the new message, it will discard new fields unless goproto_unrecognized_all is set to true. In that case, new fields will be stored in the XXX_unrecognized []byte field in binary form.

Both TiDB, PD and their tools are not using the XXX_unrecognized, making it essentially dead code, which is ok to remove.

Copy link
Contributor

@zhangjinpeng87 zhangjinpeng87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@you06 you06 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR LGTM, but pay attention that the padding in TiDB need to be removed

https://github.com/pingcap/tidb/blob/5129bb37dd49d38f0d174e327c92345b4565caf6/pkg/kv/key.go#L102-L104

Copy link

ti-chi-bot bot commented Nov 13, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BornChanger, cfzjywxk, flowbehappy, MyonKeminta, nolouch, overvenus, tiancaiamao, you06, zhangjinpeng87

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [BornChanger,MyonKeminta,cfzjywxk,overvenus,tiancaiamao,you06]
  • proto/OWNERS [BornChanger,MyonKeminta,cfzjywxk,flowbehappy,nolouch,overvenus,tiancaiamao,you06,zhangjinpeng87]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Nov 13, 2024
@ti-chi-bot ti-chi-bot bot merged commit e1fa7ea into pingcap:master Nov 13, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unify protobuf generation options
9 participants