Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](detailMessage) add AvailCapacity prompt in detailMessage #26328

Merged
merged 1 commit into from
Nov 17, 2023

Conversation

xingyingone
Copy link
Contributor

@xingyingone xingyingone commented Nov 2, 2023

Proposed changes

Issue Number: close #26253

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@xingyingone
Copy link
Contributor Author

run buildall

Copy link
Contributor

@alexxing662 alexxing662 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Nov 2, 2023

PR approved by anyone and no changes requested.

@@ -593,7 +593,7 @@ public Map<Tag, List<Long>> selectBackendIdsForReplicaCreation(
if (!failedEntries.isEmpty()) {
String failedMsg = Joiner.on("\n").join(failedEntries);
throw new DdlException("Failed to find enough backend, please check the replication num,"
+ "replication tag and storage medium.\n" + "Create failed replications:\n" + failedMsg);
+ "replication tag and storage medium and AvailCapacity.\n" + "Create failed replications:\n" + failedMsg);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ "replication tag and storage medium and AvailCapacity.\n" + "Create failed replications:\n" + failedMsg);
+ "replication tag and storage medium and avail capacity of backends.\n" + "Create failed replications:\n" + failedMsg);

@xingyingone
Copy link
Contributor Author

run buildall

morningman
morningman previously approved these changes Nov 6, 2023
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 6, 2023
Copy link
Contributor

github-actions bot commented Nov 6, 2023

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 6, 2023
@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.63 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 34 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162014358 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.6 seconds
stream load tsv: 575 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162491237 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.42 seconds
stream load tsv: 575 seconds loaded 74807831229 Bytes, about 124 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162200690 Bytes

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.59 seconds
stream load tsv: 576 seconds loaded 74807831229 Bytes, about 123 MB/s
stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162424517 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.55 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162158888 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.42 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162124856 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.93 seconds
stream load tsv: 555 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162436530 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

1 similar comment
@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.91 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162285414 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.15 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162425753 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.53 seconds
stream load tsv: 557 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size: 17162458300 Bytes

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.02 seconds
stream load tsv: 554 seconds loaded 74807831229 Bytes, about 128 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size: 17162435073 Bytes

@xingyingone
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.85 seconds
stream load tsv: 552 seconds loaded 74807831229 Bytes, about 129 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 29.0 seconds inserted 10000000 Rows, about 344K ops/s
storage size: 17162341015 Bytes

@dataroaring
Copy link
Contributor

Could you put an example of output?

@xingyingone
Copy link
Contributor Author

Could you put an example of output?

before modification:
MySQL [test_db]> truncate table create_table_use_created_policy;
ERROR 1105 (HY000): errCode = 2, detailMessage = Failed to find enough backend, please check the replication num,replication tag and storage medium.
Create failed replications:
replication tag: {"location" : "default"}, replication num: 3, storage medium: SSD

after modification:
MySQL [test_db]> truncate table create_table_use_created_policy;
ERROR 1105 (HY000): errCode = 2, detailMessage = Failed to find enough backend, please check the replication num,replication tag and storage medium and avail capacity of backends.
Create failed replications:
replication tag: {"location" : "default"}, replication num: 3, storage medium: SSD

@xingyingone
Copy link
Contributor Author

@dataroaring

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 14, 2023
Copy link
Contributor

@yiguolei yiguolei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yiguolei
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
Tpch sf100 test result on commit 77c7e273498e99da595e6d0bff3fef382582e2c4, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4956	4697	4630	4630
q2	364	162	158	158
q3	2043	1939	1959	1939
q4	1386	1249	1221	1221
q5	3967	3966	4025	3966
q6	252	125	129	125
q7	1413	873	896	873
q8	2760	2772	2771	2771
q9	9773	9613	9546	9546
q10	3471	3570	3495	3495
q11	367	238	249	238
q12	434	294	296	294
q13	4575	3841	3784	3784
q14	317	283	283	283
q15	587	540	533	533
q16	661	586	580	580
q17	1134	973	941	941
q18	7689	7347	7351	7347
q19	1663	1668	1655	1655
q20	539	291	299	291
q21	4380	3989	3963	3963
q22	484	385	400	385
Total cold run time: 53215 ms
Total hot run time: 49018 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4588	4593	4592	4592
q2	340	238	248	238
q3	4005	3970	3977	3970
q4	2704	2716	2688	2688
q5	9626	9473	9534	9473
q6	244	125	124	124
q7	2628	2328	2333	2328
q8	4421	4434	4446	4434
q9	13271	13151	13159	13151
q10	4075	4163	4161	4161
q11	755	635	641	635
q12	995	821	825	821
q13	4315	3557	3607	3557
q14	390	345	366	345
q15	583	515	522	515
q16	731	687	665	665
q17	3891	3853	3919	3853
q18	9600	8932	9043	8932
q19	1812	1760	1783	1760
q20	2397	2075	2031	2031
q21	8776	8542	8624	8542
q22	928	841	803	803
Total cold run time: 81075 ms
Total hot run time: 77618 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 44.89 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.8 seconds inserted 10000000 Rows, about 347K ops/s
storage size: 17098862508 Bytes

@xingyingone
Copy link
Contributor Author

run feut

@yiguolei yiguolei merged commit ab322ea into apache:master Nov 17, 2023
18 checks passed
superdiaodiao pushed a commit to superdiaodiao/doris that referenced this pull request Nov 21, 2023
seawinde pushed a commit to seawinde/doris that referenced this pull request Nov 28, 2023
XuJianxu pushed a commit to XuJianxu/doris that referenced this pull request Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] failed to truncate table
6 participants