Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

negative updated results #36

Open
florakarniav opened this issue Mar 22, 2018 · 4 comments
Open

negative updated results #36

florakarniav opened this issue Mar 22, 2018 · 4 comments

Comments

@florakarniav
Copy link

florakarniav commented Mar 22, 2018

Hello, I've been using the yahoo streaming benchmark in different environments and since yesterday I tried it on an 32-core intel xeon machine (but using only 8 cores for flink with taskset). I see negative results in the updated file (e.g. -70646). Can I get some help with this issue?

@Yitian-Zhang
Copy link

This problem is most likely caused by the non-synchronization of the system time among worker nodes in your cluster. When the benchmark system runs, the start timestamp will be created based on the system time of one node in the cluster, and then the last updated timestamp will be created based on other worker node (if the tasks were distributed among different nodes) when the final result written to Redis. Hence, if the time among different nodes is not synchronized, it is possible that the difference between the two timestamps is negative. So you can check the system time of worker nodes in your cluster and then run the benchmark after keeping them synchronization.

@varuy322
Copy link

This problem is most likely caused by the non-synchronization of the system time among worker nodes in your cluster. When the benchmark system runs, the start timestamp will be created based on the system time of one node in the cluster, and then the last updated timestamp will be created based on other worker node (if the tasks were distributed among different nodes) when the final result written to Redis. Hence, if the time among different nodes is not synchronized, it is possible that the difference between the two timestamps is negative. So you can check the system time of worker nodes in your cluster and then run the benchmark after keeping them synchronization.

That's odd, I came across this issue. I checked the component configuration and code, nothing seemed to be wrong. Do you solve this by synchronize cluster system time?

@Yitian-Zhang
Copy link

This problem is most likely caused by the non-synchronization of the system time among worker nodes in your cluster. When the benchmark system runs, the start timestamp will be created based on the system time of one node in the cluster, and then the last updated timestamp will be created based on other worker node (if the tasks were distributed among different nodes) when the final result written to Redis. Hence, if the time among different nodes is not synchronized, it is possible that the difference between the two timestamps is negative. So you can check the system time of worker nodes in your cluster and then run the benchmark after keeping them synchronization.

That's odd, I came across this issue. I checked the component configuration and code, nothing seemed to be wrong. Do you solve this by synchronize cluster system time?

Yes, I encountered the same issue that negative updated values. And I solved this problem by synchronize cluster system time.

@HQBIAO
Copy link

HQBIAO commented May 28, 2020

I came across this issue

Have you solved this problem,I came across this issue too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants