Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some metering hive tables are not getting updated #1074

Open
amalkurup89 opened this issue Jan 9, 2020 · 5 comments
Open

Some metering hive tables are not getting updated #1074

amalkurup89 opened this issue Jan 9, 2020 · 5 comments

Comments

@amalkurup89
Copy link
Contributor

I can see some of the metering hive tables are not getting updated

PFA the screenshot highlighting this issue

Please help
metering_tables_refresh_issue

@timflannagan
Copy link
Contributor

@amalkurup89 hmm, can you post a dump of the reporting-operator logs? I believe you said you were deploying Metering w/o TLS support, so it should just be kubectl -n $METERING_NAMESPACE logs <reporting-operator pod name>.

@amalkurup89
Copy link
Contributor Author

rpt-operator-logs-1.txt

@timflannagan
Copy link
Contributor

I haven't seen this error before, did you change anything post install? In the case where you updated the top-level tls.enabled: false in your MeteringConfig CR post-install, it may be causing problems, and you would need to nuke your namespace and re-deploy metering.

time="2020-01-09T09:46:42Z" level=error msg="error syncing ReportDataSource \"metering/node-allocatable-memory-bytes\", adding back to queue" ReportDataSource=metering/node-allocatable-memory-bytes app=metering component=reportDataSourceWorker error="ImportFromLastTimestamp errored: failed to store Prometheus metrics into table hive.metering.datasource_metering_node_allocatable_memory_bytes for the range 2020-01-08 01:33:00 +0000 UTC to 2020-01-08 01:38:00 +0000 UTC: failed to store metrics into presto: presto SQL error: presto: query failed (200 OK): \"io.prestosql.spi.PrestoException: Error moving data files from file:/tmp/presto-reporting-operator/59644000-4ad5-40da-9d3e-82fc73b18b8c/dt=2020-01-08/20200109_094550_00023_2sz2w_2928bc64-db63-470c-b602-14a285e781cd to final location file:/user/hive/warehouse/metering.db/datasource_metering_node_allocatable_memory_bytes/dt=2020-01-08/20200109_094550_00023_2sz2w_2928bc64-db63-470c-b602-14a285e781cd\"" logID=xet8foeoo6

@amalkurup89
Copy link
Contributor Author

@timflannagan1 I haven't yet updated tls.enabled to false. The above error logs are with tls setting as true

@timflannagan
Copy link
Contributor

Okay, understood. Can you walk me through the steps you used to install metering? It still looks like you may have changed something post-installation that's causing the presto insert query to fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants