the maximum number of indexes created for a cluster

[tengzhuofei]

elassandra 6.8.4.10

With the compression parameter turned on, the maximum number of indexes created for a cluster can only be about 5100?

[vroyer]

Could you please provide information to reproduce the same test and hit this limit of 5100. Of course, long index names can shorten this limit.

…

On 21 Sep 2020, at 04:49, tengzhuofei ***@***.***> wrote: elassandra 6.8.4.10 With the compression parameter turned on, the maximum number of indexes created for a cluster can only be about 5100? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#372>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACOMPGJE46MNSOSDNAPQQSLSG25MPANCNFSM4RUAQTIA>.

[tengzhuofei]

Hi! I just kept creating indexes, and did not store data.

[vroyer]

Yes, but if you created 5000 underlying Cassandra keyspaces, that’s a Cassandra limitation….

…

On 21 Sep 2020, at 11:15, tengzhuofei ***@***.***> wrote: Hi! I just kept creating indexes, and did not store data. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#372 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACOMPGJFFUCMCPE5SFSS36DSG4KSTANCNFSM4RUAQTIA>.

[tengzhuofei]

I only created 3 keyspaces and set the keyspace value through the setting of elasticsearch.
There is no limit to the content of Gossip X1. I have enabled compression?

[tengzhuofei]

I restart a pod and too many pending states error

2020-09-21 11:16:20,135 DEBUG [elasticsearch[10.8.2.86][masterService#updateTask][T#1]] org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:255) publishing cluster state version [1259] metadata=[29791b59-4b87-41ff-84e1-3c73c67ecf21/5114]
2020-09-21 11:16:20,135 DEBUG [elasticsearch[10.8.2.86][masterService#updateTask][T#1]] org.elassandra.discovery.CassandraDiscovery.publishLocalUpdate(CassandraDiscovery.java:1196) Local update source=shard-started-update-routing index=twin_943[org.elassandra.shard.CassandraShardStateListener$RoutingTableUpdateTask@2764ffc9] metadata=29791b59-4b87-41ff-84e1-3c73c67ecf21/5114
2020-09-21 11:16:20,135 WARN [elasticsearch[10.8.2.86][masterService#updateTask][T#1]] org.elassandra.discovery.PendingClusterStatesQueue.addPending(PendingClusterStatesQueue.java:90) dropping pending state.version=[235] more than [1024] pending states.
2020-09-21 11:16:20,135 DEBUG [elasticsearch[10.8.2.86][generic][T#5]] org.elasticsearch.index.engine.CombinedDeletionPolicy.updateRetentionPolicy(CombinedDeletionPolicy.java:99) Safe commit [CommitPoint{segment[segments_2], userData[{history_uuid=Oi3NwB7ISseTUo7C-2ZMfw, local_checkpoint=-1, max_seq_no=-1, max_unsafe_auto_id_timestamp=-1, translog_generation=1, translog_uuid=18R8dd0eRRm0s-hYRDZVDA}]}], last commit [CommitPoint{segment[segments_2], userData[{history_uuid=Oi3NwB7ISseTUo7C-2ZMfw, local_checkpoint=-1, max_seq_no=-1, max_unsafe_auto_id_timestamp=-1, translog_generation=1, translog_uuid=18R8dd0eRRm0s-hYRDZVDA}]}]
2020-09-21 11:16:20,135 WARN [elasticsearch[10.8.2.86][masterService#updateTask][T#1]] org.elassandra.discovery.CassandraDiscovery$4.onNewClusterStateFailed(CassandraDiscovery.java:1212) failed while applying cluster state locally source=shard-started-update-routing index=feature_134[org.elassandra.shard.CassandraShardStateListener$RoutingTableUpdateTask@5555feb2]
org.elasticsearch.ElasticsearchException: too many pending states ([1024] pending)
at org.elassandra.discovery.PendingClusterStatesQueue.addPending(PendingClusterStatesQueue.java:91)
at org.elassandra.discovery.CassandraDiscovery.publishLocalUpdate(CassandraDiscovery.java:1198)
at org.elassandra.discovery.CassandraDiscovery.publish(CassandraDiscovery.java:996)
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:257)
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:158)
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150)
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188)
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)