How to evaluate model performance?

[sgfisme]

We tried to evaluate two different model based on different cluster setting as below:

• Model 1:

  • Cluster: 1017, demand 43362, plan generation time: 35~36 mins
    • 2/3 demand distribute cluster 1.
    • Other demands distribute to 1016 clusters
  • Cluster: 14918, demand 43362, plan generation time: more than 10 hrs
    • 43362 demands distribute to all of the clusters averagely.

• Model 2:

  • Cluster: 5011, demand: 1183, plan generation time: 42~43 mins
    • All of the demands distribute to cluster 2
  • Cluster: 14197, demand: 1183, plan generation time: 17~18 mins
    • 1183 demands distribute to 116 clusters averagely.
    • 14081 clusters no any demands.

On model 1, more clusters got bad performance for planning generation. But on model 2, more cluster improve planning generation performance. How to evaluate how many clusters to get better model performance?