[Azure Cosmos Key-Value] Add support for multiple stores in a single container

[rylev]

The problem

Currently the Azure Cosmos implementation of key-value assumes a 1 to 1 relationship between a key-value store and CosmosDB container. While this makes sense at first glance, it might be too limiting for many use cases. CosmosDB has a limit on the number of database/container combinations one can have: 500 per account. This means that the current implementation will only support 500 stores.

A high level solution

Alternatively, the implementation could be changed to store many different stores in a single container. This could potentially mean that for multi-tenant use cases of Spin, all key-value pairs would be stored in a single container. Single tenant users who know they will never exceed more than 500 stores can continue using a single container per store or switch to multiple stores per container if they wish to.

Details

In order to model multiple stores in a single container, we'll need to add additional information to each key/value pair. Instead of only storing the key and value as the current implementation does, we'll also want to store the store name. Since store names are not globally unique, we also want to allow storing some additional globally unique application identifier. For single tenant users this can be left blank or hardcoded to any value, but for multi-tenant users who care about supporting multiple apps, they can store some globally unique app identifier.

cc @radu-matei and @devigned who have both worked on the current CosmosDB implementation for their thoughts.

[devigned]

Off the top of my head, this sounds like a great idea and seems to mimic the use of SQLite. One thing to consider is possibly using the combination of app identifier and store name as a key for partitioning. I believe that would reduce the cost of queries by targeting a specific partition rather than a cross partition query.

Additionally, I'm not sure what kind of migration would need to happen for existing data. Is a migration from old to new schema something that would be desired?