Azure Cosmos DB - Partitioning

Partitioning in Azure Cosmos DB and why it is important?

Azure Cosmos DB uses partitioning to scale individual containers in a database to meet the performance needs of your application.

There are two types of partitioning in Azure Cosmos DB:

Physical Partitions

Azure Cosmos DB is designed to scale horizontally based on the distribution of data between physical partitions. Physical partitions are internal elements of the platform, and they are managed by Azure.

Logical Partitions

Think of logical partitions as buckets of documents which share the same characteristic - The Partition Key. The related documents, or logical partitions, are supposed to be stored - fully- in the same physical partition.

Azure Cosmos DB manages the placement of logical partitions on physical partitions to efficiently satisfy the scalability and performance needs of the container.

How to choose the right partition key?

The right answer is it depends on your application. However, I feel you did not decide to read this blog post to get such a generic answer! So here are a set of questions and best practices that may help you to form the right answer:

• Analyse the data consumption pattern.

  • Daytime activity (e.g., end of the day transaction processing)

  • Seasonal load or spike (e.g., end of the year sales)

  • Special event short-time high-intensity workout (e.g., special event registration)

  • Balanced or Linear

  • Discrete or Continuous

• Request unit (RU) consumption and data storage should be spread evenly across all logical partitions.

• Granular partitioning (or too granular) leads to RU overhead; This is caused by queries fan-out between physical partitions.

• For large read-heavy containers, choose a partition key that appears frequently as a filter in your queries.

• Choos partition key for high cardinality per container.

• Aim for small partitions and avoid large partitions as much as possible - unless there’s a very good reason to do so.

• Always think about how you can avoid cross partitions queries as much as possible (or think when you don’t have money!)

• Assess your data and partitioning strategies against two key limitations of partitions:

  • Logical partition has a 20GB limit in size.

  • Physical partition has a 50GB limit in size.