Choosing the correct shard key

After the previous section, it's now self-evident that we need to take the choice of our shard key into consideration as it is a decision that we have to stick with.

A great shard key has three characteristics:

  • High cardinality
  • Low frequency
  • Nonmonotonic changes in value

We will go over the definitions of these three properties first to understand what they mean:

  • High cardinality: It means that the shard key must have as many distinct values as possible. A Boolean can take only the values of true/false, and so it is a bad shard key choice. A 64-bit long value field that can take any value from −(2^63) to 2^63−1 is a good shard key choice, in terms of cardinality.
  • Low frequency: It directly relates to the argument about high cardinality. A low-frequency shard key will have a distribution of values as close to a perfectly random/uniform distribution. Using the example of our 64-bit long value, it is of little use to us if we have a field that can take values ranging from −(2^63) to 2^63−1 if we end up observing the values of zero and one all the time. In fact, it is as bad as using a Boolean field, which can also take only two values. If we have a shard key with high-frequency values, we will end up with chunks that are indivisible. These chunks cannot be further divided and will grow in size, negatively affecting the performance of the shard that contains them.
  • Nonmonotonically changing values: It mean that our shard key should not be, for example, an integer that always increases with every new insert. If we choose a monotonically increasing value as our shard key, this will result in all writes ending up in the last of all of our shards, limiting our write performance.
If we want to use a monotonically changing value as the shard key, we should consider using hash-based sharding.

In the next section, we will describe different sharding strategies, including their advantages and disadvantages.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset