MongoDB: Sharding

  1. What is sharding?
  2. How is it done?
  3. Pitfalls & how to design your data distribution strategy?
  4. When it should be done?
  5. When is it too late?

Horizontal scaling (scaling-out)

It achieves the scaling requirements by scaling out, i.e. by adding more nodes (machines). This is also called horizontal scaling. Horizontal scaling is cheaper than vertical scaling. With vertical scaling, you are forced to choose high-cost hardware, whereas with horizontal scaling you can go with multiple relatively low-cost hardware.

Horizontal partitioning

MongoDB does horizontal partitioning of data, unlike RDBMS solutions that do vertical partitioning of data.

  1. Shard-key is immutable
  2. Shard-key once defined cannot be changed
  3. A sharded collection cannot be un-sharded
  • An application that requires segmenting user data based on the geographic country — For example, certain business is governed by data-protection rules.
  • A database that requires resource allocation based on geographic country.
  1. Data protection rules can be satisfied
  2. Data is collocated within the fixed set of shards within the zone
  3. Read-scalability — Such cases, the queries are also zone-specific, so fewer shards have to work for a given query.
  1. Scaling can be limited to the zone. Even if servers in one zone are over-loaded they cannot leverage the idle capacity in the other zone.

When should you go for sharding?

There are overheads going the sharding way for your scaling

When is it too late to scale-out?

  1. When the working set is already to the brim
  2. When the CPU is already working above the threshold
  3. When the storage limitations thresholds have crossed.

Conclusion

Even with all the complexity, it is this ability to scale-out that MongoDB has established itself as a major player in the big-data world. In some deployments, there are over 1000 nodes in a cluster hosting petabytes of data & crunching and analyzing this amount of data is no easy feat that Mongo pulls of effortlessly.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Sarada Sastri

Sarada Sastri

Java Architect | MongoDB | Oracle DB| Application Performance Tuning | Design Thinking | https://www.linkedin.com/in/saradasastri/