Skip to main content

Sharding Methods For MongoDB


Sharding Introduction

Sharding is a method for storing data across multiple machines.Sharding, or horizontal scaling, by contrast, divides the data set and distributes the data over multiple servers, or shards. Each shard is an independent database, and collectively, the shards make up a single logical database.


Sharding in MongoDB

MongoDB supports sharding through the configuration of a sharded clusters.
Components in Sharded Cluster :
  1. Shard
  2. Query Routers
  3. Config Servers
  • Shards store the data. To provide high availability and data consistency, in a production sharded cluster, each shard is a replica set .
  • Query Routers, or mongos instances, interface with client applications and direct operations to the appropriate shard or shards. The query router processes and targets operations to shards and then returns results to the clients. A sharded cluster can contain more than one query router to divide the client request load. A client sends requests to one query router. Most sharded clusters have many query routers.
  • Config servers store the cluster’s metadata. This data contains a mapping of the cluster’s data set to the shards. The query router uses this metadata to target operations to specific shards. Production sharded clusters have exactly 3 config servers.

Data Partitioning

MongoDB distributes data, or shards, at the collection level. Sharding partitions a collection’s data by the shard key.
A shard key is either an indexed field or an indexed compound field that exists in every document in the collection. MongoDB divides the shard key values into chunks and distributes the chunks evenly across the shards.
  1. Range Based Sharding
  2. Hash Based Sharding
  3. Customized Data Distribution with Tag Aware Sharding
  • For range-based sharding, MongoDB divides the data set into ranges determined by the shard key values to provide range based partitioning.
  • For hash based partitioning, MongoDB computes a hash of a field’s value, and then uses these hashes to create chunks.
  • For Customized Data Distribution, MongoDB allows administrators to direct the balancing policy using tag aware sharding
Facts of Data Partitioning

  • Range Based Sharding
    • supports more efficient range queries
    • router can easily determine which chunks overlap that range
    • route the query to only those shards that contain these chunks
    • uneven distribution of data(Unbalanced)
  • Hash Based Sharding
    • even distribution of data
    • expense of efficient range queries
    • Hashed key values results in random distribution of data across chunks
    • every shard in order to return a result
    • will not be able to target a few shards
  • Customized Data Distribution with Tag Aware Sharding
    • Administrators create and associate tags with ranges of the shard key
    • assign those tags to the shards
    • the balancer migrates tagged data to the appropriate shards
    • cluster always enforces the distribution of data that the tags describe
    • tag aware sharding serves to improve the locality of data for sharded clusters that span multiple data centers

Balancing In Data Distribution

Data distribution balancing has more importance within the cluster ,such as a particular shard contains significantly more chunks than another shard or a size of a chunk is significantly greater than other chunk sizes.
  1. Splitting
  2. Balancer
  • Splitting
    • Splitting is a background process that keeps chunks from growing too large
    • MongoDB splits the chunk in half when beyond a specified chunk size
    • Splits are an efficient meta-data change
    • To create splits, MongoDB does not migrate any data or affect the shards
  • Balancer
    • Balancer is a background process that manages chunk migrations
    • Balancer can run from any of the query routers in a cluster
    • If there’s an error during the migration, the balancer aborts the process leaving the chunk unchanged on the origin shard
    • MongoDB removes the chunk’s data from the origin shard after the migration completes successfully

More details of Sharding Methods For MongoDB are available on official site on mongodb.

Comments

Post a Comment

Popular posts from this blog

What exactly means MVW design pattern ?

What is a MVW framework? The abbreviation stands for 'Model - View - Whatever'.  Well there are many different JavaScript frameworks available , all invented for the same purpose. They all try to separate the presentation logic from the business logic where JavaScript holds the model and logic, and html the presentation layer. Quick overview : You can change the model without changing the view and vice-versa Unit testing is easy These are the core benefits of using such a framework. By separating presentation and business logic changes to the code are easier to implement and take less time. Another benefit is that code can be reused from a controller thus saving more time. Angularjs makes the code also shorter in comparison to other frameworks, which improves code stability. Less code means minor potential for bugs. For several years +AngularJS was closer to MVC (or rather one of its client-side variants), but over time and thanks to many refactorings
1 comment
Read more

File Upload & Download With ng-cordova File Transfer Plugin In Ionic Framework

Using the AngularJS extension set , ngCordova , with Ionic Framework and the Apache Cordova File Transfer plugin, you can easily upload files to a remote server and download files from a remote server. 1.Create the project ionic start Test blank cd Test ionic platform add android 2.Add Plugin org.apache.cordova.file-transfer https://github.com/apache/cordova-plugin-file-transfer This plugin allows you to upload and download files. This plugin defines global FileTransfer, FileUploadOptions Constructors. Although in the global scope, they are not available until after the deviceready event . Installation cordova plugin add cordova plugin add cordova-plugin-file-transfer     2.    org.apache.cordova.file             https://github.com/apache/cordova-plugin-file             This plugin implements a File API allowing read/write access to files residing on the    device.             Installation cordova plugin add cordova-pl
20 comments
Read more

Steps To Create Cordova Plugin

Introduction A plugin is a package of injected code that allows the Cordova webview within which the app renders to communicate with the native platform on which it runs. Plugins comprise a single JavaScript interface along with corresponding native code libraries for each supported platform. Due to following reason, we will go for plugin development To get the native capabilities of the device in the cordova based app Build the common functionality across the number of Cordova application(Reusability) Prerequisites For the creation of plugin you need to install plugman . npm install -g plugman Plugin Structure Decide Plugin ID & Name The plugin ID should be unique Plugin name should follow the convention “cordova.plugin.[your.plugin.name]” Create the structure Use the following command to create the plugin with plugin ID “cordova.plugin.test” and name “Test”. plugman create --name Test --
Post a Comment
Read more