Heartbeat for Hadoop

articles
tips-and-tricks

#1

In Hadoop Name node and data node does communicate using Heartbeat. Therefore Heartbeat is the signal that is sent by the data node to the namenode after the regular interval to time to indicate its presence, i.e. to indicate that it is alive.
If after a certain time of heartbeat Name Node does not receive any response from Data Node, then that particular Data Node used to be declared as dead.

The default heartbeat interval is 3 seconds. If the DataNode in HDFS does not send heartbeat to NameNode in ten minutes, then NameNode considers the DataNode to be out of service and the Blocks replicas hosted by that DataNode to be unavailable. The NameNode then schedules the creation of new replicas of those blocks on other DataNodes.

NameNode that receives the Heartbeats from a DataNode also carries information like total storage capacity, the fraction of storage in use, and the number of data transfers currently in progress. For the NameNode’s block allocation and load balancing decisions, we use these statistics.