What is HA in YARN ResourceManager?

The YARN Resource Manager is responsible for managing the resources in a cluster and scheduling applications.Prior to Hadoop 2.4, the Resource Manager was a single point of failure in a YARN cluster.

The Resource Manager provides High Availability (HA) by implementing an active-standby Resource Manager pair to remove this single point of failure. When the active Resource Manager Node fails, the control switches to the standby Resource Manager, and all halted applications resume from the last state saved in the state store. This allows handling failover without any performance degradation in the following situations:

  • Unplanned events such as machine crashes
  • Planned maintenance events of software or hardware upgrades to the machine running the ResourceManager
  • ResourceManager HA requires the ZooKeeper and HDFS services to be running