Heartbeats
Heartbeats
In the cluster, heartbeats perform the following roles:
- To continually monitor the other nodes in the cluster, ensuring they are active and available.
- Communicate cluster and service status to the other nodes in the cluster. Status information includes
mode and state for every service on that node (
manual
/automatic
running
/stopped
etc), along with any services that are currently blocked. - A checksum of the active cluster configuration on that node.
Configuration checksums
The configuration checksums must match on all cluster nodes to ensure the validity of the cluster; should a mismatch be detected then the cluster will lock the current state of the all services (active or not) until the mismatch is resolved. This safety feature protects against unexpected behaviour as a result of unsynchronised configuration.
The cluster supports two types of heartbeats:
- Network heartbeats
- Disk heartbeats
Heartbeats are unidirectional therefore for each heartbeat configured there will be two channels (one to send and one to receive).
The same information and structures is transmitted over each type of heartbeat. The cluster supports multiple heartbeats of each type. When the cluster is first created a network hearbeat is automatically configured between cluster nodes using the node hostnames as the endpoints. Disk heartbeats are automatically configured when a service is created and under normal circumstances require no user intervention.
It is recommended practice to configure network heartbeats across any additional network interfaces.
For example, if the hostnames are on a 10.x.x.x
network, and an additional private network exists
with 192.x.x.x
addresses, then an additional heartbeat can be configured on that private network.
Using the following example hosts file an additional network heartbeat can be configured using the
node-a-priv
and node-b-priv
addresses as endpoints:
10.0.0.1 node-a
10.0.0.2 node-b
192.168.72.1 node-a-priv
192.168.72.2 node-b-priv
By specifying the endpoint using the address of an additional interface the cluster will automatically route heartbeat packets down the correct network for that interface.
To view the cluster heartbeats navigate to HA-Cluster -> Heartbeats
on the left side-menu:
Adding a Network Heartbeat
To add an additional network heartbeat to the cluster, select Add
Network Heartbeat Pair
.
In this example an additional physical network connection exists between the two nodes.
The end points for this additional network are given the names
SAM node-a-priv
and node-b-priv
respectively. These
hostnames are then used when configuring the additional heartbeat:
Click Submit
to add the heartbeat.
The new heartbeat will now be displayed on the Heartbeats
status page:
Removing a Network Heartbeat
To remove a network heartbeat select the heartbeat using the slider on the
left hand side of the table and click the remove selected
button:
Finally, confirm the action:
Disk heartbeats
Under normal circumstances it should not be necessary to add or remove disk heartbeats as this is handled automatically by the cluster.
-
RSF-1 uses broadcast packets to detect cluster nodes on the local network. Broadcast packets are usually blocked from traversing other networks and therefore cluster node discovery is usually limited to the local network only. ↩