Skip to content
HA Cluster Software | FAQ's | High Availability

Shared Nothing

Using SSH to bind two nodes together

A shared nothing cluster operates by creating incremental snapshots of data sets and then synchronising them between cluster nodes using ZFS send/receive over an ssh tunnel.

The ssh tunnel created and used by the synchronisation process needs to be passwordless and therefore the two nodes need to be ssh-bound. To configure ssh binding perform the following steps on each node:

  1. Create your ssh keys as the root user (press return to accept the defaults for all prompts):

    # ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key (/root/.ssh/id_rsa):
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /root/.ssh/id_rsa
    Your public key has been saved in /root/.ssh/
    The key fingerprint is:
    SHA256:2dGrTFvaGz8QJbVeVGS5sFv/deJRngPvSOr6v1SaMXc root@NodeA
    The key's randomart image is:
    +---[RSA 3072]----+
    |             ...B|
    |           ....= |
    |          . .o+ o|
    |         o ..= +.|
    |        S o o+*=E|
    |         o *.oX+*|
    |          = =*oo=|
    |           ..=o..|
    |         .+oooo. |

  2. Once ssh-keygen has been run, a public key is saved to /root/.ssh/ This public key now needs to be added to the file /root/.ssh/authorized_keys on the other node (if the authorized_keys file does not exist simply create one).

  3. Manually ssh NodeA > NodeB then NodeB > NodeA and accept the prompt to add each machine to the list of known hosts:

    root@NodeA:~# ssh root@NodeB
    The authenticity of host 'NodeB (' can't be established.
    ED25519 key fingerprint is SHA256:EDmzS45TqKabZ53/35vXb4YyKTQuzJxNnbFuIwFj9UU.
    Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
    Warning: Permanently added 'NodeB,' (ED25519) to the list of known hosts.
    Last login: Tue Sep 12 09:54:49 2023 from
    Oracle Solaris                  Assembled December 2021

Once this process has been completed you should be able to ssh between nodes without being prompted for a password.

ssh login between nodes taking a long time

If ssh is taking a long time, try running ssh -v to see any errors that may be causing the delay. A common issue is with GSS/Kerberos:

debug1: Next authentication method: gssapi-with-mic 

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_1000' not found 

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_1000' not found 

This can be disabled on the nodes by modifying /etc/ssh/ssh_config and disabling all Host options that begin with GSS. For example:

Host *
#   ForwardAgent no
#   ForwardX11 no
#   ForwardX11Trusted yes
#   PasswordAuthentication yes
#   HostbasedAuthentication no
   GSSAPIAuthentication no
   GSSAPIDelegateCredentials no
   GSSAPIKeyExchange no
   GSSAPITrustDNS no
#   BatchMode no
#   CheckHostIP yes
#   AddressFamily any
#   ConnectTimeout 0
#   StrictHostKeyChecking ask
#   IdentityFile ~/.ssh/id_rsa
#   IdentityFile ~/.ssh/id_dsa
#   IdentityFile ~/.ssh/id_ecdsa
#   IdentityFile ~/.ssh/id_ed25519
#   Port 22
#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc
#   MACs hmac-md5,hmac-sha1,
#   EscapeChar ~
#   Tunnel no
#   TunnelDevice any:any
#   PermitLocalCommand no
#   VisualHostKey no
#   ProxyCommand ssh -q -W %h:%p
#   RekeyLimit 1G 1h
#   UserKnownHostsFile ~/.ssh/known_hosts.d/%k
    SendEnv LANG LC_*