High Availability Setup

High Availability can be applied to Helix TeamHub after the Helix TeamHub Cluster installation has been performed, and Helix TeamHub has been bootstrapped.

Step 1: Scaling up with Load Balancer

As was mentioned in HA Deployment, an SSL load balancer is required, which will decrypt SSL connections and also balance requests across the Helix TeamHub Web servers.

The Helix TeamHub package does not include load balancer, therefore it needs to be installed separately. The following guide is recommended to setup the load balancer first if none exists.

Step 2: Mounting Shared Storage

With a load balancer, the user requests will be randomly distributed across the cluster nodes, so the data will become immediately out of sync. To fix this issue, attach the same shared storage to each Helix TeamHub Web server. If existing storage with a clustered file system doesn't exist, contact the Support team for further help.

After shared storage is available, stop Helix TeamHub, then mount the storage to /var/opt/hth/shared. Next, bring Helix TeamHub back online:

sudo hth-ctl stop
sudo mv /var/opt/hth/shared /var/opt/hth/shared.bak
# Mount storage to /var/opt/hth/shared and sync back the data
sudo rsync -av /var/opt/hth/shared.bak/ /var/opt/hth/shared/
rm -rf /var/opt/hth/shared.bak
sudo hth-ctl start

Step 3: Synchronizing SSH Host Keys

Since the SSH host keys will differ between the cluster nodes, they need to be synchronized. Helix TeamHub configuration process can use the ssh directory on the shared storage and copy the SSH host keys to the usual /etc/ssh. This will enable every new Helix TeamHub Web server added to the cluster to have the same SSH host keys. So on the first Helix TeamHub Web server, create the directory and copy SSH host keys:

mkdir -p /var/opt/hth/shared/ssh
cp /etc/ssh/ssh_host_* /var/opt/hth/shared/ssh/
chown root.root /var/opt/hth/shared/ssh/*
chmod 600 /var/opt/hth/shared/ssh/*

Step 4: Adding More Helix TeamHub Web Servers

After steps 1 through 3 are performed, additional Helix TeamHub Web servers can be added to the cluster. Since the Helix TeamHub Configuration file is stored on a shared partition /var/opt/hth/shared, simply install the Helix TeamHub Web package and reconfigure it, and Helix TeamHub will automatically pick up the needed configurations:

RHEL and CentOS

yum install python-devel gettext python-docutils pam-devel cronie postfix libidn libidn-devel
rpm -ivh hth-web-X.X.X-stable.el6.x86_64.rpm

Debian and Ubuntu

apt-get install python-dev python-docutils gettext sudo libpam0g-dev postfix libidn11 libidn11-dev cron
dpkg -i hth_web_X.X.X_amd64.deb

Step 5: Changing Hostname

At this stage everything should be up and running, and requests should be distributed across all Helix TeamHub Web servers. However, Helix TeamHub is still bootstrapped with the hostname of the first Helix TeamHub Web server installed. To fix this, go to Helix TeamHub Admin and change the hostname to the load balancer.

Updated on: 5 October 2017