Converting a Standalone Seq Instance into a Cluster

It's possible to convert an existing standalone Seq node to a cluster node, then add additional nodes to the cluster. This slightly modifies the sequence of steps performed to set up the cluster.

📘

These steps assume you're running the very latest Seq version. Update first, if necessary.

Set up the Cluster State Database

The first step when enabling clustering on an existing Seq instance is to migrate its metadata store to the database that will be shared between cluster nodes.

Configure the database using the information in Configuring a Cluster State Database .

Then, use the metastore to-mssql or metastore to-postgresql commands to populate the database. Both commands have very similar syntax; the to-mssql version is shown below.

seq service stop
seq metastore to-mssql --connection-string="{Cluster State Database Connection String}"
seq service start

The connection string will be encrypted with the Seq instance's secret key, and stored in the Seq storage directory under Secrets/.

👍

Cluster Worksheet Update

The Setting up a Seq HA Cluster page includes a a worksheet for recording important configuration items during cluster setup. Record the Cluster State Database Connection String there.

Record the Secret Key

Later, to set up the additional nodes, you'll need the secret key from the existing node. If you don't have it in external storage already, you should retrieve it now using:

seq show-key

It's very important that the correct secret key is recorded: nodes won't be able to join the cluster without it.

👍

Cluster Worksheet Update

Record the Secret Key in the cluster worksheet.

Set up the Cluster Diagnostic Instance

See the instructions in Configuring a Cluster Diagnostic Instance for how to set up an additional Seq instance to receive diagnostic information from the cluster. As we all know 😁 diagnosing issues in distributed systems is hard, so setting up a Seq instance to record logs, traces, and metrics from the cluster can save a lot of time when troubleshooting stability or performance problems down the track.

Once the diagnostic instance is available, send logs from this instance to it:

seq config set -k diagnostics.otlp.endpoint -v "{Diagnostic Instance Ingestion URI}"
seq config set -k diagnostics.otlp.protocol -v "http/protobuf"
seq secret set -k diagnostics.otlp.headers -v "X-Seq-ApiKey={Diagnostic Instance API Key}"
seq config set -k diagnostics.otlp.metricsSamplingIntervalSeconds -v 10
seq service restart

You should now see your existing Seq instance's internal logs in the diagnostic instance.

Inbound Ports

See Inbound Ports for a list of ports that need to be open on this machine for other nodes to connect to it.

Enable Clustering

seq service stop
seq config set -k api.canonicalUri -v "{Load Balancer URI}"
seq config set -k cluster.clusterListenUri -v "{Node Cluster URI}"
seq config set -k cluster.internalApiUri -v "{Node Internal API URI}"
seq config set -k cluster.isEnabled -v True
seq service start

Set up the Load Balancer

Now, follow the steps in Configuring a Cluster Load Balancer to set up the load balancer. There's one important variation to the process to look out for.

❗️

Add only one Seq node (the existing one) to the load balancer configuration. Don't add additional nodes until they're already configured - see the instructions below.

Add Additional Cluster Nodes

To add additional nodes, follow the instructions in Adding, Updating, and Removing Nodes.

🔅

Tip

Additional nodes will be configured slightly differently. It's best if all nodes in the cluster are configured the same way, so once you've added some more nodes, consider shutting and retiring the original one.