How To Setup etcd Clusters

Good day. It's Tuesday, Aug. 27, and in this issue, we're covering:

• How to Setup etcd Clusters

• PostgreSQL Databases under attack

• Optimize Messaging with IBM MQ on Red Hat OpenShift Service on AWS

• How To Structure Terraform Deployments At Scale

• Advanced DevOps Techniques: Scaling Microservices with Kubernetes

• Machine Learning for Beginners - 25 Lessons

You share. We listen. As always, send us feedback at [email protected]

Use Case

How To Setup etcd Clusters

To begin with - Etcd is a distributed, reliable key-value store designed to securely store configuration data and state information for distributed systems like Kubernetes.

It acts as the backbone of the Kubernetes control plane, ensuring that all components within the cluster are in sync and that the desired state of the cluster is maintained.

How ETCD works:

Etcd operates using a consensus algorithm called RAFT to maintain consistency across the distributed nodes.

In an etcd cluster, one node is elected as the leader, while the others are followers. The leader handles all write requests and propagates these changes to the followers to ensure data consistency across the cluster.

If the leader fails, a new leader is elected from the remaining nodes to maintain cluster operations without downtime.

Deployment Types:

Stacked etcd cluster - etcd instances run on the same nodes as the Kubernetes control plane components. This setup is simple but offers less resilience in the event of node failures.

This is generally suitable for smaller environments or development clusters where ease of setup and management is prioritized over high availability.

External etcd cluster - etcd runs on dedicated nodes separate from the control plane, offering enhanced resilience and fault tolerance.

This setup enhances resilience and fault tolerance, as failures in the control plane do not directly impact etcd, and vice versa.

It provides a higher level of availability, making it the preferred choice for production environments where maintaining cluster stability is crucial.

Practical Step-by-Step Guide:

Step 1. Download and Install etcd

wget -q --show-progress --https-only --timestamping \
"https://github.com/etcd-io/etcd/releases/download/v3.5.15/etcd-v3.5.15-linux-amd64.tar.gz"

tar -xvf etcd-v3.5.15-linux-amd64.tar.gz
sudo mv etcd-v3.5.15-linux-amd64/etcd* /usr/local/bin/

Step 2. Configure etcd

sudo mkdir -p /etc/etcd /var/lib/etcd
sudo cp ca.pem kubernetes-key.pem kubernetes.pem /etc/etcd/

Step 3. Setup Environment Variables

# Example for AWS
INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)
ETCD_NAME=$(hostname)

# Example for Azure
INTERNAL_IP=$(curl -H Metadata:true -s "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2021-02-01&format=text")
ETCD_NAME=$(hostname)

# Example for on-prem with static IPs
INTERNAL_IP=<Your_Static_IP>
ETCD_NAME=$(hostname -s)

Step 4. Create etcd Systemd Unit File

cat <<EOF | sudo tee /etc/systemd/system/etcd.service
[Unit]

[Service]
Type=notify
ExecStart=/usr/local/bin/etcd \\
  --name ${ETCD_NAME} \\
  --cert-file=/etc/etcd/kubernetes.pem \\
  --key-file=/etc/etcd/kubernetes-key.pem \\
  --peer-cert-file=/etc/etcd/kubernetes.pem \\
  --peer-key-file=/etc/etcd/kubernetes-key.pem \\
  --trusted-ca-file=/etc/etcd/ca.pem \\
  --peer-trusted-ca-file=/etc/etcd/ca.pem \\
  --peer-client-cert-auth \\
  --client-cert-auth \\
  --initial-advertise-peer-urls https://${INTERNAL_IP}:2380 \\
  --listen-peer-urls https://${INTERNAL_IP}:2380 \\
  --listen-client-urls https://${INTERNAL_IP}:2379,https://127.0.0.1:2379 \\
  --advertise-client-urls https://${INTERNAL_IP}:2379 \\
  --initial-cluster-token etcd-cluster-0 \\
  --initial-cluster controller-0=https://10.240.0.10:2380,controller-1=https://10.240.0.11:2380,controller-2=https://10.240.0.12:2380 \\
  --initial-cluster-state new \\
  --data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

Step 5. Start the etcd Service

sudo systemctl daemon-reload
sudo systemctl enable etcd
sudo systemctl start etcd

Step 6. Verify etcd Cluster Members

sudo ETCDCTL_API=3 etcdctl member list \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/etcd/ca.pem \
  --cert=/etc/etcd/kubernetes.pem \
  --key=/etc/etcd/kubernetes-key.pem

Tips and Tricks:

• Backup Regularly: Etcd stores critical Kubernetes data; regular backups are essential. Use the etcdctl snapshot save command to create backups.

• Monitor Leader Election: Keep an eye on the leader election process using etcd logs. Frequent elections might indicate an unstable cluster.

• Use TLS Encryption: Always configure TLS encryption for communication between etcd nodes and clients to secure the cluster.

Final reminder,

etcd is the backbone of Kubernetes and keeps everything in sync — get it right, and your Kubernetes stays solid.

p.s. if you think someone else you know may like this newsletter, share with them to join here

Tool Of The Day

Dive - A tool for exploring a docker image, layer contents, and discovering ways to shrink the size of your Docker/OCI image.

Try it out !

Trends & Updates

Resources & Tutorials

Picture Of The Day

Did someone forward this email to you? Sign up here