By apipark — 25 Dec 2025

Docker Compose Redis Cluster: GitHub Setup Guide

docker-compose redis cluster github

In the dynamic landscape of modern application development, the demand for high-performance, scalable, and resilient data storage solutions is ever-increasing. From real-time analytics to distributed caching and session management, applications across various domains rely heavily on efficient data access. Redis, an in-memory data structure store, has emerged as a cornerstone for many such requirements, prized for its unparalleled speed and versatility. However, a standalone Redis instance, while incredibly fast, presents inherent limitations in terms of scalability and high availability, making it less suitable for production environments where uninterrupted service and growing data volumes are paramount. This is where Redis Cluster steps in, offering automatic sharding across multiple nodes and providing robust fault tolerance through master-slave replication.

Orchestrating a complex distributed system like a Redis Cluster manually can be a daunting task, fraught with configuration complexities and dependency management challenges. This is precisely where Docker Compose shines. Docker Compose provides an elegant and efficient way to define and run multi-container Docker applications, allowing developers to declare all services, networks, and volumes in a single, version-controlled YAML file. By leveraging Docker Compose, the intricate setup of a Redis Cluster transforms from a labyrinthine manual process into a repeatable, portable, and easily shareable configuration.

This comprehensive guide aims to demystify the process of setting up a production-ready Redis Cluster using Docker Compose, providing a step-by-step walkthrough suitable for deployment and management via GitHub. We will delve into the core architectural principles of Redis Cluster, elaborate on the role of Docker Compose in orchestrating its various components, and provide practical, executable code examples that you can readily adapt for your own projects. Our journey will cover everything from configuring individual Redis nodes and defining their interconnections within Docker, to initializing the cluster and ensuring its resilience. By the end of this guide, you will possess a robust, scalable Redis Cluster setup, perfectly configured and ready for integration into your microservices or enterprise applications, accessible and manageable through a well-structured GitHub repository. We will also touch upon how such a resilient backend data store complements the broader ecosystem of API management, where tools like an API gateway play a crucial role in securing and routing traffic to applications leveraging this powerful Redis infrastructure.

Unpacking the Fundamentals: Why Redis Cluster is Indispensable

Before we dive into the practicalities of Docker Compose, it's crucial to understand the "why" behind Redis Cluster and its fundamental architectural principles. A solid grasp of these concepts will illuminate the design choices we make in our Docker Compose configuration and empower you to troubleshoot effectively.

The Limitations of a Single Redis Instance

While a single Redis instance offers impressive performance, it suffers from two major drawbacks in a production setting:

Single Point of Failure (SPOF): If the solitary Redis server crashes, your application loses its data store, leading to service disruption. This is unacceptable for mission-critical applications that demand high availability. Recovery time objective (RTO) and recovery point objective (RPO) metrics are severely impacted by an SPOF. Even with diligent backup strategies, the time taken to restore a single large instance can be substantial, leading to extended downtime.
Scalability Bottleneck: All data resides on a single machine, limiting the total memory and CPU resources available. As your application grows and data volume or request rates increase, a single instance will eventually hit its performance ceiling, leading to slower response times and potential service degradation. Vertical scaling (adding more RAM, CPU to the same machine) eventually becomes cost-prohibitive and reaches physical limits. Horizontal scaling, distributing the load across multiple machines, is the only sustainable long-term solution for growing applications.

The Power of Redis Cluster: High Availability and Scalability

Redis Cluster addresses these limitations by providing a distributed, fault-tolerant, and scalable implementation of Redis. It achieves this through a clever combination of data sharding and master-slave replication:

Automatic Data Sharding (Horizontal Scaling): Instead of storing all data on one node, Redis Cluster automatically distributes your dataset across multiple master nodes. The entire key space of Redis is partitioned into 16384 hash slots. When you store a key, Redis calculates a hash of the key and maps it to one of these slots. Each master node in the cluster is responsible for a subset of these hash slots. This means that as your data grows, you can simply add more master nodes, and the cluster will automatically rebalance the hash slots, effectively distributing the load and memory requirements horizontally across a larger pool of resources. This significantly enhances the total memory capacity and CPU throughput of your Redis deployment.
Master-Slave Replication (High Availability and Fault Tolerance): For each master node, you can configure one or more replica (slave) nodes. These replica nodes asynchronously mirror the data of their respective masters. If a master node fails, the cluster automatically initiates a failover process, promoting one of its healthy replicas to become the new master. This ensures continuous operation without manual intervention, dramatically reducing downtime and improving the reliability of your data layer. The cluster uses a consensus mechanism (based on a variant of Raft or Paxos, specifically a gossip protocol and a majority vote from reachable master nodes) to detect failures and elect new masters, ensuring data consistency even during network partitions or node outages. The number of replicas you choose for each master directly impacts the cluster's resilience; more replicas mean a higher degree of fault tolerance.
Client Redirection: When a client sends a command for a specific key, it might initially connect to any node in the cluster. If that node does not own the hash slot for the requested key, it transparently redirects the client to the correct master node that does own the slot. This redirection mechanism is handled by intelligent client libraries, abstracting the underlying sharding logic from the application developer. The client libraries learn the cluster topology and cache slot-to-node mappings, updating them as the cluster reconfigures (e.g., after a failover or resharding). This makes interacting with a Redis Cluster almost as straightforward as interacting with a single instance, from the application's perspective.
Cluster Bus: Nodes within the cluster communicate with each other using a dedicated TCP bus, which runs on a separate port (usually the client port + 10000, so 16379 if the client port is 6379). This bus is used for heartbeat signals, propagating configuration updates, failure detection, and various other control plane operations, forming the backbone of the cluster's self-organizing capabilities.

Minimum Cluster Requirements

For a Redis Cluster to be fully fault-tolerant and capable of surviving at least one master node failure, it requires a minimum of 3 master nodes, each with at least 1 replica (slave) node. This configuration results in a total of 6 Redis nodes (3 masters, 3 slaves). With this setup, if one master fails, its slave can be promoted. If a second master fails, the cluster can still operate, but it becomes vulnerable to further failures. This 3-master, 3-slave setup is the recommended minimum for a production environment to ensure both data distribution and high availability.

The ability of Redis Cluster to gracefully handle node failures and scale horizontally makes it an indispensable component for high-traffic, data-intensive applications. It provides a robust, resilient, and performant backbone for caching, real-time data processing, and distributed session management, ensuring that your application's data layer can keep pace with demanding workloads.

The Orchestrator: Docker Compose for Seamless Deployment

Setting up a distributed system like Redis Cluster involves coordinating multiple independent processes. Each Redis node needs its own configuration, persistent storage, and network identity. Manually managing these aspects for six or more nodes can quickly become overwhelming and error-prone. This is precisely where Docker Compose simplifies the entire process, transforming complex setups into declarative configurations.

Why Docker Compose is the Ideal Tool

Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services, making the entire setup:

Simple and Declarative: Instead of running multiple docker run commands with complex arguments, you define all your services, their images, ports, volumes, networks, and environment variables in a single docker-compose.yml file. This declarative approach makes the setup easy to understand, review, and maintain. It clearly articulates the desired state of your application stack.
Portable and Reproducible: The docker-compose.yml file becomes the blueprint for your entire application stack. Anyone with Docker and Docker Compose installed can launch your Redis Cluster (or any other multi-service application) with a single command (docker-compose up). This ensures consistent environments across development, testing, and production, eliminating the infamous "it works on my machine" problem. For sharing configurations, especially on platforms like GitHub, this portability is invaluable.
Isolated and Consistent Environments: Each service in your Docker Compose application runs in its own isolated container. This prevents conflicts between dependencies and ensures that each Redis node operates within a clean, dedicated environment. Docker Compose also creates a default network for your services, allowing them to communicate with each other using their service names, which simplifies inter-service connectivity.
Version Control Friendly: Because the entire configuration is defined in a single text file, it can be easily version-controlled using Git and hosted on platforms like GitHub. This allows for tracking changes, collaborating with teams, and rolling back to previous configurations if needed. A well-documented docker-compose.yml within a GitHub repository serves as a self-contained, executable specification of your infrastructure.

Key Docker Compose Concepts for Redis Cluster Setup

To effectively use Docker Compose for our Redis Cluster, we'll leverage several core concepts:

services: This top-level key defines the individual containers that make up your application. Each Redis node will be a separate service. We will define a unique service for each master and slave node (e.g., redis-node-1 to redis-node-6).
- image: Specifies the Docker image to use for the service (e.g., redis:7.2.4-alpine). Using a specific version is crucial for reproducibility and stability.
- container_name: Assigns a predictable name to the container, making it easier to reference.
- ports: Maps container ports to host ports. For Redis Cluster, it's essential that clients can reach all master nodes (and often their replicas for topology discovery), so we'll map distinct external ports for each Redis node. Remember, Redis Cluster also uses a cluster bus port which is typically the client port + 10000.
- volumes: Mounts host paths or named volumes into containers for persistent storage. This is critical for Redis to ensure data is not lost when containers are stopped or recreated. Each Redis node will need its own dedicated volume for its data directory (/data) and its nodes.conf file.
- networks: Defines custom bridge networks. By default, Compose creates a single network for all services. However, explicit network definition allows for better isolation and naming. We'll use a dedicated network for our Redis Cluster for clear segmentation.
- environment: Passes environment variables to the container. These are invaluable for dynamic configuration, such as setting Redis passwords, or, crucially for Redis Cluster in Docker, for instructing Redis to announce its correct external IP and port.
- command or entrypoint: Overrides the default command/entrypoint of the Docker image. We'll use this to start the redis-server with our specific configuration file.
- healthcheck: Defines how to check if a containerized service is ready and healthy. This is vital for our cluster initialization script to ensure all Redis instances are actually up and responsive before attempting to form the cluster.

By meticulously defining these elements within our docker-compose.yml file, we can create a highly organized, self-documenting, and easily manageable Redis Cluster setup. This approach streamlines deployment, simplifies maintenance, and significantly reduces the operational overhead typically associated with distributed systems. It also makes the setup highly shareable, a perfect candidate for version control on GitHub, enabling teams to collaborate effectively on a standardized infrastructure.

Step-by-Step GitHub Setup Guide: Building Your Redis Cluster

Now, let's roll up our sleeves and build the Docker Compose Redis Cluster. This section provides detailed instructions, code snippets, and explanations to guide you through each stage, culminating in a functional and persistent Redis Cluster hosted on GitHub.

Prerequisites

Before you begin, ensure you have the following installed on your system:

Docker Desktop (or Docker Engine): Required to run Docker containers and Docker Compose. Download and install from the official Docker website.
Git: Essential for version control and interacting with GitHub. Install from git-scm.com.
A Text Editor/IDE: Such as VS Code, Sublime Text, or Atom, for editing configuration files.
Basic Command Line Familiarity: You'll be using your terminal extensively.

Step 1: Initialize Your GitHub Repository and Project Structure

First, create a new directory for your project and initialize a Git repository. This will be the root of your Redis Cluster configuration.

mkdir redis-cluster-docker
cd redis-cluster-docker
git init

Next, let's establish a clear and organized project structure. This enhances readability and maintainability, especially when collaborating on GitHub.

redis-cluster-docker/
├── docker-compose.yml         # Defines all Redis services
├── entrypoint.sh              # Custom entrypoint for Redis containers
├── redis.tmpl.conf            # Template for Redis configuration
├── init-cluster.sh            # Script to initialize the Redis Cluster
├── .gitignore                 # Specifies files/directories to ignore in Git
└── README.md                  # Project documentation and usage instructions

Step 2: Crafting the `redis.tmpl.conf` (Redis Configuration Template)

Instead of having six identical redis.conf files, we'll create a single template and inject environment variables at runtime for dynamic configuration. This significantly reduces redundancy. Create a file named redis.tmpl.conf in your redis-cluster-docker directory:

# redis.tmpl.conf

# Standard Redis configuration
bind 0.0.0.0             # Allow connections from any IP inside the Docker network
port 6379                # Default Redis client port inside the container
protected-mode no        # Disable protected mode for Docker environments.
                         # In production, ensure Docker network isolation and
                         # strong passwords are used to compensate.

# Persistence configuration
appendonly yes           # Enable AOF (Append Only File) persistence
appendfsync everysec     # Sync AOF to disk every second (good balance of performance/durability)
dir /data                # Directory where RDB snapshot and AOF files will be stored

# Cluster specific configuration
cluster-enabled yes      # Enable Redis Cluster mode
cluster-config-file nodes.conf # The cluster configuration file generated by Redis
cluster-node-timeout 5000 # Timeout in milliseconds for a node to be considered failed
cluster-replica-validity-factor 10 # Replicas will try to reconnect master after this many seconds
cluster-migration-barrier 1 # Minimum number of replicas a master must have to migrate a replica
cluster-require-full-coverage no # Allow cluster to operate when some hash slots are not covered.
                                 # Set to yes for strict production environments if you prefer
                                 # the cluster to stop instead of having uncovered slots.

# Security settings
requirepass ${REDIS_PASSWORD} # Require a password for client connections
masterauth ${REDIS_PASSWORD}  # Password for slaves to authenticate with masters

# Crucial for Docker Compose in a cluster: announce IP and port
# These values will be dynamically injected via environment variables
# during container startup, allowing external clients to correctly
# connect to the cluster after receiving redirection messages.
cluster-announce-ip ${REDIS_CLUSTER_ANNOUNCE_IP}
cluster-announce-port ${REDIS_CLUSTER_ANNOUNCE_PORT}
cluster-announce-bus-port ${REDIS_CLUSTER_ANNOUNCE_BUS_PORT}

Explanation of Key Settings:

bind 0.0.0.0: Makes Redis listen on all available network interfaces within the Docker container, allowing other containers in the Docker network to connect.
protected-mode no: Disables a security feature that prevents connections from non-local clients. This is typically set to no in Docker environments because network isolation is managed by Docker itself. Crucially, ensure strong passwords (requirepass) are always used.
appendonly yes & appendfsync everysec: Configures Redis to use AOF persistence, which is generally more robust for data durability than RDB snapshots, as it logs every write operation. everysec offers a good balance between performance and data safety.
dir /data: Specifies the directory inside the container where Redis will store its persistence files (appendonly.aof, dump.rdb, nodes.conf). This directory will be mapped to a Docker volume to ensure data persistence across container restarts.
cluster-enabled yes: This is the most important setting, enabling Redis Cluster mode for the instance.
cluster-config-file nodes.conf: Redis Cluster manages its state and topology in this file. It's automatically generated and updated by Redis. It must be unique per node and should persist.
cluster-node-timeout 5000: The maximum amount of time (in milliseconds) a Redis Cluster node can be unreachable before it's considered to be down by the rest of the cluster. A shorter timeout means faster failure detection but potentially more false positives in high-latency networks.
cluster-require-full-coverage no: By default (yes), if even one hash slot is not covered by a master (e.g., due to a master failure without a healthy replica), the entire cluster stops accepting writes. Setting it to no allows the cluster to continue operating for the available slots. This is often preferred in development or less strict production scenarios.
requirepass ${REDIS_PASSWORD}: Enforces client authentication. Never use a Redis Cluster without a password in production. The ${REDIS_PASSWORD} placeholder will be replaced by an environment variable.
masterauth ${REDIS_PASSWORD}: Required for slave nodes to authenticate with their masters.
cluster-announce-ip, cluster-announce-port, cluster-announce-bus-port: These are critical for Redis Cluster running in Docker Compose, especially when interacting with clients outside the Docker network. Redis nodes, by default, announce their internal Docker IP and port. However, external clients need to be redirected to the host IP and mapped external port. These placeholders will be filled dynamically to ensure proper redirection. For local development, REDIS_CLUSTER_ANNOUNCE_IP will typically be 127.0.0.1 or the Docker host's IP.

Step 3: Creating the `entrypoint.sh` Script

This script will be executed when each Redis container starts. Its primary role is to process redis.tmpl.conf using the environment variables and then start the redis-server. Create entrypoint.sh in the project root:

#!/bin/sh

# entrypoint.sh

# Wait for a few seconds to ensure Docker network is fully established
# This can help prevent initial network related issues, though healthchecks are more robust.
sleep 3

# Use envsubst to replace placeholders in redis.tmpl.conf
# This ensures that cluster-announce-ip, cluster-announce-port,
# cluster-announce-bus-port, and REDIS_PASSWORD are dynamically
# injected into the Redis configuration.
echo "INFO: Generating redis.conf from template with dynamic values..."
envsubst < /usr/local/etc/redis/redis.tmpl.conf > /usr/local/etc/redis/redis.conf

# Check if the generated redis.conf exists and has content
if [ ! -s /usr/local/etc/redis/redis.conf ]; then
    echo "ERROR: Failed to generate redis.conf or it's empty."
    exit 1
fi

echo "INFO: Starting Redis server..."
# Execute the original Redis entrypoint (redis-server) with our custom config file.
# The `exec` command replaces the current shell with the Redis server process,
# ensuring signals are properly forwarded to Redis.
exec redis-server /usr/local/etc/redis/redis.conf

Make the entrypoint.sh executable:

chmod +x entrypoint.sh

Explanation:

#!/bin/sh: Shebang line, specifies the interpreter.
sleep 3: A small delay, sometimes helpful for network readiness, though healthcheck in docker-compose.yml is more reliable.
envsubst < /usr/local/etc/redis/redis.tmpl.conf > /usr/local/etc/redis/redis.conf: This crucial command reads our template, substitutes all ${VAR_NAME} placeholders with the values from the container's environment variables, and writes the result to a new redis.conf file. envsubst is often found in gettext package, which is usually pre-installed in alpine based images or can be easily added.
exec redis-server ...: Starts the Redis server using our generated configuration. exec is important for proper signal handling in Docker.

Step 4: Designing `docker-compose.yml` (The Cluster Blueprint)

This is the core of our setup. We will define six Redis services (three masters, three slaves), a dedicated network, and persistent volumes. Create docker-compose.yml in your project root:

# docker-compose.yml

version: '3.8'

# Define custom named volumes for each Redis node to ensure data persistence
volumes:
  redis-data-1:
  redis-data-2:
  redis-data-3:
  redis-data-4:
  redis-data-5:
  redis-data-6:

services:
  # Redis Node 1 (Potential Master)
  redis-node-1:
    image: redis:7.2.4-alpine # Using a specific, stable Redis image
    container_name: redis-node-1
    hostname: redis-node-1 # Sets the hostname inside the container
    # Mount our custom entrypoint and template config
    # Mount a named volume for persistent data storage
    volumes:
      - ./entrypoint.sh:/usr/local/bin/entrypoint.sh # Custom entrypoint
      - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf # Config template
      - redis-data-1:/data # Persistent volume for this node's data
    ports:
      - "7000:6379" # Client port mapping
      - "17000:16379" # Cluster bus port mapping (client port + 10000)
    networks:
      - redis-cluster-net # Connect to our custom network
    environment:
      # These environment variables are crucial for the entrypoint.sh
      # to configure Redis for cluster mode and proper external announcement.
      - REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123} # Default password if not set
      - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1 # IP that this node advertises to the cluster for client redirection
      - REDIS_CLUSTER_ANNOUNCE_PORT=7000 # Mapped external client port
      - REDIS_CLUSTER_ANNOUNCE_BUS_PORT=17000 # Mapped external cluster bus port
    command: ["/usr/local/bin/entrypoint.sh"] # Use our custom entrypoint
    healthcheck: # Healthcheck ensures Redis is running and responsive
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_password_123}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s # Give Redis time to start up initially

  # Redis Node 2 (Potential Master)
  redis-node-2:
    image: redis:7.2.4-alpine
    container_name: redis-node-2
    hostname: redis-node-2
    volumes:
      - ./entrypoint.sh:/usr/local/bin/entrypoint.sh
      - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf
      - redis-data-2:/data
    ports:
      - "7001:6379"
      - "17001:16379"
    networks:
      - redis-cluster-net
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123}
      - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1
      - REDIS_CLUSTER_ANNOUNCE_PORT=7001
      - REDIS_CLUSTER_ANNOUNCE_BUS_PORT=17001
    command: ["/usr/local/bin/entrypoint.sh"]
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_password_123}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s

  # Redis Node 3 (Potential Master)
  redis-node-3:
    image: redis:7.2.4-alpine
    container_name: redis-node-3
    hostname: redis-node-3
    volumes:
      - ./entrypoint.sh:/usr/local/bin/entrypoint.sh
      - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf
      - redis-data-3:/data
    ports:
      - "7002:6379"
      - "17002:16379"
    networks:
      - redis-cluster-net
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123}
      - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1
      - REDIS_CLUSTER_ANNOUNCE_PORT=7002
      - REDIS_CLUSTER_ANNOUNCE_BUS_PORT=17002
    command: ["/usr/local/bin/entrypoint.sh"]
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_password_123}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s

  # Redis Node 4 (Potential Slave for Node 1)
  redis-node-4:
    image: redis:7.2.4-alpine
    container_name: redis-node-4
    hostname: redis-node-4
    volumes:
      - ./entrypoint.sh:/usr/local/bin/entrypoint.sh
      - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf
      - redis-data-4:/data
    ports:
      - "7003:6379"
      - "17003:16379"
    networks:
      - redis-cluster-net
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123}
      - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1
      - REDIS_CLUSTER_ANNOUNCE_PORT=7003
      - REDIS_CLUSTER_ANNOUNCE_BUS_PORT=17003
    command: ["/usr/local/bin/entrypoint.sh"]
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_password_123}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s

  # Redis Node 5 (Potential Slave for Node 2)
  redis-node-5:
    image: redis:7.2.4-alpine
    container_name: redis-node-5
    hostname: redis-node-5
    volumes:
      - ./entrypoint.sh:/usr/local/bin/entrypoint.sh
      - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf
      - redis-data-5:/data
    ports:
      - "7004:6379"
      - "17004:16379"
    networks:
      - redis-cluster-net
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123}
      - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1
      - REDIS_CLUSTER_ANNOUNCE_PORT=7004
      - REDIS_CLUSTER_ANNOUNCE_BUS_PORT=17004
    command: ["/usr/local/bin/entrypoint.sh"]
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_password_123}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s

  # Redis Node 6 (Potential Slave for Node 3)
  redis-node-6:
    image: redis:7.2.4-alpine
    container_name: redis-node-6
    hostname: redis-node-6
    volumes:
      - ./entrypoint.sh:/usr/local/bin/entrypoint.sh
      - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf
      - redis-data-6:/data
    ports:
      - "7005:6379"
      - "17005:16379"
    networks:
      - redis-cluster-net
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123}
      - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1
      - REDIS_CLUSTER_ANNOUNCE_PORT=7005
      - REDIS_CLUSTER_ANNOUNCE_BUS_PORT=17005
    command: ["/usr/local/bin/entrypoint.sh"]
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-redis_password_123}", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s

# Define a custom bridge network for internal communication
networks:
  redis-cluster-net:
    driver: bridge
    # You can specify a custom subnet if needed for advanced network setups
    # ipam:
    #   driver: default
    #   config:
    #     - subnet: 172.20.0.0/24

Explanation of docker-compose.yml:

version: '3.8': Specifies the Docker Compose file format version.
volumes: Defines named volumes (redis-data-1 to redis-data-6). Each node gets its own volume to ensure its /data directory (which contains AOF, RDB, and nodes.conf) persists independently. If a container is removed and recreated, its data will be reattached.
services: We define six services, redis-node-1 through redis-node-6. Each service block is largely identical, except for the container_name, hostname, ports, and volumes mappings.
- image: redis:7.2.4-alpine: We opt for a specific Redis version (7.2.4) and the alpine variant, which is lightweight and secure.
- volumes:
  - ./entrypoint.sh:/usr/local/bin/entrypoint.sh: Mounts our custom entrypoint script into the container, making it executable.
  - ./redis.tmpl.conf:/usr/local/etc/redis/redis.tmpl.conf: Mounts our template configuration. The entrypoint.sh will then process this.
  - redis-data-X:/data: Mounts a unique named volume for each node's data persistence.
- ports: This is critical.
  - "700X:6379": Maps a unique host port (e.g., 7000, 7001) to the container's internal Redis client port (6379). This allows external clients (like redis-cli) to connect to individual nodes.
  - "1700X:16379": Maps a unique host port (e.g., 17000, 17001) to the container's internal Redis Cluster bus port (16379). This is 6379 + 10000. It ensures that when nodes advertise their cluster bus port for gossip protocol and state synchronization, they correctly point to the host's accessible port, rather than an internal container-only port.
- networks: - redis-cluster-net: All Redis nodes are connected to a single custom bridge network, allowing them to communicate with each other using their service names (e.g., redis-node-1 can reach redis-node-2 at redis-node-2:6379).
- environment:
  - REDIS_PASSWORD: Sets the password. We use ${REDIS_PASSWORD:-redis_password_123} to allow setting it via a .env file or environment variable, with a default fallback. Change this default to a strong, unique password!
  - REDIS_CLUSTER_ANNOUNCE_IP=127.0.0.1: For local development, 127.0.0.1 is used. In a cloud or multi-host environment, this would be the actual public/private IP of the host machine running these Docker containers. This is the IP that Redis will tell clients to connect to if they need to be redirected to this particular node.
  - REDIS_CLUSTER_ANNOUNCE_PORT: This must match the externally mapped client port (e.g., 7000, 7001) for that specific node.
  - REDIS_CLUSTER_ANNOUNCE_BUS_PORT: This must match the externally mapped cluster bus port (e.g., 17000, 17001) for that specific node.
- command: ["/usr/local/bin/entrypoint.sh"]: Overrides the default Redis container command to execute our custom entrypoint script.
- healthcheck: Defines a health check that uses redis-cli to ping the Redis server. This tells Docker Compose (and any orchestrator) when a container is truly ready to accept connections. The start_period gives Redis enough time to initialize before health check failures start counting.
networks: Defines the redis-cluster-net as a simple bridge network. This creates an isolated network for our Redis services.

Security and Best Practices:

Strong Passwords: Always use strong, randomly generated passwords for REDIS_PASSWORD. For production, manage this securely (e.g., Docker Secrets, Kubernetes Secrets, Vault).
REDIS_CLUSTER_ANNOUNCE_IP: For cloud deployments, this IP must be the publicly accessible IP of the host running the Docker containers, or an internal network IP reachable by your clients. Misconfiguring this is a common source of Redis Cluster client redirection issues.
Firewall Rules: Ensure that the mapped ports (7000-7005 and 17000-17005) are open on your host's firewall if you intend to access the cluster from other machines.
Resource Limits: For production, consider adding deploy.resources limits in Docker Compose to restrict CPU and memory usage for each Redis node, preventing a single node from consuming all host resources.

Step 5: Creating the `init-cluster.sh` Script (Cluster Initialization)

After all Redis nodes are up and healthy, they need to be told to form a cluster. This script automates that process. Create init-cluster.sh in your project root:

#!/bin/sh

# init-cluster.sh

# Load environment variables if they exist (e.g., from a .env file)
if [ -f .env ]; then
  export $(grep -v '^#' .env | xargs)
fi

REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123} # Use the same default or provided password

echo "Waiting for Redis containers to be healthy..."
# Loop and check the health of each Redis service.
# This ensures all nodes are fully ready before attempting cluster creation.
for i in $(seq 1 6); do
  until [ "$(docker inspect -f '{{.State.Health.Status}}' redis-node-$i)" = "healthy" ]; do
    echo "redis-node-$i is not yet healthy... waiting"
    sleep 5
  done
  echo "redis-node-$i is healthy!"
done

echo "All Redis nodes are healthy. Proceeding to create the cluster."

# List all node IPs and client ports.
# Since we are creating the cluster from the host, we use 127.0.0.1 and the mapped external ports.
# The --cluster-replicas 1 option tells Redis to create 1 slave for each master.
# The -a (or --cluster-password) option provides the password for authentication.
docker run --rm --network host redis:7.2.4-alpine redis-cli \
  --cluster create \
  127.0.0.1:7000 \
  127.0.0.1:7001 \
  127.0.0.1:7002 \
  127.0.0.1:7003 \
  127.0.0.1:7004 \
  127.0.0.1:7005 \
  --cluster-replicas 1 \
  --cluster-yes \
  -a "${REDIS_PASSWORD}"

# The previous command assigns 3 masters and 3 slaves randomly.
# To check the cluster info:
echo "Verifying cluster status..."
redis_cli_command="redis-cli -c -h 127.0.0.1 -p 7000 -a \"${REDIS_PASSWORD}\" cluster info"
echo "Executing: $redis_cli_command"
docker run --rm --network host redis:7.2.4-alpine sh -c "${redis_cli_command}"

echo ""
echo "Redis Cluster setup complete! You can now connect using:"
echo "redis-cli -c -h 127.0.0.1 -p 7000 -a \"${REDIS_PASSWORD}\""
echo "Remember to use the -c flag for cluster mode."

Make the init-cluster.sh executable:

chmod +x init-cluster.sh

Explanation of init-cluster.sh:

REDIS_PASSWORD=${REDIS_PASSWORD:-redis_password_123}: Ensures the script uses the correct password for redis-cli authentication, either from an environment variable or the default.
for i in $(seq 1 6); do ... done: This loop iterates through all six Redis nodes, checking their health status using docker inspect. The script waits until all nodes report healthy before proceeding. This is critical for reliable cluster creation.
docker run --rm --network host redis:7.2.4-alpine redis-cli ...: This command creates a temporary redis-cli container.
- --rm: Automatically removes the container after it exits.
- --network host: Allows the redis-cli container to communicate directly with the host's network interfaces, enabling it to connect to 127.0.0.1 and the externally mapped ports (7000-7005).
- --cluster create ...: This is the Redis Cluster command to form a new cluster.
  - It takes a list of IP:Port pairs for all nodes that will participate in the cluster. We list all six externally mapped ports.
  - --cluster-replicas 1: Instructs redis-cli to assign one replica (slave) for each master node. Given six nodes, this will result in 3 masters and 3 slaves.
  - --cluster-yes: Confirms the cluster creation prompt automatically.
  - -a "${REDIS_PASSWORD}": Provides the password for authentication to the Redis nodes.
redis-cli -c -h 127.0.0.1 -p 7000 -a "${REDIS_PASSWORD}" cluster info: After cluster creation, this command connects to one of the cluster nodes (node 1, port 7000) in cluster mode (-c) and fetches the cluster info, which shows the health, state, and topology of the newly formed cluster.

Step 6: Create `.env` File (Optional but Recommended)

To manage your Redis password securely without hardcoding it directly in docker-compose.yml or init-cluster.sh, create a .env file in the project root:

# .env
REDIS_PASSWORD=YourSecureRedisPasswordHere

Replace YourSecureRedisPasswordHere with a strong, unique password. Docker Compose automatically loads environment variables from a .env file if it's present in the same directory as docker-compose.yml. The init-cluster.sh also explicitly loads it.

Step 7: Create `README.md`

A good README.md is crucial for any GitHub project. It should explain what the project is, how to set it up, and how to use it.

# Docker Compose Redis Cluster: GitHub Setup Guide

This repository provides a robust and easily deployable Redis Cluster setup using Docker Compose. The configuration is designed for high availability and scalability, suitable for development and testing environments, and serves as a solid foundation for production deployments.

## Features

*   **Redis Cluster:** 3 Master nodes, 3 Slave nodes for fault tolerance and horizontal scaling.
*   **Docker Compose:** Simplifies orchestration of multi-container Redis setup.
*   **Persistence:** Each Redis node uses a dedicated Docker volume for data persistence.
*   **Security:** Password-protected Redis instances.
*   **Dynamic Configuration:** Uses an `entrypoint.sh` script and `redis.tmpl.conf` for flexible configuration via environment variables.
*   **GitHub Ready:** Designed for easy sharing, collaboration, and version control.

## Project Structure

redis-cluster-docker/ ├── docker-compose.yml # Defines all Redis services, volumes, networks, and environment variables. ├── entrypoint.sh # Custom entrypoint script for Redis containers to dynamically configure Redis. ├── redis.tmpl.conf # Template for Redis configuration, processed by entrypoint.sh. ├── init-cluster.sh # Script to initialize the Redis Cluster after containers are up. ├── .env # Environment variables, especially for the Redis password (ignored by Git). ├── .gitignore # Specifies files/directories to ignore in Git. └── README.md # This documentation file.


## Prerequisites

*   **Docker Desktop** (or Docker Engine) installed and running.
*   **Git** installed.
*   Basic familiarity with Docker and Redis Cluster concepts.


> [APIPark](https://apipark.com/) is a high-performance AI gateway that allows you to securely access the most comprehensive LLM APIs globally on the [APIPark](https://apipark.com/) platform, including OpenAI, Anthropic, Mistral, Llama2, Google Gemini, and more.Try [APIPark](https://apipark.com/) now! 👇👇👇

<div class="kg-card kg-button-card kg-align-center"><a href="https://github.com/APIParkLab/APIPark?ref=8.222.204.118" class="kg-btn kg-btn-accent">Install APIPark – it’s
free</a></div>

## Setup and Usage

Follow these steps to get your Redis Cluster up and running:

### 1. Clone the Repository (or create files manually)

```bash
git clone https://github.com/your-username/redis-cluster-docker.git
cd redis-cluster-docker

If you manually created the files, you can skip git clone and simply ensure you are in the redis-cluster-docker directory.

2. Configure Environment Variables

Create a .env file in the project root directory and define your Redis password. This is crucial for security.

# .env
REDIS_PASSWORD=YourSecureRedisPasswordHere

Replace YourSecureRedisPasswordHere with a strong, unique password.

3. Start the Redis Containers

Docker Compose will build and start all six Redis nodes, create the necessary networks and volumes.

docker-compose up -d

The -d flag runs the containers in detached mode (in the background).

4. Initialize the Redis Cluster

Once all containers are running and healthy, run the init-cluster.sh script to form the Redis Cluster. This script will wait for all nodes to be healthy before proceeding.

./init-cluster.sh

This script will output information about the cluster creation and its final status.

5. Interact with the Redis Cluster

You can now connect to your Redis Cluster using redis-cli. Remember to use the -c flag for cluster mode and the password.

# Example: Connect to the first node's client port
docker run --rm --network host redis:7.2.4-alpine redis-cli -c -h 127.0.0.1 -p 7000 -a "YourSecureRedisPasswordHere"

# Once connected, you can perform Redis operations:
127.0.0.1:7000> SET mykey "Hello Redis Cluster"
-> Redirected to slot 14614...
127.0.0.1:7001> "OK"
127.0.0.1:7001> GET mykey
"Hello Redis Cluster"
127.0.0.1:7001> CLUSTER NODES
# ... (outputs cluster topology)
127.0.0.1:7001> CLUSTER INFO
# ... (outputs cluster state)

Note: Replace YourSecureRedisPasswordHere with the actual password you set in .env.

6. Stop and Remove the Cluster

To stop the running containers:

docker-compose stop

To stop and remove all containers, networks, and volumes (this will delete your Redis data!):

docker-compose down -v

Use -v to remove the named volumes. If you omit -v, the data volumes will persist, allowing you to restart the cluster with existing data.

Advanced Considerations

Production Deployment: For production, consider using a more robust orchestration tool like Kubernetes, which natively handles service discovery, scaling, and secrets management. This Docker Compose setup can serve as an excellent local development and testing environment.
Monitoring: Integrate with Prometheus/Grafana for detailed Redis metrics.
Backup & Restore: Implement a strategy for backing up your Redis data volumes.
Security: Ensure proper firewall rules are in place for mapped ports, especially in public-facing environments. Consider using TLS for Redis client connections.
API Management: When deploying applications that leverage this Redis Cluster, especially in a microservices architecture, managing access to these services often involves an API gateway. An API gateway acts as a single entry point for all apis, handling authentication, authorization, routing, rate limiting, and analytics. For example, platforms like APIPark provide comprehensive API management capabilities, helping you securely expose and govern the functionalities built on top of robust data layers like our Redis Cluster. This ensures that while Redis provides the high-performance data backbone, the external access to your application logic is well-controlled and optimized.

Contributing

Feel free to open issues or submit pull requests to improve this guide and configuration.

License

This project is open-sourced under the MIT License.


### 8. Push to GitHub

Finally, commit your changes and push them to your GitHub repository.

```bash
git add .
git commit -m "Initial setup of Docker Compose Redis Cluster"
# Replace with your actual GitHub repository URL
git remote add origin https://github.com/your-username/redis-cluster-docker.git
git push -u origin master

Now, your complete Docker Compose Redis Cluster setup is living on GitHub, ready for collaboration, deployment, and usage!

Table: Docker Compose Redis Node Configuration Summary

Service Name	External Client Port	External Bus Port	Volume Name	`REDIS_CLUSTER_ANNOUNCE_PORT`	`REDIS_CLUSTER_ANNOUNCE_BUS_PORT`	Role (After Init)
`redis-node-1`	`7000`	`17000`	`redis-data-1`	`7000`	`17000`	Master
`redis-node-2`	`7001`	`17001`	`redis-data-2`	`7001`	`17001`	Master
`redis-node-3`	`7002`	`17002`	`redis-data-3`	`7002`	`17002`	Master
`redis-node-4`	`7003`	`17003`	`redis-data-4`	`7003`	`17003`	Slave (of node 1)
`redis-node-5`	`7004`	`17004`	`redis-data-5`	`7004`	`17004`	Slave (of node 2)
`redis-node-6`	`7005`	`17005`	`redis-data-6`	`7005`	`17005`	Slave (of node 3)

(Note: The exact master/slave assignments can vary slightly depending on how redis-cli --cluster create processes the input nodes, but the 3 Masters, 3 Slaves ratio will be maintained.)

Advanced Considerations and Best Practices for Your Redis Cluster

While the Docker Compose setup provides a robust foundation, deploying a Redis Cluster, especially in production, warrants deeper consideration of several advanced topics. These practices enhance security, manageability, and overall system resilience.

Persistence Strategy: AOF vs. RDB

Redis offers two primary mechanisms for data persistence:

RDB (Redis Database) Snapshots: This method takes point-in-time snapshots of your dataset at specified intervals. RDB files are compact and optimized for disaster recovery, making them excellent for backups. However, if Redis crashes between snapshots, you might lose some recent data. The redis.tmpl.conf includes dir /data for where RDB snapshots would reside if enabled.
AOF (Append Only File): AOF logs every write operation received by the server. When Redis restarts, it replays the AOF to reconstruct the dataset. AOF typically provides better data durability than RDB, especially with an appendfsync everysec policy, where data loss is limited to a maximum of one second's worth of writes. This is the strategy we've opted for in our redis.tmpl.conf.

Best Practice: For critical data, a combination of both AOF and periodic RDB snapshots is often recommended. AOF offers finer-grained durability, while RDB provides efficient, compact backups for long-term archival and faster full recovery. Ensure your chosen persistence method is robustly tested and that the /data volumes are properly backed up off-host.

Security Hardening

Security is paramount for any production system, especially for a data store like Redis.

Strong Passwords (requirepass): We've already implemented requirepass and masterauth. Never use default or weak passwords. For production, leverage secret management solutions (e.g., Docker Secrets, Kubernetes Secrets, HashiCorp Vault) to inject these passwords securely, rather than hardcoding or relying solely on .env files.
Network Isolation: Docker Compose creates a default bridge network, which offers a degree of isolation. However, for true production security, consider custom Docker networks with stricter ingress/egress rules, or deploy in an environment like Kubernetes where network policies can control inter-service communication granularly. The mapped external ports (7000-7005, 17000-17005) expose your Redis nodes to the host network.
Firewall Rules: On the host machine, configure firewall rules (e.g., ufw on Linux, Windows Firewall) to restrict access to the Redis client ports (7000-7005) and cluster bus ports (17000-17005) only to trusted IPs or networks. This is a critical layer of defense.
protected-mode: While we disabled protected-mode in redis.tmpl.conf for Docker convenience, in environments where Docker's network isolation isn't sufficient or for non-Docker deployments, re-enabling it (protected-mode yes) along with bind to specific internal IPs is a strong security measure.
TLS/SSL: For enhanced security, especially over untrusted networks, consider enabling TLS/SSL encryption for Redis client connections. This requires additional configuration and potentially a TLS-enabled Redis proxy or a Redis version compiled with TLS support.

Monitoring and Alerting

A healthy Redis Cluster is a monitored Redis Cluster.

Redis INFO: The INFO command provides a wealth of metrics about the Redis server, clients, memory, persistence, and replication. Regularly collect and analyze this data.
CLUSTER INFO and CLUSTER NODES: These commands are essential for understanding the cluster's health, topology, and identifying potential issues like failed nodes or uncovered slots.
External Monitoring Tools: Integrate Redis metrics with comprehensive monitoring solutions like Prometheus and Grafana. Exporters like redis_exporter can expose Redis metrics in a format consumable by Prometheus, allowing for historical trend analysis and custom dashboards.
Alerting: Set up alerts based on key metrics (e.g., memory usage, client connections, replication lag, cluster health status) to proactively detect and respond to issues before they impact your application.

Scaling and Resharding

Redis Cluster is designed for dynamic scaling.

Adding Nodes: You can add new master nodes to increase capacity (memory and throughput) or new replica nodes to enhance fault tolerance. Redis provides redis-cli --cluster add-node and redis-cli --cluster reshard commands to seamlessly integrate new nodes and redistribute hash slots.
Removing Nodes: Similarly, nodes can be gracefully removed from the cluster using redis-cli --cluster del-node. This involves migrating their hash slots and replicas to other healthy nodes before decommissioning.

Backup and Restore Strategies

Despite persistence, external backups are crucial.

Volume Backups: For Docker-based deployments, regularly back up the Docker volumes mounted to your Redis nodes. This can involve stopping the containers, copying the volume data, or using Docker volume backup tools.
redis-cli SAVE / BGSAVE: You can manually trigger RDB saves for backup purposes. For AOF persistence, simply backing up the AOF file is sufficient.
Cloud Provider Snapshots: If deploying on cloud VMs, leverage cloud provider snapshot features for your disk volumes.

Client Libraries and Application Integration

Modern Redis client libraries are "cluster-aware."

When connecting to a Redis Cluster, clients typically need a list of one or more seed nodes. The client then connects to one of these nodes, fetches the cluster topology (slot-to-node mapping), and caches it.
Upon subsequent requests, if a client attempts to access a key on the wrong node, it will receive a redirection command (MOVED or ASK) from Redis. The client library handles this transparently by updating its topology cache and re-issuing the command to the correct node.
Ensure your application's Redis client library supports Redis Cluster mode. Popular libraries (e.g., redis-py-cluster for Python, lettuce for Java, ioredis for Node.js) provide this functionality.

The Role of an API Gateway in a Modern Architecture

In a microservices ecosystem where a robust Redis Cluster provides a high-performance data layer, the way applications expose their functionalities becomes critical. This is where an API gateway becomes an indispensable architectural component.

An API gateway acts as a single, centralized entry point for all client requests to your backend services. It abstracts the complexity of your microservices architecture, providing a consistent interface for consuming APIs. For instance, imagine your application has multiple services that interact with the Redis Cluster for caching user sessions, storing real-time analytics, or managing game states. Instead of clients directly calling each of these services, they route through the API gateway.

Platforms like APIPark offer comprehensive API management solutions that fit perfectly into this scenario. By utilizing APIPark as your API gateway, you can:

Centralize Authentication and Authorization: Secure access to your microservices that leverage Redis by enforcing security policies at the gateway level. This ensures that only authorized applications or users can interact with your backend apis.
Traffic Management: Handle load balancing, request routing, and rate limiting. For example, if a specific service backed by Redis is experiencing high load, the API gateway can intelligently distribute requests or apply throttling to prevent overload.
Monitoring and Analytics: Gain insights into API usage, performance, and error rates, providing a holistic view of how your backend services (including those interacting with Redis) are performing.
Protocol Transformation: If your internal services use different protocols, the API gateway can translate them into a unified external api format.
Developer Portal: Provide a self-service portal for developers to discover, subscribe to, and test your apis, simplifying integration.

Integrating a powerful API gateway like APIPark ensures that your applications, while benefiting from the scalability and performance of a Docker Compose Redis Cluster, also maintain high standards of security, manageability, and accessibility for their exposed apis. It creates a robust bridge between your backend infrastructure and the consuming client applications.

Troubleshooting Common Issues

Even with a well-defined setup, you might encounter issues. Here are some common problems and their solutions:

Nodes not joining the cluster:
- Symptom: CLUSTER INFO shows nodes in fail state, or some nodes are not listed in CLUSTER NODES.
- Possible Causes:
  - Incorrect cluster-announce-ip or cluster-announce-port: The nodes are advertising unreachable IPs/ports. Double-check the environment variables in docker-compose.yml.
  - Firewall issues: Host firewall blocking communication on client or bus ports.
  - Incorrect REDIS_PASSWORD: Nodes cannot authenticate with each other. Ensure it's consistent across all nodes and in init-cluster.sh.
  - cluster-config-file issues: If nodes.conf gets corrupted or misconfigured, clear the /data volume for the problematic node (use docker-compose down -v with caution, or target specific volumes).
- Solution: Verify all ports mappings, environment variables, and check docker logs <container_name> for specific error messages. Ensure firewall rules are permissive during initial setup.
Client Redirection Issues (MOVED errors without proper handling):
- Symptom: When using redis-cli without -c or a non-cluster-aware client, you get (error) MOVED <slot> <IP>:<Port> and the client doesn't automatically reconnect.
- Possible Causes:
  - Client is not cluster-aware: Ensure you use redis-cli -c or a Redis Cluster-aware client library.
  - cluster-announce-ip is wrong: The IP and port in the MOVED response are unreachable by the client. This means REDIS_CLUSTER_ANNOUNCE_IP and REDIS_CLUSTER_ANNOUNCE_PORT are not correctly set for the external network.
- Solution: Always use a cluster-aware client. Double-check REDIS_CLUSTER_ANNOUNCE_IP (should be 127.0.0.1 for local, or the host IP for remote) and REDIS_CLUSTER_ANNOUNCE_PORT in docker-compose.yml.
CROSSSLOT error:
- Symptom: You try to perform an operation (e.g., MSET, DEL with multiple keys, Lua script) on multiple keys that belong to different hash slots.
- Possible Cause: Redis Cluster design constraint. Operations involving multiple keys must have all keys residing in the same hash slot.
- Solution:
  - For multi-key operations, ensure keys are in the same hash slot by using hash tags ({mykey}). For example, SET {user:1}name Alice and SET {user:1}email alice@example.com will put both keys in the same slot.
  - Avoid multi-key operations that span slots if possible, or refactor your application logic.
Persistence Not Working:
- Symptom: Data is lost after docker-compose down -v (which explicitly removes volumes), or after restarting containers without removing volumes.
- Possible Causes:
  - Volumes not correctly mounted: Check docker-compose.yml volume paths.
  - dir /data in redis.conf is incorrect: Ensure it matches the mounted volume path.
  - AOF or RDB not enabled: Check appendonly yes or RDB save settings in redis.tmpl.conf.
- Solution: Verify volume mounts and Redis persistence settings. Always use named volumes for persistence in Docker Compose (redis-data-X in our setup).
Cannot get config from nodes: connection refused during init-cluster.sh:
- Symptom: The cluster initialization script fails immediately.
- Possible Cause: Redis containers are not yet running or healthy when the script tries to connect.
- Solution: Ensure the init-cluster.sh script's "Waiting for Redis containers to be healthy..." loop is robust and actually waits. You can also manually check docker ps and docker inspect -f '{{.State.Health.Status}}' <container_name> to verify container states. Increase start_period for healthchecks if needed.

By understanding these common pitfalls and their respective solutions, you can proactively build and maintain a more stable and reliable Redis Cluster, ensuring that your application's data layer remains robust and performant.

Conclusion: A Resilient Foundation for Modern Applications

Setting up a highly available and scalable Redis Cluster is a fundamental step towards building robust, modern applications. Through this comprehensive guide, we've walked through the intricate process of orchestrating such a cluster using Docker Compose, leveraging its declarative power to define six interconnected Redis nodes, complete with persistence and dynamic configuration. We've explored the core principles of Redis Cluster, including data sharding, master-slave replication, and client redirection, demonstrating how these mechanisms ensure both horizontal scalability and fault tolerance.

By packaging this entire configuration into a well-structured GitHub repository, we've emphasized the importance of portability, reproducibility, and collaborative development. The docker-compose.yml, redis.tmpl.conf, entrypoint.sh, and init-cluster.sh scripts provide a self-contained blueprint that can be deployed with minimal effort, whether for local development, testing, or as a robust starting point for production-grade environments. The explicit mapping of external ports and the dynamic configuration of cluster-announce-ip were highlighted as crucial elements for ensuring seamless client interaction and cluster operability within a Dockerized context.

Furthermore, we delved into advanced considerations such as persistence strategies, rigorous security practices, proactive monitoring, and the dynamic scaling capabilities inherent in Redis Cluster. We also touched upon the broader architectural context, where a resilient data backend like our Redis Cluster seamlessly integrates with advanced API management solutions. The discussion around api gateways and platforms like APIPark underscored how such tools are essential for securely exposing and efficiently managing the apis that consume data from high-performance systems. This holistic approach ensures that not only is your data layer robust, but your entire application ecosystem is well-governed and optimized for performance and security.

This Docker Compose Redis Cluster setup represents more than just a collection of services; it's a foundation for applications demanding speed, reliability, and growth. As you embark on your development journey, remember that while this guide provides a powerful starting point, continuous learning, adaptation, and adherence to best practices will be key to harnessing the full potential of distributed systems. Embrace the flexibility of Docker Compose, the resilience of Redis Cluster, and the collaborative spirit of GitHub to build the next generation of innovative and reliable software.

Frequently Asked Questions (FAQs)

1. What is the minimum number of nodes required for a Redis Cluster, and why? A Redis Cluster requires a minimum of 3 master nodes, each with at least 1 replica (slave) node, totaling 6 nodes (3 masters, 3 slaves). This configuration ensures that the cluster can survive the failure of at least one master node (its slave can be promoted) and still maintain a majority of masters to continue operating. If there were only 2 masters, the failure of one would leave no majority, making the cluster unable to make critical decisions.

2. How does Redis Cluster handle data distribution and failover? Redis Cluster distributes data across master nodes by partitioning the key space into 16384 hash slots. Each master node is responsible for a subset of these slots. When a client stores a key, its hash determines which slot, and thus which master, holds the data. For failover, each master has one or more replicas. If a master node becomes unreachable, the other masters detect the failure through a gossip protocol. A consensus mechanism (requiring a majority of masters) then elects one of the failed master's replicas to take over its slots, ensuring continuous data availability.

3. Why is cluster-announce-ip and cluster-announce-port important when running Redis Cluster in Docker Compose? When Redis nodes run inside Docker containers, they typically see their own internal Docker network IP addresses. If these internal IPs/ports were advertised by default, external clients (or clients on the host network) would receive redirection messages pointing to unreachable internal Docker IPs. cluster-announce-ip and cluster-announce-port (and cluster-announce-bus-port) are crucial for telling Redis to advertise the externally accessible IP address (e.g., 127.0.0.1 for local setup, or the host's public IP for remote access) and the mapped external port that clients should use to connect to that specific node. This ensures that client redirection works correctly outside the Docker internal network.

4. How can I ensure my Redis Cluster data persists even if containers are restarted or recreated? Data persistence in our Docker Compose setup is achieved through Docker named volumes. In the docker-compose.yml, each Redis service mounts a unique named volume (e.g., redis-data-1:/data). Redis is configured (via appendonly yes in redis.tmpl.conf) to store its data and cluster configuration (nodes.conf) within the /data directory inside the container. Since Docker volumes exist independently of containers, stopping, removing, and then recreating containers (without using docker-compose down -v) will automatically reattach these volumes, preserving your data.

5. How do client applications connect to and interact with a Redis Cluster? Client applications interact with a Redis Cluster using "cluster-aware" client libraries. Instead of connecting to a single Redis instance, these libraries are given a list of one or more "seed" nodes from the cluster. The client library connects to a seed node, fetches the current cluster topology (which hash slots are owned by which master nodes), and caches this mapping. When the application performs an operation on a key, the client library internally determines the correct master node for that key's hash slot and sends the command directly to it. If the cluster topology changes (e.g., due to a failover or resharding), the client library updates its cached mapping upon receiving a redirection response from Redis, ensuring seamless operation.

🚀You can securely and efficiently call the OpenAI API on APIPark in just two steps:

Step 1: Deploy the APIPark AI gateway in 5 minutes.

APIPark is developed based on Golang, offering strong product performance and low development and maintenance costs. You can deploy APIPark with a single command line.

curl -sSO https://download.apipark.com/install/quick-start.sh; bash quick-start.sh

In my experience, you can see the successful deployment interface within 5 to 10 minutes. Then, you can log in to APIPark using your account.

Step 2: Call the OpenAI API.

Install APIPark – it’s free

Docker Compose Redis Cluster: GitHub Setup Guide

Unpacking the Fundamentals: Why Redis Cluster is Indispensable

The Limitations of a Single Redis Instance

The Power of Redis Cluster: High Availability and Scalability

Minimum Cluster Requirements

The Orchestrator: Docker Compose for Seamless Deployment

Why Docker Compose is the Ideal Tool

Key Docker Compose Concepts for Redis Cluster Setup

Step-by-Step GitHub Setup Guide: Building Your Redis Cluster

Prerequisites

Step 1: Initialize Your GitHub Repository and Project Structure

Step 2: Crafting the `redis.tmpl.conf` (Redis Configuration Template)

Step 3: Creating the `entrypoint.sh` Script

Step 4: Designing `docker-compose.yml` (The Cluster Blueprint)

Step 5: Creating the `init-cluster.sh` Script (Cluster Initialization)

Step 6: Create `.env` File (Optional but Recommended)

Step 7: Create `README.md`

2. Configure Environment Variables

3. Start the Redis Containers

4. Initialize the Redis Cluster

5. Interact with the Redis Cluster

6. Stop and Remove the Cluster

Advanced Considerations

Contributing

License

Advanced Considerations and Best Practices for Your Redis Cluster

Persistence Strategy: AOF vs. RDB

Security Hardening

Monitoring and Alerting

Scaling and Resharding

Backup and Restore Strategies

Client Libraries and Application Integration

The Role of an API Gateway in a Modern Architecture

Troubleshooting Common Issues

Conclusion: A Resilient Foundation for Modern Applications

Frequently Asked Questions (FAQs)

🚀You can securely and efficiently call the OpenAI API on APIPark in just two steps:

How to Download Claude: A Simple Guide

DeepSeek AI: Revolutionizing Large Language Models

Unpacking the Fundamentals: Why Redis Cluster is Indispensable

The Limitations of a Single Redis Instance

The Power of Redis Cluster: High Availability and Scalability

Minimum Cluster Requirements

The Orchestrator: Docker Compose for Seamless Deployment

Why Docker Compose is the Ideal Tool

Key Docker Compose Concepts for Redis Cluster Setup

Step-by-Step GitHub Setup Guide: Building Your Redis Cluster

Prerequisites

Step 1: Initialize Your GitHub Repository and Project Structure

Step 2: Crafting the redis.tmpl.conf (Redis Configuration Template)

Step 3: Creating the entrypoint.sh Script

Step 4: Designing docker-compose.yml (The Cluster Blueprint)

Step 5: Creating the init-cluster.sh Script (Cluster Initialization)

Step 6: Create .env File (Optional but Recommended)

Step 7: Create README.md

2. Configure Environment Variables

3. Start the Redis Containers

4. Initialize the Redis Cluster

5. Interact with the Redis Cluster

6. Stop and Remove the Cluster

Advanced Considerations

Contributing

License

Advanced Considerations and Best Practices for Your Redis Cluster

Persistence Strategy: AOF vs. RDB

Security Hardening

Monitoring and Alerting

Scaling and Resharding

Backup and Restore Strategies

Client Libraries and Application Integration

The Role of an API Gateway in a Modern Architecture

Troubleshooting Common Issues

Conclusion: A Resilient Foundation for Modern Applications

Frequently Asked Questions (FAQs)

🚀You can securely and efficiently call the OpenAI API on APIPark in just two steps:

How to Download Claude: A Simple Guide

DeepSeek AI: Revolutionizing Large Language Models

Step 2: Crafting the `redis.tmpl.conf` (Redis Configuration Template)

Step 3: Creating the `entrypoint.sh` Script

Step 4: Designing `docker-compose.yml` (The Cluster Blueprint)

Step 5: Creating the `init-cluster.sh` Script (Cluster Initialization)

Step 6: Create `.env` File (Optional but Recommended)

Step 7: Create `README.md`