Deploying a multi-pool Memcached setup on Bahriya

meezaan

Deploying a multi-pool Memcached setup on Bahriya

Memcached is one of those tools that does one thing and does it well — it caches objects in memory so your application does not hit the database on every request. On Bahriya, Memcached is a managed service that runs alongside your containers with project-private networking, so your cache is only accessible to the containers that need it.

This article walks through setting up multiple Memcached pools for different caching workloads — a common pattern when you want to isolate session data from application cache, or when different parts of your application have different eviction and sizing requirements.

Why multiple pools?

A single Memcached instance works fine for small applications, but as your application grows you will run into a few problems:

Eviction conflicts — a flood of short-lived API response caches can evict your long-lived session data
Sizing mismatch — sessions might need 64 MB per node while your object cache needs 512 MB
Failure isolation — if your cache pool restarts, you do not want user sessions to disappear at the same time
Regional strategy — you might want sessions in every region but object cache only in regions with heavy read traffic

Bahriya lets you create multiple Memcached instances within the same project, each with its own sizing, node count, and region selection.

Setting up the pools

Pool 1: Session cache

Sessions are small, accessed frequently, and critical for user experience. We want this pool in every region, with enough nodes for redundancy.

Using the Reis CLI:

reis memcached:create \
  --project my-app \
  --name "Session Cache" \
  --handle session-cache \
  --memory 128 \
  --nodes 2 \
  --regions falkenstein-1,helsinki-1,virginia-1,singapore-1

Or in YAML:

kind: memcached
project: my-app
handle: session-cache
name: Session Cache
regions:
  - falkenstein-1
  - helsinki-1
  - virginia-1
  - singapore-1
tuning:
  memory: 128
topology:
  nodes: 2

128 MB per node with 2 nodes gives you 256 MB of session storage per region, with redundancy if one node goes down.

Pool 2: Object cache

Application-level caching — database query results, rendered templates, serialised API responses. This pool handles larger objects and can tolerate cold-start latency, so we size it bigger but deploy it to fewer regions.

reis memcached:create \
  --project my-app \
  --name "Object Cache" \
  --handle object-cache \
  --memory 512 \
  --nodes 3 \
  --regions falkenstein-1,virginia-1

Or in YAML:

kind: memcached
project: my-app
handle: object-cache
name: Object Cache
regions:
  - falkenstein-1
  - virginia-1
tuning:
  memory: 512
topology:
  nodes: 3

512 MB per node with 3 nodes gives you 1.5 GB of cache per region. The extra node provides better key distribution across the consistent hashing ring.

Pool 3: Rate limiting counters

If your application implements rate limiting, you can use a dedicated pool with minimal memory. Rate limit counters are tiny (a few bytes each) but are written on every request, so isolation prevents them from interfering with your other caches.

kind: memcached
project: my-app
handle: rate-limits
name: Rate Limits
regions:
  - falkenstein-1
  - helsinki-1
  - virginia-1
  - singapore-1
tuning:
  memory: 64
topology:
  nodes: 1

Connecting from your container

Each Memcached instance is accessible from containers within the same project using its handle as the hostname. Your application connects to the pool using standard Memcached clients.

For the session cache pool:

Python

import pymemcache

session_client = pymemcache.Client('session-cache:11211')
session_client.set('session:abc123', user_data, expire=3600)

PHP (Kipchak)

use Kipchak\Drivers\Cache\CacheInterface;

// Kipchak injects the cache driver configured for 'session-cache'
$sessionCache = $container->get(CacheInterface::class);

$sessionCache->set('session:abc123', $userData, 3600);

Node.js

import Memcached from 'memcached';

const sessionClient = new Memcached('session-cache:11211');

sessionClient.set('session:abc123', JSON.stringify(userData), 3600, (err) => {
    if (err) console.error('Cache write failed:', err);
});

For the object cache pool:

Python

object_client = pymemcache.Client('object-cache:11211')
cached = object_client.get('products:featured')
if cached is None:
    cached = db.query("SELECT * FROM products WHERE featured = 1")
    object_client.set('products:featured', cached, expire=300)

PHP (Kipchak)

use Kipchak\Drivers\Cache\CacheInterface;

$objectCache = $container->get(CacheInterface::class);

$cached = $objectCache->get('products:featured');
if ($cached === null) {
    $cached = $db->fetchAll('SELECT * FROM products WHERE featured = 1');
    $objectCache->set('products:featured', json_encode($cached), 300);
}

$products = json_decode($cached, true);

Node.js

const objectClient = new Memcached('object-cache:11211');

function getFeaturedProducts() {
    return new Promise((resolve, reject) => {
        objectClient.get('products:featured', (err, data) => {
            if (err) return reject(err);

            if (data) return resolve(JSON.parse(data));

            db.query('SELECT * FROM products WHERE featured = 1', (err, results) => {
                if (err) return reject(err);
                objectClient.set('products:featured', JSON.stringify(results), 300, () => {});
                resolve(results);
            });
        });
    });
}

For multi-node pools, use a hash client that distributes keys across nodes:

Python

from pymemcache.client.hash import HashClient

object_client = HashClient([
    ('object-cache:11211', 11211),
])

PHP (Kipchak)

// Kipchak's Memcached driver handles multi-node consistent hashing
// automatically when configured with multiple nodes. No code change needed.

Node.js

// The memcached package supports multiple servers natively
const objectClient = new Memcached(['object-cache:11211']);

The client libraries handle consistent hashing across the nodes automatically.

Applying it all at once

Combine everything into a single YAML file and apply it in one command:

kind: memcached
project: my-app
handle: session-cache
name: Session Cache
regions:
  - falkenstein-1
  - helsinki-1
  - virginia-1
  - singapore-1
tuning:
  memory: 128
topology:
  nodes: 2

---

kind: memcached
project: my-app
handle: object-cache
name: Object Cache
regions:
  - falkenstein-1
  - virginia-1
tuning:
  memory: 512
topology:
  nodes: 3

---

kind: memcached
project: my-app
handle: rate-limits
name: Rate Limits
regions:
  - falkenstein-1
  - helsinki-1
  - virginia-1
  - singapore-1
tuning:
  memory: 64
topology:
  nodes: 1

reis apply -f memcached-pools.yml

Three pools, deployed across multiple regions, each sized for its workload — in one command.

Monitoring and scaling

You can scale any pool independently. If your object cache starts evicting too aggressively, increase the memory or add nodes:

reis memcached:update object-cache --project my-app --memory 1024

Or add a node:

reis memcached:update object-cache --project my-app --nodes 4

Each change triggers a redeployment of that pool only. Your other pools are unaffected.

Summary

Multiple Memcached pools let you match cache sizing, durability, and regional placement to each workload's requirements. On Bahriya, each pool is a first-class managed resource with its own handle, its own nodes, and its own regions — no shared infrastructure, no noisy neighbours, no manual configuration.