RLdis

Overview

RLdis started as a side experiment after getting mildly irritated with one specific thing about caches: deciding expiry times for everything. Either keys live too long and waste memory, or they disappear too early and cause unnecessary misses. Most of the time, TTLs end up being educated guesses anyway.

So instead of tuning them manually, I built a Redis-compatible in-memory server that tries to figure them out on its own.

The project is written in Go, partly because it fits the kind of lightweight systems programming Redis is known for, and partly because I wanted to stay close to the spirit of the original Redis project while experimenting with reinforcement learning ideas.

---

Problem

Caches are simple until they are not.

Every application ends up with different kinds of data:

• Some keys are accessed constantly
• Some are used once and forgotten
• Some become stale almost immediately
• Some should probably never expire at all

Picking TTLs manually for all of this is annoying. In practice, most expiry values are arbitrary numbers copied around from older projects or adjusted through trial and error.

I wanted to see whether a system could learn these patterns automatically instead of relying on hardcoded decisions.

---

Solution

RLdis uses a small reinforcement learning experiment based on Q-learning to predict sensible expiry times for keys automatically.

If I do not explicitly provide a TTL, the server watches how keys behave over time:

• Was the key accessed before expiry?
• Did it expire too early?
• Did it sit around doing nothing?

From there, it gradually adjusts future decisions.

The idea was not to build some production-grade AI cache layer. It was mostly an experiment to see whether reinforcement learning could make something as mundane as cache expiry slightly less tedious.

The result is a tiny Redis-compatible server that behaves mostly like normal Redis, except it quietly tries to learn from usage patterns in the background.

---

Developer Notes

This project was mainly built for fun and curiosity.

I wanted to experiment with:

• reinforcement learning in a systems project
• Q-learning outside of textbook examples
• building networked infrastructure software in Go
• recreating parts of Redis from scratch to understand the design better

The reinforcement learning part is intentionally lightweight. The interesting part for me was not making an advanced ML system, but integrating adaptive behaviour into something as ordinary as a key/value store.

It also gave me an excuse to build a small Redis-style server entirely from scratch, including command parsing, persistence, multiple data structures, and protocol handling, while keeping the project relatively compact.