UUIDs in distributed systems: IDs without asking
Why UUIDs in distributed systems have less to do with uniqueness than with independence – on idempotency, data sets growing together, and the price the index pays for it.
The wrong question is usually asked about UUIDs. It tends to be “so which version do we take” – when the more interesting question is why you need one at all.
Because a database has long been able to issue unique keys by itself. A sequence, an IDENTITY,
an AUTO_INCREMENT: consecutive, compact, sorted, human-readable. For a single application with
a single database there is little reason to deviate from that.
The reason arrives as soon as the system is no longer made of one piece.
The versions, briefly
First the map. Since RFC 9562 of May 2024 – the successor to the old RFC 4122 – there are essentially these families:
- v1 and v6 – timestamp plus node identifier, classically from the MAC address. v6 is v1 resorted, with the time at the front.
- v3 and v5 – computed from a namespace and a name, using MD5 and SHA-1 respectively. Same input, same UUID – reproducible and without randomness.
- v4 – 122 bits of randomness. The classic, and what most libraries hand you when you do not specify anything else.
- v7 – a 48-bit Unix timestamp in milliseconds, at the front, followed by randomness. The current recommendation for new keys.
- v8 – deliberately left free for your own formats.
If you want to see the bit layout in detail, throw a UUID into uuid.rellit.de – it takes one apart and generates them, timestamp and variant included. What this post is about is what you use the whole thing for.
The actual point: no asking
A sequence has one unpleasant property: it needs a central authority to manage it. Whoever wants an ID has to ask.
As long as everything hangs off one database, that goes unnoticed. It gets noticed as soon as:
- several services create records that reference one another,
- data arises in several regions or data centres,
- a client is meant to keep working even when there is no connection,
- or two separately grown systems get merged.
In all of these cases the central issuing authority is precisely the point where things jam. A UUID solves that by having every participant simply produce an ID itself – without coordination, without network access, with the probability of a collision as sufficient assurance.
That sounds mundane, but it has consequences that reach well beyond key allocation.
The ID exists before the record
If I produce the ID myself, I no longer have to wait for the database to tell me what the new record is called.
I can build a whole object graph in memory – order, line items, references between them – and then write it in one go. I can create data in a user interface that has not been saved at all yet, and still reference it. The sequence “save first, then learn the key, then fix up the references” disappears entirely.
Anyone who has ever written an import in which a thousand records had to be inserted one at a time just to read back each generated key for the dependent rows knows what that is worth.
Idempotency, the underrated use case
For me this is the strongest argument of all.
Distributed systems rarely deliver exactly once. The usual guarantees read “at least once” – and that means: a call runs into a timeout, the caller does not know whether the other side processed it, and tries again. With a message queue the same applies: if the acknowledgement is not set in time, the message arrives a second time.
If the recipient issues the ID, a second record is created in the process. The order is there twice, the payment has been booked twice.
If, on the other hand, the sender issues the ID, and does so once on the first attempt, then the retry carries the same ID as the original. A silent duplicate turns into a key conflict – and that is recognisable and manageable as such: detected, discarded, acknowledged, done. This is exactly what sits behind what many APIs offer as an idempotency key.
The retry is therefore no longer dangerous. And that is the property that dependable distributed processing is built on in the first place.
Merging without a numbering-range battle
The third case happens sooner or later almost every time: two data sets are to become one.
Two tenants get merged. An acquired system is taken over. A branch office that has been working offline synchronises its data. With consecutive numbers this is where the exercise begins that everyone who has had to do it once knows: separate the numbering ranges, add offsets, rewrite foreign keys – and hope at the end that nothing was overlooked.
With UUIDs that step disappears. The keys were unique across system boundaries from the start; the data can simply be poured together.
One side effect is often sold as a security benefit: consecutive numbers give away business
figures. Whoever receives invoice 1042 and, a month later, invoice 1102 can do the arithmetic.
And whoever sees an id=1042 in a URL will, in my experience, also have a go at 1043.
That is a real advantage – but please do not mistake it for a protective measure. RFC 9562 is explicit on this point: UUIDs must not be treated as access credentials whose mere possession grants access. An ID that is hard to guess does not replace an authorisation check, it merely delays the attempt.
The price: the index
So much for the good news. Now the bill.
A v4 key is pure randomness, and that is exactly why it is a poor neighbour for an index. Consecutive keys always append new rows on the right: the same page is written every time, it sits in the cache, everything is fast. Random keys, by contrast, scatter every write across the whole tree. Pages have to be split in the middle of the data, the fill factor drops, and instead of one hot page the database suddenly needs a large part of the index in memory. With a clustered index – the one the table is physically sorted by – this hits with full force.
That is precisely what v7 was designed for. The timestamp sits in the leading bits, new values are therefore larger than all previous ones, and appending on the right works again. On top of that you get the creation time for free.
The storage form matters almost more than the version, though: a UUID belongs in a binary column as 16 bytes, not in a column as 36 characters of text. That is the piece of advice with the largest effect, and it works in every index that carries the key along.
What remains
UUIDs are not a replacement for consecutive numbers, they are an answer to a different question. They cost space and, depending on version and database, write performance. In return you get independence from a central issuing authority – and with it repeatability, mergeability, and the option of letting data arise where it occurs.
For new keys, v7 is the obvious choice. Where the creation time has no business being visible, v4 remains right. And when the same input should reliably yield the same ID, v5 and v8 are the tools too rarely thought of.
To look at, generate and take apart: uuid.rellit.de.