A third way to coordinate shared state: optimistic, language-level, and modeled directly on database transactions.
Most Java developers reach for one of two tools when state needs protecting from concurrent access: a lock, synchronized or ReentrantLock, that blocks competing threads until it’s free, or a CAS-based primitive from java.util.concurrent, an AtomicInteger or similar, that retries a single-word update until it succeeds. Both work well for what they were built for. Neither one, on its own, solves the harder problem of updating two or more related pieces of state together, atomically, without a hand-managed lock order. Clojure answered that problem differently, by borrowing the transaction model from databases and building it directly into the language.
Where the Idea Actually Came From
Software transactional memory isn’t a Clojure invention. Nir Shavit and Dan Touitou described the core idea in a 1995 PODC paper, and Maurice Herlihy, Victor Luchangco, Mark Moir, and William Scherer extended it to dynamic-sized data structures in 2003, also at PODC. Haskell got there in production form first, with Tim Harris, Simon Marlow, Simon Peyton Jones, and Herlihy publishing “Composable Memory Transactions” at PPoPP in 2005, introducing the atomically, retry, and orElse vocabulary that later STM systems, Clojure’s included, still use in spirit. Rich Hickey released Clojure in 2007 after roughly two and a half years developing it largely alone and self-funded, and by his own description in a 2009 interview, its STM is built specifically on multi-version concurrency control, the same technique most SQL databases use to let readers and writers avoid blocking each other.
What Clojure’s Refs Actually Do
A Clojure ref holds a value that can only change inside a dosync transaction. Reading a ref never blocks. Writing one speculatively computes a new value, and Clojure validates at commit time that nothing the transaction read has changed since it started; if something has, the transaction retries automatically, transparently, with no code in the transaction body aware that it happened. The classic illustration, and one Hickey himself uses, is a bank transfer:
(def account-a (ref 1000))
(def account-b (ref 500))
(defn transfer [from to amount]
(dosync
(alter from - amount)
(alter to + amount)))
(transfer account-a account-b 200)
Nothing here specifies a lock order between account-a and account-b. There’s no risk of the classic deadlock where one thread locks A then waits for B while another locks B then waits for A, because no lock is being held while the transaction decides what to do. Clojure’s actual implementation is more sophisticated than a single global lock, it uses fine-grained per-ref locking at commit time along with a mechanism called barging, where an older transaction can detect a conflict with a younger one and preempt it, specifically to guarantee progress rather than starvation.
A Self-Run Comparison: Lock Ordering vs. Optimistic Retry
To see the trade-off directly, rather than take it on faith, I wrote and ran a small Java benchmark modeling the same bank-transfer scenario two ways: one using per-account ReentrantLocks acquired in a fixed order to avoid deadlock, the other using an optimistic approach, lock-free reads and speculative computation, with a short, shared critical section only at the final commit step that validates nothing changed and retries the whole transaction if it did. Twenty accounts, up to 32 threads, 200,000 transfer attempts per thread, run on OpenJDK 21.
The honest result: in this simplified version, the lock-based approach ran as fast as or slightly faster than the optimistic one at every thread count tested, because my optimistic implementation validates and commits under one shared lock rather than Clojure’s actual per-ref, fine-grained scheme, so it isn’t really more parallel, just differently structured. What it does show clearly is the retry count climbing with contention, seven retries at two threads, 335 at thirty-two, the signature behavior of every optimistic concurrency scheme, STM included. A comparison in the other direction backs this up: a from-scratch, teaching-oriented reimplementation of Clojure’s actual STM algorithm, built by researcher Tom Van Cutsem to illustrate the underlying MVCC mechanics, measured at three to six times slower than Clojure’s real, production-optimized version. The engineering Clojure put into fine-grained locking and barging isn’t a minor detail, it’s most of what separates a working STM from a merely correct one.
What Java’s Own Tools Don’t Cover
None of this makes synchronized or java.util.concurrent obsolete, because they’re solving adjacent problems. synchronized has been part of Java since version 1.0 in 1996. The CAS-based tools, AtomicInteger, ReentrantLock, the concurrent collections, arrived far later, in J2SE 5.0 in 2004 through JSR-166, led by Doug Lea with Josh Bloch, Brian Goetz, David Holmes, and others. What CAS gives you is atomicity for a single variable, cheaply and without blocking. What it doesn’t give you is a way to update two independent CAS-protected variables as one atomic unit; the moment a transfer needs to touch two accounts together, you’re back to locks, and back to reasoning about lock order by hand.
synchronized / locks | CAS (java.util.concurrent) | Clojure STM | |
|---|---|---|---|
| Coordinates | Any code inside the block | A single variable | Any number of refs together |
| Readers block | Yes, if a writer holds the lock | No | No |
| Deadlock risk | Yes, if lock order isn’t managed | No, single variable only | No, no locks held during the transaction body |
| Added to the platform | 1996 (Java 1.0) | 2004 (JSR-166) | 2007 (Clojure’s first release) |
When STM’s model actually fits
- The operation touches more than one piece of shared state that has to change together, not just one counter
- You want to avoid hand-managing lock acquisition order to prevent deadlock
- Reads are frequent relative to writes, since STM readers never block, unlike lock-based readers competing with a writer
- You can tolerate a transaction body running more than once on retry, which means it needs to be free of side effects that aren’t themselves transactional
What We Learned
Clojure’s STM isn’t a faster version of what java.util.concurrent already does, it answers a question CAS-based tools were never built to answer: how do you update several related pieces of shared state together, atomically, without a human deciding a lock order in advance. Locks and CAS both solve real problems well within their scope, one variable at a time for CAS, arbitrary code for locks at the cost of deadlock risk if that code touches more than one resource.
STM, built on multi-version concurrency control and academic work going back to 1995, trades a guaranteed win on raw throughput, my own benchmark shows a simplified version of it can lose to careful lock ordering, for composability: transactions that combine safely without anyone having to reason about acquisition order, at the cost of a transaction body that might run more than once before it commits.
