The Nox post covered the network layer of off-chain privacy: hiding who you are when you talk to a blockchain. This one is about the other half of that gap, the part that survives even when your IP is perfectly hidden.
Routing every request through a network like Tor or a mixnet hides where the connection came from. It does not hide the contents of the request. The RPC provider still sees the query itself: which records were asked for, which lookups were run. The IP is gone, but the question is still in plain view, and the question is often enough to figure out who asked it.
What a wallet actually asks for
To see why hiding the IP isn't enough, look at the reads a shielded wallet has to do. They aren't bulk scans of public state. They're targeted lookups, each one keyed by something private:
- Checking a balance. For every shielded note the wallet owns, it asks the server about that specific commitment. Is this note still unspent? Which block did it appear in? Which compliance list is it on right now?
- Building a spend. The wallet fetches the Merkle siblings for the exact leaf it intends to spend. One request, one leaf index.
- Compliance check. Before transacting, the wallet asks whether a given commitment is in good standing on each compliance list it cares about. The request carries the commitment hash itself.
This is the leak every shielded protocol inherits. Railgun, Privacy Pools, Tornado, Semaphore, Zcash. The cryptography hides what's in the proof. It doesn't hide the reads that built it.
This isn't a flaw in the cryptography. It's a gap in the infrastructure that undermines the privacy model the cryptography was built for.
Raven closes that gap on the query layer.
Raven is a PIR framework for off-chain reads, built so a wallet can fetch state from a chain without the server ever learning what state was fetched. The wallet sends an encrypted query. The Raven node answers without decrypting it. The wallet recovers the record locally. Same proofs, same wallet, same chain. The read that builds the proof is now blind.
The framework is built around the sidecar design proposed for Ethereum state PIR: a heavyweight engine over a stable snapshot of the database, paired with a small auxiliary engine that absorbs what the chain has changed in the last few blocks. iSimplePIR drives the main engine. InsPIRe is the scheme matched to the sidecar slot. The first deployment is built for Railgun, walked through in a follow-up post.
Open source. Built for the Railgun wallet on Ethereum mainnet to fetch private reads. Try the demo.
How Raven works comes down to two parts. The cryptographic primitive that lets the server answer a query it can't read, and the engineering that keeps the database in sync with the chain so those queries return fresh data. Start with the primitive.
Private Information Retrieval
The goal of PIR is plain: fetch a record from a database without the server learning which record you asked for.
The naive version is to download the entire database and look up locally. Perfect privacy, terrible scaling. A million records means a million records on the wire every time.
Real PIR schemes do better. The client sends an encrypted query. The server runs computation over the database without ever decrypting which row is the target. The client decrypts the response and gets back exactly the record it wanted. The server's view of "what was asked" is mathematically blind.
Single-server only
Multi-server PIR is faster, but it requires two or more servers that don't talk to each other. In practice, that assumption doesn't hold. Two providers in the same data center, the same cloud, or under the same legal jurisdiction can quietly collude. Single-server PIR removes the assumption entirely. That's where Raven lives.
The landscape
Several single-server PIR schemes have come out in the last few years. InsPIRe, HintlessPIR, YPIR, Plinko, SimplePIR, ChalametPIR. Each makes a different bet about where to spend cost. Some assume the database is static and trade fast queries for slow rebuilds. Some give up a stateless client for sublinear queries. Some keep bandwidth small and pay in server compute. The Ethereum Foundation maintains a public benchmark page comparing them on real workloads, including blockchain reads.
Studying them surfaces the same conclusion. No single scheme is general-purpose. Each one wins on a few properties and gives up on others. There is a structural reason for that.
The PIR trilemma
Single-server PIR sits inside a known tradeoff between three properties:
- Stateless client. No large hints to download, no local state to keep in sync.
- Sublinear query. The server doesn't have to touch every record every time. Communication shrinks well below the database size.
- Dynamic updates. Adding or modifying records doesn't force a full rebuild.
No known scheme gives all three. Pick two and you can have them. Pick the third and one of the others has to bend.
This is the open problem in PIR research. Stateless and sublinear and dynamic, all at once, is still out of reach.
What we needed
For a blockchain workload, the requirements are unforgiving.
Blockchain state is dynamic. New commitments arrive every block. The PIR layer has to answer queries against state that's at most a few seconds old.
Clients are constrained. Wallets run in browsers and on phones, not workstations. They can't carry gigabytes of preprocessing hints or keep a hint cache in sync while the database changes underneath. Statelessness is the only shape that fits.
Setup has to be cheap. The rebuild interval bounds query freshness. A two-hour setup means a two-hour stale window, no matter how fast each query is.
Latency has to feel like an RPC call. A typical Infura request finishes in 50 to 200 milliseconds. PIR has to sit in the same neighborhood or the wallet feels broken.
Most PIR schemes optimize for one or two of these. None optimize for all four. The remaining properties don't all have to come from the cryptography. Some can be supplied at the infrastructure layer instead. That is the bet Raven makes.
Raven
Raven is a general-purpose single-server PIR framework. It's scheme-agnostic by design: today it ships with iSimplePIR and InsPIRe behind one API, and any scheme that fits the trait can plug in. The same engine drives query privacy for two very different shapes of data:
- Key-value stores. Any append-mostly or static keyed dataset where clients want to fetch one value without revealing the key.
- Merkle trees. Blockchain commitment trees where clients fetch sibling paths or membership status without revealing which leaf is theirs.
The framework handles the shared infrastructure once: indexing live chain state, sharding the encoded database, swapping fresh state in atomically, sticky sessions across rebuilds, and a wasm-friendly client. You bring the data model. Raven turns it into something the PIR server can serve queries against.
The sidecar approach
The hardest property to recover from the trilemma is dynamic updates. The PIR schemes that give us sublinear queries and stateless clients assume the database is fixed. New commitments break that assumption every twelve seconds.
The architecture Raven adopts comes from the Sharded PIR design for Ethereum state proposed on ethresear.ch. Credit for the pattern belongs there. Instead of rebuilding the whole database on every update, two engines run side by side, each tuned for a different update profile.
The main engine serves queries against a stable snapshot of the database. iSimplePIR drives this half: sublinear queries against preprocessed state, paid for by a heavy server-side build. As long as the snapshot doesn't move, the preprocessing carries forward and queries stay fast.
The sidecar engine handles only the entries that have changed since the snapshot. InsPIRe is the scheme matched to this slot: minimal server-side preprocessing, and a database small enough (a few thousand recent commitments instead of millions) that per-update cost is negligible. Rebuilding it for the latest block takes milliseconds.
The client queries both engines for the same record. If the record was changed since the snapshot, the sidecar returns the current value. Otherwise the main engine returns the snapshot value. Either way the server learns nothing about which record was asked for.
Every few thousand blocks, the sidecar's accumulated entries get folded into the main engine, the snapshot advances, and the sidecar resets to empty. Folding is where the blue-green pattern earns its keep.
Blue-green rebuilds
Folding the sidecar into the main engine means re-preprocessing the main engine at a new snapshot. That's the expensive step. Doing it inline would take the main engine offline for as long as the rebuild takes, which is unacceptable for a service that wants to keep answering queries.
Raven runs the rebuild on a parallel copy of the engine instead. The live copy keeps serving queries against the old snapshot. The other copy preprocesses the new snapshot in the background. When the rebuild is ready, the orchestrator atomically swaps in the new state. In-flight queries on the old state finish on their own copy; new queries route to the new state immediately. No restart, no downtime, no torn reads.
The rebuild stays cheap for two reasons. Only the shards that actually changed get re-encoded. Everything else carries forward from the previous build untouched. And most of the server's preprocessing survives the rebuild, because it depends on the shape of the database rather than its contents, and the shape rarely changes.
For the workloads Raven serves (shielded balance checks, membership lookups, path retrievals), queries hit data within a few seconds of chain head. Every response also carries a freshness signal, so a client that detects unusually stale data can fall back to a non-PIR provider for that one call.
Turning a data model into a PIR table
A PIR server doesn't see Merkle trees, lists, or key-value maps. It sees one thing: a flat array of fixed-width rows, sharded into chunks, indexed by row number. Anything you want to query privately has to be converted into that shape first.
PIR is also index-based. The protocol only knows how to fetch row N. Applications never query that way. They ask for the value at a key, the membership status of a commitment, the sibling path for a leaf. The bridge between "I have a key" and "give me row N" is where the data model lives, and Raven exposes it as an encoder.
An encoder is a pure function: given a logical store (a tree, a list, a map) and a shard id, it materializes the bytes for that shard. It also fixes how a logical query maps to a row index, so the client computes the index locally and issues a private read for it. Different encoders express different query shapes over the same underlying data.
For a key-value store, the encoder uses a binary fuse filter or cuckoo hashing to assign every key a deterministic row. The mapping is public and shared with the client. The client hashes its key, computes the row, and issues one PIR query for that index. The server only ever sees a private query for an opaque row number.
PIR for Merkle trees
A Merkle proof of membership needs one sibling hash per level of the tree: the sibling of the leaf, then the sibling of the parent, then the sibling of the grandparent, all the way up to the root. The wallet has to fetch all of them without telling the server which leaf they're for. That is harder than a plain private record lookup, because the path, not just the leaf, has to stay hidden.
The naive version is to ask the server for "the proof attached to this leaf." Perfect for the wallet, awful for privacy: the shape of the question leaks the answer.
The real version flattens the tree into a plain PIR database and lets the wallet assemble the proof on its own side.
The Railgun tree. Railgun keeps shielded commitments in an append-only Merkle tree of depth 16. So every membership proof is exactly 16 sibling hashes long, one per level. New commitments are written left to right as leaves and never moved; the hashes along their paths to the root are computed at insert time. The tree holds up to 216 = 65,536 commitments, and a new tree opens when it fills. Every position (level, position) is well-defined and stable.
Layout. Raven uses what it calls the per-node encoder. Each tree node becomes one 32-byte row in a single flat array, with the rows arranged level by level. Leaves first at rows [0, 216). Level-1 hashes follow at [216, 216 + 215). Each subsequent level follows the previous, halving in size each time, up to a single root at the last row. Total: 217 - 1 = 131,071 rows. The mapping from (level, position) to row index is a closed-form formula known to both server and wallet.
The tree's structure lives entirely on the wallet now. The server stores a 2D matrix of records and nothing else.
The walk. To build a proof for leaf L, the wallet walks up the tree locally. At each level it XORs the current position with 1 to pick the sibling, computes the row index for (level, sibling_position), and shifts right to climb. After 16 iterations, one per level of the depth-16 tree, it has 16 row indices.
It issues 16 parallel PIR queries against the same flat array, one query per row index. Each query is encrypted; the server runs PIR computation against the full 131k-row database and returns an encrypted response. The wallet decrypts the 16 responses, assembles them into the sibling path, and feeds the path into the spend proof.
What the server learns. 16 opaque index reads against a 131k-row table. Nothing on the wire identifies the reads as a path, correlates them to a leaf, or recovers the tree's structure. The structure lives entirely on the client.
This is the read pattern every time a Railgun wallet builds a spend, walked through in Raven for Railgun.
The per-node layout described above is one of three useful encoders for a commitment tree. The three make different bets on row size versus query count.
| Encoder | Row at index i |
Row size | Used for |
|---|---|---|---|
| per-leaf | Raw 32-byte commitment at leaf i |
32 B | "Is my commitment in the tree?" |
| per-path | 16 sibling hashes along the path from leaf i to root |
512 B | "Give me the proof I need for ZK." |
| per-node | One 32-byte internal node at flat index i |
32 B | Custom traversals, partial proofs. |
Same tree. Three layouts. The per-node encoder is what the Railgun deployment uses: 16 queries, a thin row, a small simple database. The per-path encoder gives the whole sibling vector in one round-trip but pays for it with a much wider row. The framework lets the operator pick the right tradeoff for the workload, and the encoder is the only thing that has to change.
Every encoder also declares a fixed record_size and entries_per_shard, which gives the framework a deterministic mapping from logical entry to physical shard and offset. That mapping is what makes incremental updates cheap. When an entry changes, the encoder reports the bounded set of shards it touches and the sidecar re-encodes only those. The same abstraction carries beyond trees and maps: any data model that can be flattened to fixed-width rows fits, with the same orchestrator, the same sidecar pattern, and the same client surface. The encoder is the only thing that changes.
What's next
The first deployment is built for the Railgun wallet stack. The two specific query-side leaks in a Railgun client, and how Raven plugs in with a one-line swap, are walked through in Raven for Railgun.
The same engine, with a different encoder, is what we'll point at the next workload: other shielded pools, identity sets, anything that's a tree or a keyed map sitting behind a public endpoint.
Combine Raven with Nox and you have both halves of the off-chain privacy stack: who you are, and what you're asking for, hidden together. The cryptography on-chain finally has infrastructure that matches it.