Consensus Engine: XRP Ledger Consensus Protocol

← Back to Rippled II Overview

---

Introduction

The Consensus Engine is the heart of the XRP Ledger's ability to reach agreement on transaction sets and ledger state without requiring proof-of-work mining or proof-of-stake validation. Understanding the consensus mechanism is crucial for anyone working on the core protocol, as it's what makes the XRP Ledger both fast (3-5 second confirmation times) and secure (Byzantine fault tolerant).

Unlike blockchain systems that require computational work to achieve consensus, the XRP Ledger uses a consensus protocol where a network of independent validators exchanges proposals and votes to agree on which transactions to include in the next ledger. This approach enables the network to process transactions quickly while maintaining strong security guarantees.

---

Consensus Protocol Overview

What is Consensus?

Consensus is the process by which a distributed network of nodes agrees on a single shared state. In the context of the XRP Ledger, this means agreeing on:

1. Which transactions to include in the next ledger

2. The order of those transactions (deterministic ordering)

3. The resulting ledger state after applying all transactions

Without consensus, different nodes might have different views of the ledger, leading to double-spending and inconsistencies.

Why Not Proof-of-Work?

Traditional blockchains like Bitcoin use proof-of-work (PoW):

• Miners compete to solve cryptographic puzzles

• Winner proposes the next block

• Requires massive computational resources

• Block confirmation takes ~10 minutes (Bitcoin) or ~15 seconds (Ethereum)

XRP Ledger's approach is different:

• No mining or computational puzzles

• Validators vote on transaction sets

• Consensus achieved in rounds lasting 3-5 seconds

• Energy efficient (no wasted computation)

• Fast finality (transactions confirmed quickly)

Byzantine Fault Tolerance

The XRP Ledger consensus protocol is Byzantine Fault Tolerant (BFT), meaning it can tolerate some validators being:

• Offline or unreachable

• Malicious (trying to disrupt consensus)

• Byzantine (behaving arbitrarily or incorrectly)

Key Property: As long as >80% of trusted validators are honest and online, consensus will be reached correctly.

Security Model:

Network can tolerate up to 20% faulty validators
Examples:
- 100 validators → Can handle 20 failures
- 50 validators → Can handle 10 failures
- 35 validators (XRP Ledger mainnet) → Can handle 7 failures

---

Validators and UNL

Validators are nodes that participate in the XRP Ledger consensus process, validating transactions and agreeing on the state of the ledger. Each validator proposes and votes on ledger updates during consensus rounds.

A Unique Node List (UNL) is a trusted set of validators chosen by a participant. By relying on their UNL, a node can efficiently reach consensus while protecting against faulty or malicious validators. Proper UNL selection is crucial for network security, decentralization, and ledger reliability.

What is a Validator?

A validator is a rippled server configured to participate in consensus by:

• Proposing transaction sets

• Voting on other validators' proposals

• Signing validated ledgers

Not all rippled servers are validators. Most servers are:

• Tracking servers: Follow the network and process transactions but don't participate in consensus

• Stock servers: Serve API requests but don't store full history

To become a validator, a server needs:

1. A validator key pair (generated with validator-keys tool)

2. Configuration in rippled.cfg to enable validation

3. To be trusted by other validators (added to their UNLs)

Unique Node List (UNL)

Each validator maintains a Unique Node List (UNL)—a list of validators it trusts to be honest and not collude.

Key Concepts:

Personal Choice: Each validator operator chooses their own UNL based on their trust relationships.

Overlap Required: For the network to reach consensus, there must be sufficient overlap between validators' UNLs. The protocol requires >90% overlap to ensure agreement.

Default UNL: Most operators use the default UNL provided by the XRP Ledger Foundation, which is regularly updated and reviewed.

Dynamic Updates: UNLs can be updated over time as validators join or leave the network.

UNL Configuration

In validators.txt:

# Validator List (maintained by XRP Ledger Foundation)
# Format: validator_public_key [optional_comment]

nHUon2tpyJEHHYGmxqeGu37cvPYHzrMtUNQFVdCgGNvYkr4k
nHBidG3pZK11zQD6kpNDoAhDxH6WLGui6ZxSbUx7LSqLHsgzMPe
nHUcNC5ni7XjVYfCMe38Rm3KQaq27jw7wJpcUYdo4miWwpNePRTw
nHU95JxeaHJoSdpE7R49Mxp4611Yk5yL9SGEc12UDJLr4oEUN
# ... more validators

# Optional: Add custom validators
# nH... My Custom Validator

Validator List Management

The validator list can be automatically fetched from trusted sources:

[validators_file]
validators.txt

[validator_list_sites]
https://vl.ripple.com
https://vl.xrplf.org

[validator_list_keys]
ED2677ABFFD1B33AC6FBC3062B71F1E8397C1505E1C42C64D11AD1B28FF73F4734

This allows dynamic updates without manual configuration changes.

---

Consensus Rounds

Round Structure

Consensus operates in discrete rounds, each typically lasting 3-5 seconds. Each round attempts to agree on the next ledger.

Round Phases:

Open Phase (20-50s) → Establish Phase (2-4s) → Accepted Phase (instant) → Validated Phase (1-2s)
       ↓                     ↓                        ↓                        ↓
  Collect TXs          Exchange Proposals      Reach Agreement         Confirm Ledger

Phase 1: Open Phase

Duration: Variable (typically 20-50 seconds, but can be shorter)

Purpose: Collect transactions for the next ledger

What Happens:

• Transactions arrive from clients and peers

• Transactions are validated and added to the open ledger

• Each validator builds its own transaction set

• Open ledger is tentatively applied (provides immediate feedback)

Key Point: The open ledger is not final—it shows what might be in the next ledger, but consensus hasn't been reached yet.

// Transactions entering open ledger
void NetworkOPs::processTransaction(
    std::shared_ptr<Transaction> const& transaction)
{
    // Apply to open ledger
    auto const result = app_.openLedger().modify(
        [&](OpenView& view)
        {
            return transaction->apply(app_, view);
        });
    
    if (result.second)  // Transaction applied successfully
    {
        // Relay to network
        app_.overlay().relay(transaction);
    }
}

Phase 2: Establish Phase (Consensus Rounds)

Duration: 2-4 seconds (multiple sub-rounds with 50% increase each time)

Purpose: Validators exchange proposals and converge on a common transaction set

Process:

Initial Proposal

Each validator creates a proposal containing:

• Hash of their proposed transaction set

• Their validator signature

• Previous ledger hash

• Proposed close time

// Simplified proposal structure
struct ConsensusProposal
{
    uint256 previousLedger;     // Hash of previous ledger
    uint256 position;           // Hash of proposed transaction set
    NetClock::time_point closeTime;  // Proposed close time
    PublicKey publicKey;        // Validator's public key
    Signature signature;        // Proposal signature
};

Proposal Exchange

Validators broadcast proposals to the network using tmPROPOSE_LEDGER messages.

Agreement Threshold

Validators track which transactions appear in proposals from their UNL:

• 80% agreement: Transaction is considered "likely to be included"

• 50% agreement: Transaction is "disputed"

• <50% agreement: Transaction is "unlikely to be included"

Iterative Refinement

Multiple rounds of proposals:

Round 1 (Initial): Each validator proposes their transaction set

Round 2 (50% threshold): Validators update proposals, including only transactions with >50% support

Round 3+ (Increasing threshold): Threshold increases each round, converging toward agreement

Round 1: 50% threshold, 2 second timer
Round 2: 65% threshold, 3 second timer (50% increase)
Round 3: 80% threshold, 4.5 second timer
Round 4: 95% threshold, 6.75 second timer
...

Avalanche Effect

Once enough validators converge on the same set, others quickly follow (avalanche effect), achieving rapid consensus.

Phase 3: Accepted Phase

Duration: Instant (threshold is reached)

Purpose: Consensus is reached, transaction set is accepted

Trigger: When a validator sees >80% of its UNL agreeing on the same transaction set

What Happens:

• The agreed-upon transaction set becomes the "accepted" set

• Ledger is closed with this transaction set

• Validators compute the resulting ledger hash

• Consensus round ends

// Simplified consensus acceptance check
bool hasConsensus(ConsensusMode mode, int validations)
{
    if (mode == ConsensusMode::proposing)
    {
        // Need 80% of UNL to agree
        return validations >= (unlSize_ * 4 / 5);
    }
    return false;
}

Phase 4: Validated Phase

Duration: 1-2 seconds

Purpose: Validators sign and broadcast validations

What Happens:

• Each validator applies the agreed transaction set

• Computes the resulting ledger hash

• Creates a validation message

• Signs and broadcasts the validation

Validation Message (tmVALIDATION):

struct STValidation
{
    uint256 ledgerHash;          // Hash of validated ledger
    uint32 ledgerSequence;       // Ledger sequence number
    NetClock::time_point signTime;    // When validation was signed
    PublicKey publicKey;         // Validator's public key
    Signature signature;         // Validation signature
    bool full;                   // Full validation vs partial
};

Validation Collection:

• Nodes collect validations from validators

• When >80% of trusted validators validate the same ledger hash, it's considered fully validated

• The ledger becomes immutable and part of the permanent ledger history

Consensus Round Timeline

Time 0s: Open Phase begins
  ↓
  ... transactions accumulate ...
  ↓
Time 20-50s: Consensus triggered (sufficient transactions or timer)
  ↓
Time 0s (consensus start): Initial proposals broadcast
  ↓
Time 2s: Round 1 complete, update proposals
  ↓
Time 5s: Round 2 complete, update proposals
  ↓
Time 9.5s: Round 3 complete, 80% threshold reached
  ↓
Time 9.5s: Accepted phase - consensus reached
  ↓
Time 9.5-11.5s: Validators apply transactions and create validations
  ↓
Time 11.5s: Validation phase - ledger fully validated
  ↓
Time 11.5s: Next open phase begins

Total time from consensus start to validation: ~10 seconds Total time from transaction submission to confirmation: ~15-60 seconds (depending on when submitted during open phase)

---

Transaction Ordering and Determinism

Why Ordering Matters

For all validators to reach the same ledger state, they must apply transactions in exactly the same order. Different orders can produce different results:

Example:

Account A has 100 XRP

Transaction 1: Send 60 XRP to B
Transaction 2: Send 60 XRP to C

Order 1 (TX1 then TX2):
  - TX1 succeeds: A has 40 XRP, B has 60 XRP
  - TX2 fails: insufficient balance
  
Order 2 (TX2 then TX1):
  - TX2 succeeds: A has 40 XRP, C has 60 XRP
  - TX1 fails: insufficient balance

Different results!

Canonical Ordering

The XRP Ledger uses canonical ordering to ensure determinism:

Primary Sort: By account (lexicographic order of account IDs)

Secondary Sort: By transaction sequence number (nonce)

// Canonical transaction ordering
bool txOrderCompare(STTx const& tx1, STTx const& tx2)
{
    // First, sort by account
    if (tx1.getAccountID(sfAccount) < tx2.getAccountID(sfAccount))
        return true;
    if (tx1.getAccountID(sfAccount) > tx2.getAccountID(sfAccount))
        return false;
    
    // Same account, sort by sequence number
    return tx1.getSequence() < tx2.getSequence();
}

This ensures:

• All transactions from the same account are processed in sequence order

• Transactions from different accounts are processed in a deterministic order

• All validators apply transactions identically

Transaction Set Hash

The transaction set is represented by a hash:

// Calculate transaction set hash
uint256 calculateTxSetHash(std::vector<STTx> const& transactions)
{
    // Sort transactions canonically
    auto sortedTxs = transactions;
    std::sort(sortedTxs.begin(), sortedTxs.end(), txOrderCompare);
    
    // Hash all transactions together
    Serializer s;
    for (auto const& tx : sortedTxs)
    {
        s.addBitString(tx.getHash());
    }
    
    return s.getSHA512Half();
}

This hash is what validators include in their proposals—a compact representation of the entire transaction set.

---

Dispute Resolution

What is a Dispute?

A dispute occurs when validators initially disagree about which transactions should be included in the next ledger. This is normal and expected—validators may have different views due to:

• Network latency (different arrival times)

• Transaction validity questions

• Different open ledger states

Resolution Process

Disputes are resolved through the iterative consensus rounds:

Round 1: Initial Disagreement

Validator A proposes: {TX1, TX2, TX3, TX4, TX5}
Validator B proposes: {TX1, TX2, TX3, TX6, TX7}
Validator C proposes: {TX1, TX2, TX4, TX5, TX8}

Agreement:
- TX1: 100% (all three)
- TX2: 100% (all three)
- TX3: 67% (A, B)
- TX4: 67% (A, C)
- TX5: 67% (A, C)
- TX6: 33% (B only)
- TX7: 33% (B only)
- TX8: 33% (C only)

Round 2: Converge on High-Agreement TXs

Validators drop transactions with <50% support:

Validator A proposes: {TX1, TX2, TX3, TX4, TX5}  (drops nothing, all >50%)
Validator B proposes: {TX1, TX2, TX3}            (drops TX6, TX7)
Validator C proposes: {TX1, TX2, TX4, TX5}       (drops TX8)

Agreement:
- TX1: 100%
- TX2: 100%
- TX3: 67% (A, B)
- TX4: 67% (A, C)
- TX5: 67% (A, C)

Round 3: Further Convergence

As threshold increases to 80%, validators must drop disputed transactions:

All validators propose: {TX1, TX2}

Agreement:
- TX1: 100% ✓ (exceeds 80%)
- TX2: 100% ✓ (exceeds 80%)

Consensus reached on {TX1, TX2}
TX3, TX4, TX5 deferred to next ledger

Deferred Transactions

Transactions that don't reach consensus are not lost:

• They remain in the open ledger

• They'll be included in the next consensus round

• They're only dropped if they become invalid

Byzantine Validators

If a validator behaves maliciously:

• Their proposals are signed, so misbehavior is detectable

• Other validators ignore proposals that don't follow protocol rules

• Byzantine validators cannot force consensus on invalid states (requires >80% support)

• Operators can remove misbehaving validators from their UNL

---

Ledger Close Process

Close Triggers

A ledger close is triggered when:

Timer-based: Minimum time has elapsed (typically 2-10 seconds)

Transaction threshold: Sufficient transactions have accumulated

Consensus readiness: Validators are ready to reach agreement

// Simplified close trigger logic
bool shouldCloseLedger()
{
    auto const elapsed = std::chrono::steady_clock::now() - lastClose_;
    
    // Minimum close interval elapsed?
    if (elapsed < minCloseInterval_)
        return false;
    
    // Sufficient transactions?
    if (openLedger_.size() >= closeThreshold_)
        return true;
    
    // Maximum close interval elapsed?
    if (elapsed >= maxCloseInterval_)
        return true;
    
    return false;
}

Close Time Agreement

Validators must also agree on the close time of the ledger:

Why it matters: Some transactions are time-dependent (escrows, offers with expiration)

Process:

• Each validator proposes a close time

• Consensus includes the close time in proposals

• Final close time is the median of proposed times (Byzantine fault tolerant)

Close Time Resolution:

• Rounded to nearest 10 seconds for efficiency

• Prevents clock skew from causing issues

Post-Consensus Application

After consensus is reached:

Step 1: Apply agreed transaction set

// Apply transactions in canonical order
for (auto const& tx : canonicalOrder(agreedTxSet))
{
    auto const result = applyTransaction(tx, view);
    // Record metadata
}

Step 2: Compute ledger hash

// Hash includes:
// - Parent ledger hash
// - Transaction set hash
// - Account state hash
// - Close time
// - Ledger sequence
auto ledgerHash = computeLedgerHash(
    parentHash,
    txSetHash,
    stateHash,
    closeTime,
    ledgerSeq);

Step 3: Create and broadcast validation

STValidation validation;
validation.setLedgerHash(ledgerHash);
validation.setLedgerSequence(ledgerSeq);
validation.setSignTime(now);
validation.sign(validatorKey);

overlay().broadcast(validation);

Step 4: Collect validations

// Wait for validations from UNL
while (validationCount < unlSize_ * 4 / 5)
{
    // Process incoming validations
    auto val = receiveValidation();
    
    if (val.getLedgerHash() == ledgerHash)
        validationCount++;
}

// Ledger is now fully validated
ledgerMaster_.setFullyValidated(ledger);

---

Codebase Deep Dive

Key Files and Directories

Consensus Core:

• src/ripple/consensus/Consensus.h - Main consensus engine interface

• src/ripple/consensus/ConsensusProposal.h - Proposal structure

• src/ripple/consensus/Validations.h - Validation tracking

Consensus Implementation:

• src/ripple/app/consensus/RCLConsensus.h - XRP Ledger-specific consensus

• src/ripple/app/consensus/RCLValidations.cpp - Validation handling

Network Messages:

• src/ripple/overlay/impl/ProtocolMessage.h - tmPROPOSE_LEDGER, tmVALIDATION

Configuration:

• validators.txt - UNL configuration

• rippled.cfg - Validator key configuration

Key Classes

Consensus Class

template <class Adaptor>
class Consensus
{
public:
    // Start new consensus round
    void startRound(
        LedgerHash const& prevLedgerHash,
        Ledger const& prevLedger,
        NetClock::time_point closeTime);
    
    // Process peer proposal
    void peerProposal(
        NetClock::time_point now,
        ConsensusProposal const& proposal);
    
    // Simulate new round
    void timerEntry(NetClock::time_point now);
    
    // Check if consensus reached
    bool haveConsensus() const;
    
private:
    // Current round state
    ConsensusPhase phase_;
    std::map<NodeID, ConsensusProposal> peerProposals_;
    std::set<TxID> disputes_;
    TxSet ourPosition_;
};

RCLConsensus (Ripple Consensus Ledger)

XRP Ledger-specific consensus implementation:

class RCLConsensus
{
public:
    // Handle consensus result
    void onAccept(
        Result const& result,
        RCLCxLedger const& prevLedger,
        NetClock::duration closeResolution,
        CloseTimes const& rawCloseTimes,
        ConsensusMode mode);
    
    // Create initial position
    RCLTxSet getInitialPosition(
        RCLCxLedger const& prevLedger);
    
    // Check if we should close ledger
    void checkClose(NetClock::time_point now);
};

Code Navigation Tips

Finding Consensus Start

Search for ledger close triggers:

// In NetworkOPs or LedgerMaster
void beginConsensus(LedgerHash const& prevHash)
{
    // Build initial transaction set
    auto initialSet = buildTxSet();
    
    // Start consensus round
    consensus_.startRound(
        prevHash,
        prevLedger,
        suggestCloseTime());
}

Tracing Proposal Handling

Follow proposal processing:

// Overlay receives tmPROPOSE_LEDGER message
void onProposal(std::shared_ptr<protocol::TMProposeSet> const& proposal)
{
    // Validate proposal signature
    if (!verifyProposal(proposal))
        return;
    
    // Pass to consensus engine
    consensus_.peerProposal(
        now(),
        parseProposal(proposal));
}

Understanding Validation

Follow validation creation and verification:

// Create validation
auto validation = std::make_shared<STValidation>(
    ledgerHash,
    signTime,
    publicKey,
    nodeID,
    [&](STValidation& v)
    {
        v.sign(secretKey);
    });

// Broadcast to network
overlay().send(validation);

---

Hands-On Exercise

Exercise: Observe and Analyze a Consensus Round

Objective: Watch a complete consensus round and understand the proposal exchange process.

Part 1: Setup Multi-Validator Environment

This is advanced—requires multiple validators. For learning, we'll use logs from a single validator.

Step 1: Enable detailed consensus logging

Edit rippled.cfg:

[rpc_startup]
{ "command": "log_level", "partition": "Consensus", "severity": "trace" }
{ "command": "log_level", "partition": "LedgerConsensus", "severity": "trace" }

Step 2: Start rippled

rippled --conf=rippled.cfg

Step 3: Watch the logs

tail -f /var/log/rippled/debug.log | grep -E "Consensus|Proposal|Validation"

Part 2: Identify Consensus Phases

From the logs, identify:

Open Phase Start:

"Consensus":"open ledger started, seq=12345"

Consensus Round Start:

"Consensus":"Starting consensus round, prevLedger=ABCD..."

Proposals Received:

"Consensus":"Received proposal from nHU...., position=XYZ..."

Agreement Tracking:

"Consensus":"Transaction TX1 has 85% agreement"
"Consensus":"Transaction TX2 has 45% agreement (disputed)"

Consensus Reached:

"Consensus":"Consensus reached on transaction set, hash=..."

Validation Created:

"Consensus":"Created validation for ledger 12345, hash=..."

Validation Received:

"Validations":"Received validation from nHU...., ledger=12345"

Ledger Fully Validated:

"LedgerConsensus":"Ledger 12345 fully validated with 28/35 validations"

Part 3: Timing Analysis

Measure the duration of each phase:

1. Open Phase Duration: Time between "open ledger started" and "Starting consensus round"

2. Consensus Duration: Time from "Starting consensus round" to "Consensus reached"

3. Validation Duration: Time from "Consensus reached" to "fully validated"

Create a timeline:

T+0s: Open phase begins
T+25s: Consensus triggered
T+27s: Consensus reached (2s consensus time)
T+29s: Ledger fully validated (2s validation time)
T+29s: Next open phase begins

Total cycle: 29 seconds

Part 4: Analysis Questions

Answer these based on your observations:

1. How many consensus rounds occurred?

   • Count from logs

2. What was the average consensus time?

   • Measure multiple rounds

3. How many transactions were included in each ledger?

   • Look for transaction count in logs

4. Were there any disputed transactions?

   • Look for agreement percentages <100%

5. How many validations did each ledger receive?

   • Count validation messages

6. What percentage of UNL validated each ledger?

   • Compare validations received vs UNL size

Part 5: Compare to Whitepaper

Read the XRP Ledger Consensus Protocol whitepaper and compare:

• Does the observed behavior match the description?

• Are the timing estimates accurate?

• How does the network handle disputes?

---

Key Takeaways

Core Concepts

✅ Byzantine Fault Tolerance: Network can tolerate up to 20% faulty validators while maintaining security

✅ UNL-Based Trust: Each validator chooses which other validators to trust, creating a trust graph

✅ Iterative Consensus: Multiple rounds of proposals converge on an agreed transaction set

✅ Fast Finality: 3-5 second consensus rounds enable quick transaction confirmation

✅ No Mining: Consensus achieved through voting, not computational work

✅ Deterministic Ordering: Canonical transaction ordering ensures all validators reach identical state

✅ Dispute Resolution: Disagreements resolved by dropping disputed transactions to next round

Security Properties

✅ Safety: No conflicting ledgers validated (no forks in normal operation)

✅ Liveness: Network makes progress as long as >80% of UNL is honest and responsive

✅ Censorship Resistance: No single entity can block valid transactions

✅ Sybil Resistance: Trust relationships (UNL) prevent fake validator attacks

Development Skills

✅ Codebase Location: Consensus implementation in src/ripple/consensus/ and src/ripple/app/consensus/

✅ Proposal Format: Understand ConsensusProposal structure and tmPROPOSE_LEDGER messages

✅ Validation Format: Understand STValidation structure and tmVALIDATION messages

✅ Debugging: Use consensus logs to trace round progression and identify issues

---

Common Misconceptions

Misconception 1: "Validators mine like Bitcoin"

False: Validators don't perform computational work. They simply vote on which transactions to include.

Misconception 2: "Ripple controls consensus"

False: Any organization can run validators, and each validator operator independently chooses their UNL. While many operators use the recommended UNL from the XRP Ledger Foundation, they're free to customize it.

Misconception 3: "All servers participate in consensus"

False: Most rippled servers are tracking servers that follow consensus but don't vote. Only configured validators participate.

Misconception 4: "Consensus can be blocked by one entity"

False: As long as >80% of a validator's UNL is operational and honest, consensus proceeds normally.

Misconception 5: "XRP Ledger has forked"

False: The XRP Ledger has never had a fork (competing chains). The consensus protocol prevents this by design.

---

Additional Resources

Official Documentation

• XRP Ledger Dev Portal: xrpl.org/docs

• Consensus Protocol: xrpl.org/consensus

• Run a Validator: xrpl.org/run-a-rippled-validator

Academic Papers

• Original Consensus Whitepaper: David Schwartz, Noah Youngs, Arthur Britto

• Analysis of the XRP Ledger Consensus Protocol: Brad Chase, Ethan MacBrough

• Cobalt: BFT Governance in Open Networks: Ethan MacBrough

Codebase References

• src/ripple/consensus/ - Generic consensus framework

• src/ripple/app/consensus/ - XRP Ledger-specific implementation

• src/ripple/app/ledger/ConsensusTransSetSF.cpp - Transaction set management

Related Topics

• Protocols - How consensus messages are propagated

• Application Layer - How consensus integrates with other components

• Transaction Lifecycle - How transactions flow through consensus

---