Appendix : Codebase Navigation Guide

← Back to SHAMap and NodeStore: Data Persistence and State Management

---

Introduction

This appendix helps you navigate the rippled codebase to find SHAMap and NodeStore implementations.

Directory Structure

SHAMap Source Files

rippled/src/xrpld/shamap/
├── SHAMap.h                              # Main SHAMap class
├── SHAMapInnerNode.h                     # Inner node implementation
├── SHAMapLeafNode.h                      # Leaf node implementations
├── SHAMapNodeID.h                        # Node identification
├── SHAMapHash.h                          # Hash computation
├── SHAMapMissingNode.h                   # Missing node tracking
├── detail/
│   ├── SHAMap.cpp                        # Core algorithms
│   ├── SHAMapSync.cpp                    # Synchronization logic
│   └── SHAMapDelta.cpp                   # Tree traversal

NodeStore Source Files

rippled/src/xrpld/nodestore/
├── Database.h                            # Database interface
├── Backend.h                             # Backend interface
├── Manager.h                             # NodeStore manager
├── Types.h                               # Type definitions
├── Task.h                                # Async task definitions
├── Scheduler.h                           # Scheduling
├── backend/
│   ├── RocksDBFactory.cpp                # RocksDB backend
│   ├── NuDBFactory.cpp                   # NuDB backend
│   ├── MemoryFactory.cpp                 # In-memory backend
│   └── NullFactory.cpp                   # No-op backend
├── detail/
│   ├── Database.cpp                      # Database implementation
│   ├── ManagerImp.cpp                    # Manager implementation
│   ├── BatchWriter.h/cpp                 # Batch write handling
│   ├── DatabaseNodeImp.h                 # Single-backend implementation
│   ├── DatabaseRotatingImp.h             # Rotating-database implementation
│   ├── NodeObject.cpp                    # NodeObject implementation
│   ├── EncodedBlob.h                     # Encoding/decoding
│   ├── DecodedBlob.h                     # Blob structure
│   └── varint.h                          # Variable-length integers

Integration Points

rippled/src/xrpld/app/
├── misc/SHAMapStore.h                    # SHAMap-NodeStore integration
├── misc/SHAMapStoreImp.h/cpp             # Implementation
└── main/NodeStoreScheduler.h/cpp         # Background scheduling

Key Classes and Their Locations

SHAMap Classes

Class	File	Purpose
SHAMap	shamap/SHAMap.h	Main tree class
SHAMapTreeNode	shamap/SHAMapTreeNode.h	Base node class
SHAMapInnerNode	shamap/SHAMapInnerNode.h	Inner nodes (branches)
SHAMapLeafNode	shamap/SHAMapLeafNode.h	Leaf nodes (data)
SHAMapNodeID	shamap/SHAMapNodeID.h	Node identification
SHAMapItem	shamap/SHAMapItem.h	Data in leaf nodes

NodeStore Classes

Class	File	Purpose
Database	nodestore/Database.h	High-level interface
Backend	nodestore/Backend.h	Low-level interface
NodeObject	nodestore/detail/NodeObject.h	Storage unit
DatabaseNodeImp	nodestore/detail/DatabaseNodeImp.h	Single backend
DatabaseRotatingImp	nodestore/detail/DatabaseRotatingImp.h	Rotating backend
TaggedCache	nodestore/detail/TaggedCache.h	Cache implementation

Navigation by Task

Understanding SHAMap Structure

1. Start: SHAMap.h - Overview of the class

2. Read: SHAMapTreeNode.h - Base class and type system

3. Explore: SHAMapInnerNode.h - Branch structure

4. Explore: SHAMapLeafNode.h - Data storage

5. Study: detail/SHAMap.cpp - Implementation details

Understanding Node Synchronization

1. Start: shamap/SHAMapMissingNode.h - Missing node representation

2. Study: detail/SHAMapSync.cpp - Synchronization algorithm

3. Cross-reference: nodestore/Database.h - Fetch operations called during sync

4. Understand: shamap/SHAMapNodeID.h - How nodes are identified

Understanding NodeStore Architecture

1. Start: nodestore/Database.h - Public interface

2. Understand: nodestore/Backend.h - Storage abstraction

3. Explore: nodestore/detail/NodeObject.h - Storage unit

4. Study: nodestore/detail/DatabaseNodeImp.h - Standard implementation

5. Study: nodestore/detail/DatabaseRotatingImp.h - Rotation implementation

Understanding Database Rotation

1. Read: nodestore/detail/DatabaseRotatingImp.h - Architecture

2. Study: nodestore/detail/DatabaseRotatingImp.cpp - Implementation

3. Understand: Synchronization with app/misc/SHAMapStoreImp.h

Understanding Cache Layer

1. Explore: nodestore/detail/TaggedCache.h - Cache implementation

2. Study usage in: nodestore/detail/DatabaseNodeImp.cpp

3. Understand metrics in: Database metrics methods

Common Code Patterns

Accessing a Node in SHAMap

// In shamap/SHAMap.h or detail/SHAMap.cpp
std::shared_ptr<SHAMapTreeNode> node = getNodePointer(nodeID);

Storing a Node in NodeStore

// In app/misc/SHAMapStoreImp.cpp
auto obj = NodeObject::createObject(type, data, hash);
mNodeStore->store(obj);

Retrieving a Node

// In nodestore/detail/DatabaseNodeImp.cpp
auto obj = mDatabase->fetchNodeObject(hash);

Cache Hit Path

// In nodestore/detail/DatabaseNodeImp.cpp
auto cached = mCache.get(hash);  // L1 cache
if (!cached) {
    cached = mBackend->fetch(hash);  // L2 backend
    if (cached) mCache.insert(hash, cached);
}

Important Files to Read

Essential (Required Reading)

• shamap/SHAMap.h - Core API

• shamap/SHAMapNodeID.h - Navigation understanding

• nodestore/Database.h - NodeStore API

• nodestore/Backend.h - Abstraction principle

Important (Strongly Recommended)

• shamap/detail/SHAMap.cpp - Implementation

• nodestore/detail/DatabaseNodeImp.h - Cache logic

• app/misc/SHAMapStoreImp.h - Integration

Reference (For Deep Understanding)

• shamap/SHAMapInnerNode.h - Branch structure details

• shamap/SHAMapLeafNode.h - Leaf implementations

• nodestore/detail/SHAMapSync.cpp - Sync algorithm

• nodestore/detail/DatabaseRotatingImp.h - Rotation details

Building and Exploring

Compile rippled

cd rippled
mkdir build && cd build
cmake ..
make -j4

Navigate with IDE

Using VS Code or similar:

1. Open rippled repository

2. Go to Definition (Ctrl+Click) to jump to class definitions

3. Find All References (Shift+Ctrl+F) to see usage patterns

4. Use search to navigate between related classes

Debug and Trace

// Add to your code to trace execution
#include <iostream>

std::cout << "Node hash: " << node->getHash() << std::endl;
std::cout << "Node type: " << (int)node->getNodeType() << std::endl;

Test Files

Locate tests in:

rippled/src/test/*/shamap* and */nodestore*

Study how these are tested to understand expected usage patterns.

Configuration Reference

Configuration Files

NodeStore configuration in rippled.cfg:

[node_db]
type = RocksDB          # Backend choice
path = /data/node.db    # Location
cache_size = 256        # Cache size in MB
cache_age = 60          # Max age in seconds

# For NuDB
# type = NuDB
# path = /data/nudb

# For Rotating
# online_delete = 256   # Keep last N ledgers

Common Configuration Patterns

# Small validator (less memory available)
[node_db]
cache_size = 64
cache_age = 30

# Large validator (plenty of resources)
[node_db]
cache_size = 512
cache_age = 120

Quick Reference

Task	File	Function
Create NodeObject	nodestore/detail/NodeObject.h	createObject()
Fetch Node	nodestore/Database.h	fetchNodeObject()
Store Node	nodestore/Database.h	store()
Find in SHAMap	shamap/SHAMap.h	findLeaf()
Add to SHAMap	shamap/detail/SHAMap.cpp	addNode()
Check Cache	nodestore/detail/TaggedCache.h	get()
Rotate Database	nodestore/detail/DatabaseRotatingImp.h	rotate()
Get Metrics	nodestore/Database.h	getCountsJson()

Learning Path

1. Week 1: Understand architecture (Chapters 1-4 of this module)

2. Week 2: Read SHAMap.h and understand node types

3. Week 3: Study shamap/detail/SHAMap.cpp - implementation

4. Week 4: Read nodestore/Database.h and understand NodeStore design

5. Week 5: Study nodestore/detail/DatabaseNodeImp.h - caching

6. Week 6: Trace actual code execution (single transaction flow)

7. Week 7: Build and run tests

8. Week 8: Modify and experiment

This progression takes you from conceptual understanding to implementation mastery.