# Processing Pipeline [← Back to Transactors: Understanding the Lifecycle of a Transaction](/core-dev-bootcamp/module03bis.md) *** ### Introduction Every transaction in the XRP Ledger passes through a rigorous multi-phase validation pipeline before it can modify ledger state. This pipeline is designed to catch errors early, minimize wasted computation, and ensure that only valid transactions with proper authorization can alter the shared ledger. Understanding this pipeline is crucial for implementing new transaction types, debugging validation failures, and reasoning about transaction behavior under different conditions. *** ### Pipeline Overview The transaction processing pipeline consists of four distinct phases: ``` ┌─────────────────────────────────────────────────────────────────────┐ │ TRANSACTION ARRIVES │ └─────────────────────────────────┬───────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ PHASE 1: PREFLIGHT │ │ ───────────────── │ │ • Stateless validation (no ledger access) │ │ • Format and field validation │ │ • Signature structure check (not verification) │ │ • Returns: NotTEC (cannot return tec codes) │ └─────────────────────────────────┬───────────────────────────────────┘ │ ┌────────┴────────┐ │ │ ▼ ▼ ✓ Pass ✗ Fail │ │ │ └──→ Reject (tem*, tef*, ter*) ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ PHASE 2: PRECLAIM │ │ ──────────────── │ │ • Read-only ledger access │ │ • Account existence, balance, sequence checks │ │ • Fee validation │ │ • Signature verification │ │ • Transaction-specific state checks │ │ • Returns: TER (can return tec codes) │ └─────────────────────────────────┬───────────────────────────────────┘ │ ┌────────┴────────┐ │ │ ▼ ▼ ✓ Pass ✗ Fail │ │ │ └──→ Reject or Queue ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ PHASE 3: DOAPPLY │ │ ─────────────── │ │ • Full read/write ledger access │ │ • Actual state modifications │ │ • Creates, updates, or deletes ledger entries │ │ • Returns: TER │ └─────────────────────────────────┬───────────────────────────────────┘ │ ┌────────┴────────┐ │ │ ▼ ▼ tesSUCCESS tec* Fail │ │ │ └──→ Fee charged, changes reverted ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ PHASE 4: FINALIZATION │ │ ─────────────────── │ │ • Commit changes to ledger │ │ • Record transaction metadata │ │ • Consume sequence number │ └─────────────────────────────────────────────────────────────────────┘ ``` *** ### Phase 1: Preflight Preflight performs **stateless validation**—checks that depend only on the transaction content itself, not on any ledger state. #### Characteristics * **No ledger access**: Only has access to `PreflightContext` (no `view`) * **Deterministic**: Same transaction always produces the same result * **Can run in parallel**: No shared state dependencies * **Returns `NotTEC`**: Cannot return `tec` codes (those require fee claiming) #### What Preflight Checks 1. **Amendment/Feature enablement**: Is the transaction type enabled? 2. **Flag validation**: Are only valid flags set? 3. **Field presence**: Are required fields present? 4. **Field format**: Are field values well-formed? 5. **Field ranges**: Are numeric values within valid ranges? 6. **Logical consistency**: Are field combinations valid? #### Implementation Pattern Transaction-specific preflight is implemented as a static method: ```cpp NotTEC CreateCheck::preflight(PreflightContext const& ctx) { // Check self-send if (ctx.tx[sfAccount] == ctx.tx[sfDestination]) { JLOG(ctx.j.warn()) << "Malformed transaction: Check to self."; return temREDUNDANT; } // Validate SendMax STAmount const sendMax{ctx.tx.getFieldAmount(sfSendMax)}; if (!isLegalNet(sendMax) || sendMax.signum() <= 0) { JLOG(ctx.j.warn()) << "Malformed transaction: bad sendMax amount"; return temBAD_AMOUNT; } // Validate currency if (badCurrency() == sendMax.getCurrency()) { JLOG(ctx.j.warn()) << "Malformed transaction: Bad currency."; return temBAD_CURRENCY; } // Validate expiration if present if (auto const optExpiry = ctx.tx[~sfExpiration]) { if (*optExpiry == 0) { JLOG(ctx.j.warn()) << "Malformed transaction: bad expiration"; return temBAD_EXPIRATION; } } return tesSUCCESS; } ``` #### Why Preflight Matters 1. **Early rejection**: Catches obviously invalid transactions before expensive operations 2. **Security**: Runs before signature verification, so malicious actors can't waste resources 3. **Cannot claim fees**: Returns `NotTEC`, so no fees can be claimed for malformed transactions > **Important**: Preflight cannot return `tec` codes because those codes claim the transaction fee. Since preflight runs before signature verification, allowing fee claiming would let attackers drain accounts by submitting malformed transactions with valid account IDs but invalid signatures. *** ### Phase 2: Preclaim Preclaim performs **ledger-state validation** with read-only access to the ledger. #### Characteristics * **Read-only ledger access**: Has `ReadView const& view` * **Signature verified**: By this point, the signature has been checked * **Can return `tec` codes**: Fee can be claimed * **Determines if transaction should be queued or applied** #### What Preclaim Checks 1. **Account existence**: Does the sender account exist? 2. **Sequence number**: Is the sequence correct? 3. **Fee adequacy**: Is the fee sufficient? 4. **Balance sufficiency**: Can the account pay the fee? 5. **Destination checks**: Does destination exist? Are permissions met? 6. **Trust line state**: Are trust lines frozen? Is authorization required? 7. **Expiration**: Has the transaction or related object expired? #### Implementation Pattern ```cpp TER CreateCheck::preclaim(PreclaimContext const& ctx) { // Check destination exists AccountID const dstId{ctx.tx[sfDestination]}; auto const sleDst = ctx.view.read(keylet::account(dstId)); if (!sleDst) { JLOG(ctx.j.warn()) << "Destination account does not exist."; return tecNO_DST; } auto const flags = sleDst->getFlags(); // Check incoming permission if (ctx.view.rules().enabled(featureDisallowIncoming) && (flags & lsfDisallowIncomingCheck)) return tecNO_PERMISSION; // Check pseudo-account if (isPseudoAccount(sleDst)) return tecNO_PERMISSION; // Check destination tag requirement if ((flags & lsfRequireDestTag) && !ctx.tx.isFieldPresent(sfDestinationTag)) { JLOG(ctx.j.warn()) << "Malformed transaction: DestinationTag required."; return tecDST_TAG_NEEDED; } // Check freeze status for non-XRP amounts STAmount const sendMax{ctx.tx[sfSendMax]}; if (!sendMax.native()) { AccountID const& issuerId{sendMax.getIssuer()}; if (isGlobalFrozen(ctx.view, issuerId)) { JLOG(ctx.j.warn()) << "Creating a check for frozen asset"; return tecFROZEN; } // Additional trust line freeze checks... } // Check expiration if (hasExpired(ctx.view, ctx.tx[~sfExpiration])) { JLOG(ctx.j.warn()) << "Creating a check that has already expired."; return tecEXPIRED; } return tesSUCCESS; } ``` #### Key Utility Functions for Preclaim | Function | Purpose | | -------------------------------- | ------------------------------------------------ | | `ctx.view.read(keylet)` | Read a ledger entry | | `isGlobalFrozen(view, issuer)` | Check if an issuer has globally frozen | | `isFrozen(view, account, issue)` | Check if a specific trust line is frozen | | `hasExpired(view, expiration)` | Check if a time has passed | | `isPseudoAccount(sle)` | Check if account is a pseudo-account (AMM, etc.) | *** ### Phase 3: doApply doApply performs the **actual ledger modifications**. This is where state changes happen. #### Characteristics * **Full read/write access**: Has `ApplyView& view()` * **Atomic execution**: All changes succeed or all are reverted * **Can create, update, delete ledger entries** * **Must handle reserve requirements** #### Implementation Pattern ```cpp TER CreateCheck::doApply() { auto const sle = view().peek(keylet::account(account_)); if (!sle) return tefINTERNAL; // Check reserve before creating new object STAmount const reserve{ view().fees().accountReserve(sle->getFieldU32(sfOwnerCount) + 1)}; if (mPriorBalance < reserve) return tecINSUFFICIENT_RESERVE; // Create the new ledger entry std::uint32_t const seq = ctx_.tx.getSeqValue(); Keylet const checkKeylet = keylet::check(account_, seq); auto sleCheck = std::make_shared(checkKeylet); // Set required fields sleCheck->setAccountID(sfAccount, account_); sleCheck->setAccountID(sfDestination, ctx_.tx[sfDestination]); sleCheck->setFieldU32(sfSequence, seq); sleCheck->setFieldAmount(sfSendMax, ctx_.tx[sfSendMax]); // Set optional fields if (auto const srcTag = ctx_.tx[~sfSourceTag]) sleCheck->setFieldU32(sfSourceTag, *srcTag); if (auto const dstTag = ctx_.tx[~sfDestinationTag]) sleCheck->setFieldU32(sfDestinationTag, *dstTag); if (auto const invoiceId = ctx_.tx[~sfInvoiceID]) sleCheck->setFieldH256(sfInvoiceID, *invoiceId); if (auto const expiry = ctx_.tx[~sfExpiration]) sleCheck->setFieldU32(sfExpiration, *expiry); // Insert into ledger view().insert(sleCheck); // Add to owner directories auto const page = view().dirInsert( keylet::ownerDir(account_), checkKeylet, describeOwnerDir(account_)); if (!page) return tecDIR_FULL; sleCheck->setFieldU64(sfOwnerNode, *page); // Update owner count adjustOwnerCount(view(), sle, 1, ctx_.app.journal("View")); return tesSUCCESS; } ``` #### Key Operations in doApply | Operation | Method | Description | | ------------------ | ----------------------- | --------------------------------- | | Read entry | `view().peek(keylet)` | Get modifiable reference to entry | | Create entry | `view().insert(sle)` | Add new entry to ledger | | Update entry | `view().update(sle)` | Mark entry as modified | | Delete entry | `view().erase(sle)` | Remove entry from ledger | | Add to directory | `view().dirInsert(...)` | Add entry to owner directory | | Update owner count | `adjustOwnerCount(...)` | Increment/decrement owner count | *** ### Phase 4: Finalization After doApply succeeds (or fails with a `tec` code), the transaction is finalized: 1. **Fee consumption**: Transaction fee is deducted from the sender 2. **Sequence advancement**: Account sequence number is incremented 3. **Metadata recording**: Changes are recorded in transaction metadata 4. **State commitment**: Changes are committed to the ledger (or reverted for `tec`) This phase is handled by the engine, not by individual transactors. *** ### Error Propagation Different phases can return different categories of result codes: | Phase | Can Return | Fee Charged? | Notes | | --------- | ----------------------------- | ----------------- | --------------------------- | | Preflight | `tem*`, `tef*`, `ter*`, `tes` | No | No `tec` codes allowed | | Preclaim | All codes | For `tec*` only | May queue for `ter*` | | doApply | All codes | For `tec*`, `tes` | Changes reverted for `tec*` | *** ### Practical Example: Tracing a CheckCreate Let's trace a CheckCreate transaction through all phases: ``` Transaction: CheckCreate Account: rAlice Destination: rBob SendMax: 100 XRP Expiration: 750000000 ``` **Preflight:** 1. Check `rAlice != rBob` → Pass 2. Validate `100 XRP` is positive and legal → Pass 3. Validate expiration `750000000 != 0` → Pass 4. Result: `tesSUCCESS` **Preclaim:** 1. Read Bob's account → Exists 2. Check `lsfDisallowIncomingCheck` → Not set 3. Check `isPseudoAccount(Bob)` → False 4. Check `lsfRequireDestTag` → Not set (or tag provided) 5. Check `isGlobalFrozen(XRP)` → XRP can't be frozen, skip 6. Check `hasExpired(750000000)` → Not expired 7. Result: `tesSUCCESS` **doApply:** 1. Read Alice's account (peek) 2. Calculate reserve for +1 owner count → e.g., 12 XRP 3. Check `mPriorBalance >= reserve` → Pass 4. Create Check SLE with all fields 5. Insert Check into ledger 6. Add to Bob's owner directory → Get page number 7. Add to Alice's owner directory → Get page number 8. Increment Alice's owner count 9. Result: `tesSUCCESS` **Finalization:** 1. Deduct fee from Alice 2. Increment Alice's sequence 3. Record metadata 4. Commit changes *** ### Best Practices for Implementing Transactors 1. **Preflight should be stateless**: Never access ledger state in preflight 2. **Fail early**: Check the cheapest conditions first 3. **Use appropriate error codes**: `tem*` for format errors, `tec*` for state-dependent failures 4. **Check reserves before creating objects**: Use `mPriorBalance` for reserve checks 5. **Always update owner count**: When creating or deleting owned objects 6. **Always update directories**: Add/remove from owner directories 7. **Log warnings**: Use `JLOG` to help with debugging *** ### Codebase References | File | Description | | ---------------------------------------- | -------------------------------------------- | | `src/xrpld/app/tx/detail/applySteps.cpp` | Transaction dispatch and phase orchestration | | `src/xrpld/app/tx/detail/apply.cpp` | Core apply logic | | `src/xrpld/app/tx/detail/Transactor.cpp` | Base class phase implementations | --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.xrpl-commons.org/core-dev-bootcamp/module03bis/processing-pipeline.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.