Doca Erasure Coding

DOCA Erasure Coding capabilities, version overlay, errors, observability, safety

Where to start: Pick the H2 anchor that matches your question (capabilities / version / errors / observability / safety) and read that section end-to-end. The tables in each section are the load-bearing content; the prose around them is interpretation.

Read this file when the loader sent you here from SKILL.md. For the how of executing each pattern (the verbs configure / build / modify / run / test / debug), jump to TASKS.md. For the canonical DOCA version-handling rules that this skill layers an EC overlay on top of, see doca-version/SKILL.md.

Pattern overview

Every DOCA Erasure Coding question this skill teaches resolves into one of FIVE patterns. The patterns are CLASSES — they apply across every EC release and every BlueField / ConnectX device the accelerator runs on.

Two cross-cutting rules that apply to every pattern above:

• Discover the version-installed surface, do not assume. Every pattern above gates on pkg-config --modversion doca-erasure-coding and on the doca_ec_cap_* capability queries against the active doca_devinfo. Quoting a max block size, a max N+K, a matrix type as universally available, or asserting any of the three task types is present without checking is the most common hallucination failure mode.

• Validate against a known-vector recover smoke BEFORE bulk. The cheapest way to confirm the configured EC context produces correct parity is to encode a small fixed N + K layout from a known input, drop one block (e.g. write zeroes over it), submit a recover task with the same matrix parameters, and bit-compare the recovered block against the original. This catches matrix-type mis-selection, N + K mismatch, block-size inconsistencies, and source / destination wiring bugs before any user storage data flows through the accelerator.

Capabilities and modes

DOCA Erasure Coding is a DOCA Core Context. Every doca_ec instance follows the universal cfg-create → cfg-set-* → init → start → use → stop → destroy lifecycle (see doca-programming-guide CAPABILITIES.md ## Capabilities and modes (doca-programming-guide/CAPABILITIES.md#capabilities-and-modes)). On top of that lifecycle, DOCA Erasure Coding layers its own task model with three independent task types and a matrix-type + N + K + block-size configuration surface.

Domain reminder — EC is a STORAGE primitive. Erasure coding encodes N original data blocks into K redundancy (parity) blocks, producing N + K total blocks; the layout tolerates ANY K simultaneous block losses without data loss. Canonical homes: RAID-6 (which is one specific N + K = 2 instance), distributed file system parity (Ceph, Lustre's PFL with parity, HDFS EC), object-storage erasure-coded buckets, and any data-durability workload where storing M whole copies (pure replication) is wasteful relative to N + K coded fragments. The agent should NOT recommend doca-erasure-coding for network workloads, control plane, or any non-storage problem — those have their own DOCA libraries.

The three task types. DOCA Erasure Coding exposes three task types, each of which is independently enabled and independently capability-gated. The agent must call the matching capability query before assuming any task type is available on the user's device.

Path selection — create vs recover vs update. All three task types can coexist on the same context; enable at least one before doca_ctx_start(). The right shape depends on the user's intended operation, not on convenience:

Update vs full re-encode — the agent rule. The most common mistake the agent should catch: a user describes "I have an existing N + K layout on disk and one data block was just rewritten. How do I refresh the parity?" and the wrong answer is "re-encode all N data blocks into K fresh parity via doca_ec_task_create". The right answer is doca_ec_task_update — it is purpose-built for that case and is algebraically much cheaper (it touches only the delta between the old and new source block, not all N source blocks). The agent should teach this as a first-class path; mis-routing here wastes accelerator cycles linear in N.

Path-selection — when to use DOCA Erasure Coding at all. The agent's rule:

• Use doca-erasure-coding when the input is a bulk storage workload that benefits from RAID-6-style resilience: a distributed file system or object store that wants to tolerate K block failures without keeping K + 1 whole copies, a backup system that wants efficient durability across many disks, an archival system encoding cold data into N + K coded fragments. Bulk encode of large datasets is the canonical fit; the accelerator amortizes the matrix arithmetic over many blocks.

• Do NOT use doca-erasure-coding when the right answer is pure replication (e.g. a system with only 3 disks where three copies are simpler and the wire cost is acceptable — EC's reconstruction CPU is wasted) — replication and EC are orthogonal, not interchangeable; when the user needs a non-Reed-Solomon scheme (fountain codes / LDPC / raptor codes — those belong on a CPU library, the DOCA Erasure Coding accelerator commits to Reed-Solomon); when the dataset is small enough that CPU EC finishes before the DMA-to-accelerator round-trip even completes (a one-shot encode of a few KiB); or when the user is asking about network durability (FEC on the wire — different domain, different libraries).

• Do NOT confuse EC with RAID-6. RAID-6 is one specific instance of erasure coding (N data blocks + 2 parity blocks across N + 2 disks, recoverable from any 2 disk failures); doca-erasure-coding generalizes that to arbitrary N + K. A user asking "I want RAID-6" on N + 2 disks is asking for an EC layout with K = 2; teach the general N + K vocabulary and let them pick K to match their durability target.

Coding parameters at create time. Before doca_ctx_start(), the user must commit four parameters that frame every subsequent task on the context:

Capability discovery — the only rule. Before enabling any task, picking a matrix type, or sizing N, K, or blocks, call the matching doca_ec_cap_* query against the active doca_devinfo:

Configuration shape. Mandatory configurations before doca_ctx_start(): at least one task type enabled (each with its matching set_conf call, success callback, error callback, and max-num-tasks budget); the four coding parameters committed (matrix type, N, K, block size); and the matching mmap permissions set on both source and destination buffers per the matrix in ## Safety policy. Optional configurations (queue sizing, per-task set_conf callback budget) use the standard DOCA Core set_* family with defaults coming from the library; query the active capability values via doca_ec_cap_*.

Version compatibility

For the canonical DOCA version-detection chain, the four-way match rule, NGC container semantics, and the headers-win-over-docs rule, see doca-version/SKILL.md. The body lives there; this skill does not duplicate it.

The EC-specific overlay is:

• Per-task support is device-bound, not version-bound. The agent must not infer *"DOCA version X includes create + recover

  • update"* from release notes alone; the runtime authority is the matching doca_ec_cap_task_create_is_supported(devinfo) / _task_recover_is_supported(devinfo) / _task_update_is_supported(devinfo) against the active device. Per the cross-cutting cap-query rule in doca-version CAPABILITIES.md ## Observability (doca-version/CAPABILITIES.md#observability), the cap query is the runtime authority — never quote any of the three task types as available from agent memory.

• Matrix-type support is device-bound, not version-bound. The device advertises which Reed-Solomon matrix variants its accelerator implements (Cauchy or Vandermonde — the two values of enum doca_ec_matrix_type) via the doca_ec_matrix_create() constructor success per variant (the public header does NOT ship a doca_ec_cap_get_matrix_* family). Asserting a matrix type from release notes alone is the second-most-common hallucination failure mode; the runtime query is the authority.

• doca-erasure-coding.pc plus doca-common.pc must both match doca_caps --version at the four-way-match check (per doca-version CAPABILITIES.md ## Version compatibility (doca-version/CAPABILITIES.md#version-compatibility)). A common partial-install pattern on hosts where the EC package was installed separately from the rest of DOCA is that doca-erasure-coding.pc reports release X while doca-common.pc reports release Y; route to doca-version TASKS.md ## debug (doca-version/TASKS.md#debug) layer 2 before any EC-layer diagnosis.

• doca_ec_task_update is the youngest of the three task types. When the user reports "the update API I'm reading about isn't on my install", the first hypothesis is that the install pre-dates update support. Confirm via doca_ec_cap_task_update_is_supported(devinfo); if false, route to doca-version TASKS.md ## debug (doca-version/TASKS.md#debug) before invoking the update codepath, and fall back to recompute via doca_ec_task_create in the meantime — flag the cost difference so the user understands the workaround is not permanent.

Error taxonomy

EC-specific overlays on the cross-library DOCA_ERROR_* taxonomy. The cross-library taxonomy itself lives in doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy); the rows below are the EC surface meaning that the agent must disambiguate before falling back to the cross-library response.

The agent's rule: never recommend a retry loop on DOCA_ERROR_* without first identifying which of the rows above is the cause. _AGAIN is the only one that wants a retry (after doca_pe_progress()); the others want investigation, not retry.

Observability

DOCA Erasure Coding observability is event-driven, not poll-driven. Every submitted task produces a completion event on the DOCA Core progress engine; there are no EC-specific counters the way Flow has per-pipe counters.

Three primary signals the agent should reach for:

1. Task completion events on the PE. Every submitted EC task (create, recover, or update) produces a completion event when it finishes (or errors). The completion carries the doca_error_t if it failed; the agent must inspect the per-task completion, not the doca_task_submit() return value alone. Submitted but no completion is almost always a missed doca_pe_progress() call in the user's main loop.

2. Capability snapshot at configure time. The output of every doca_ec_cap_* query is a snapshot of what the device's accelerator said was possible before any task was submitted (task support, max block size, max N + K, supported matrix types). Save it as the baseline; if a task later returns DOCA_ERROR_NOT_SUPPORTED or DOCA_ERROR_INVALID_VALUE the diff against this snapshot is the bug.

3. Known-vector recover smoke (encode → drop block → recover → bit-compare). The cheapest correctness signal for an EC context is to encode a small fixed N + K layout from a known input, simulate losing one block (zero it or otherwise mark it missing), submit a recover task, and bit-compare the recovered block against the original. If the recovered block does not match, the configuration is wrong; do not move on to bulk storage data. For an update task, the equivalent smoke is: encode → update one data block → recompute parity by create from the new state → compare to the parity produced by update. The two parity sets must match byte-for-byte.

For cross-cutting observability primitives (--sdk-log-level, the doca-<lib>-trace build flavor, the DOCA_LOG_LEVEL env var) see doca-debug CAPABILITIES.md ## Observability (doca-debug/CAPABILITIES.md#observability). For the install-tree observability (logger names, package layout) defer to doca-public-knowledge-map/SKILL.md.

Safety policy

Overlay on the bundle-wide hardware-safety meta-policy. The rules below are this skill's per-artifact overlay on the cross-cutting rules in doca-hardware-safety CAPABILITIES.md ## Safety policy (doca-hardware-safety/CAPABILITIES.md#safety-policy) (specifically ### Per-artifact overlay pattern (doca-hardware-safety/CAPABILITIES.md#per-artifact-overlay-pattern)). When the two layers disagree, the stricter wins; when either layer says STOP, the agent stops.

DOCA Erasure Coding's safety surface is buffer-permission discipline plus path-selection discipline. The library reads from multiple source buffers (N data blocks for create; data plus surviving parity blocks for recover; the changed source block for update) and writes into multiple destination buffers (K redundancy blocks for create; the missing blocks for recover; the K refreshed parity blocks for update); an incorrect permission flag on ANY of those buffers surfaces as a submit-time error rather than silent corruption — but only when the matrix is correct. The agent's job is to verify both permission and the should I use erasure coding at all question before any submission.

The permission matrix the agent must walk for any new EC setup. Note that EC tasks have MULTIPLE buffers on each side, so the matrix applies per buffer:

Path-selection — when NOT to use doca-erasure-coding. Equally important:

• Pure replication is enough. If the durability target is met by storing M whole copies of the data (e.g. a small cluster where one extra copy is the right answer), erasure coding is the wrong primitive. EC trades CPU / accelerator cycles (encoding, reconstruction, updates) for storage efficiency; on systems where the storage savings don't justify the reconstruction cost, replication wins.

• One-shot tiny inputs. Encoding a single small dataset that fits comfortably in CPU EC is faster on the CPU than the DMA-to-accelerator round-trip. Recommend doca-erasure-coding when the workload is bulk (many encodes, or large datasets) or when the data is already pinned in doca_mmap memory because another DOCA library produced it.

• Non-Reed-Solomon coding scheme. doca-erasure-coding is Reed-Solomon. If the user needs fountain codes, LDPC, raptor codes, or any other family, the answer is a CPU library implementing that family — not a DOCA workaround.

• Network durability (FEC on the wire). Forward error correction on a network channel is a different problem with different libraries; do not route network FEC questions through doca-erasure-coding.

• Single-block change → re-encode is wasteful. If the user is editing one source block in an existing layout and asks about refreshing parity, the right answer is doca_ec_task_update, not doca_ec_task_create. Re-running create on all N data blocks just to refresh K parity blocks is linear-in-N work the update task is designed to avoid.

The mmaps must stay valid until the EC context is destroyed. Destroying any source or destination mmap before doca_ctx_destroy() is a use-after-free on the library's bookkeeping; symptoms include DOCA_ERROR_BAD_STATE from subsequent calls and undefined behavior on outstanding tasks. This applies to EVERY buffer in an EC task (N data + K parity = N + K buffers per layout — all must remain valid).

Validate against a known-vector recover smoke BEFORE bulk submission. A single smoke (encode a tiny N + K layout, drop one block, recover, bit-compare against the original) catches matrix-type mis-selection, N + K mismatch, block-size inconsistencies, and source / destination wiring bugs before they corrupt a multi-GiB storage layout. For update workflows, add an update smoke (encode → update one block → re-encode and compare parity) as the second smoke before any bulk update path.

Recover with more than K missing blocks is unrecoverable. If K′ > K blocks are missing from an N + K layout, the math says the layout is gone. The agent must NOT invent a recovery workflow; the right answer is to restore from another replica (if the system layers replication on top of EC) or accept the data loss. Surfaces as DOCA_ERROR_INVALID_VALUE from doca_ec_task_recover_allocate_init or at submit.

Deferred topic boundaries

This skill scopes itself to the DOCA Erasure Coding library. Adjacent topics the agent will get asked but should route elsewhere:

• General coding-theory background (Galois field arithmetic, Vandermonde vs Cauchy matrix derivations, MDS code bounds, why Reed-Solomon achieves the Singleton bound) — outside this skill. Route to upstream coding-theory references; this skill assumes the user already knows they want Reed-Solomon erasure coding and is asking how to express it through the DOCA EC API.

• DOCA Core context and progress engine internals — owned by doca-programming-guide/SKILL.md. This skill uses the Core context lifecycle; it does not redefine it.

• Cross-cutting DOCA_ERROR_* taxonomy — owned by doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy). This skill adds the EC overlay, not the taxonomy itself.

• Storage stack design (how the user's distributed file system or object store should layer replication on top of EC, rebalance after node failure, schedule rebuilds, choose N + K per durability target) — outside this skill. The DOCA Erasure Coding library is one primitive inside a storage stack; the stack design lives upstream of this skill.

• Other DOCA accelerator libraries (DOCA SHA, DOCA Compress, DOCA AES-GCM, DOCA DMA) — separate libraries with their own skills. Path-selection guidance in ## Capabilities and modes names them; the deep per-library substance lives in the matching skill.

• Cross-library doca_caps invocation patterns — owned by the cross-library doca-caps/SKILL.md tool skill. This skill references the EC capability query family (doca_ec_cap_*), which is per-library; the cross-library capability snapshot tool (doca_caps --list-devs) is a separate surface.