Doca Compress

DOCA Compress capabilities, version overlay, errors, observability, safety

Where to start: Pick the H2 anchor that matches your question (capabilities / version / errors / observability / safety) and read that section end-to-end. The tables in each section are the load-bearing content; the prose around them is interpretation.

Read this file when the loader sent you here from SKILL.md. For the how of executing each pattern (the verbs configure / build / modify / run / test / debug), jump to TASKS.md. For the canonical DOCA version-handling rules that this skill layers a Compress overlay on top of, see doca-version/SKILL.md.

Pattern overview

Every DOCA Compress question this skill teaches resolves into one of FIVE patterns. The patterns are CLASSES — they apply across every Compress release and every BlueField / ConnectX device the accelerator runs on.

Two cross-cutting rules that apply to every pattern above:

• Discover the version-installed surface, do not assume. Every pattern above gates on pkg-config --modversion doca-compress and on the doca_compress_cap_* capability queries against the active doca_devinfo. Quoting a per-task max buffer size, or asserting either task type is universally available, without checking, is the most common hallucination failure mode.

• Validate against a round-trip BEFORE bulk. The cheapest way to confirm the configured Compress context produces a correct encoding is to compress a small fixed input and decompress the output on the accelerator (or against a published CPU encoder like zlib), compare to the original byte-for-byte, before submitting any bulk user data. For decompression-only consumers, the equivalent smoke is to decompress a small fixture produced by a published CPU encoder.

Capabilities and modes

DOCA Compress is a DOCA Core Context. Every doca_compress instance follows the universal cfg-create → cfg-set-* → init → start → use → stop → destroy lifecycle (see doca-programming-guide CAPABILITIES.md ## Capabilities and modes (doca-programming-guide/CAPABILITIES.md#capabilities-and-modes)). On top of that lifecycle, DOCA Compress layers its own task model: DEFLATE encode and DEFLATE / LZ4 decode modes, each independently capability-gated.

The task types. The agent must call the matching support, maximum-buffer-size, and (for LZ4) maximum-buffer-list-length queries before assuming a mode is available or sizing its input.

Path selection — compress vs decompress, or both. Both task types can coexist on the same context; enable at least one before doca_ctx_start(). Enabling both is appropriate when the user is running a round-trip (compress on the producer side, decompress on the consumer side, same process or same machine); enabling only one is the right shape for a one-direction pipeline.

Path-selection — when to use DOCA Compress at all. The agent's rule:

• Use doca-compress when the input is bulk (rule of thumb: ≥ a few KiB, with the canonical fit being inputs of file-size scale — typically ≥ 4 KiB and upwards), the throughput requirement is high enough that freeing the CPU for other work is valuable, or the input is already pinned in doca_mmap memory because another DOCA library produced it. File compression, deduplication preprocessing, and content-addressed storage prep are the canonical fits.

• Do NOT use doca-compress when the input is tiny one-shot (e.g. 64-byte buffers — the per-call DMA-to-accelerator setup cost dominates; CPU compression with zlib / zstd is faster); when the user needs outbound compression in an algorithm other than DEFLATE (use a CPU compression library — DOCA Compress exposes ONLY DEFLATE on the compress / encode side). However, DOCA Compress does support LZ4 decompress on the inbound / decode side, via two distinct task families (doca_compress_task_decompress_lz4_stream and doca_compress_task_decompress_lz4_block), each with its own _cap_task_decompress_lz4_{stream,block}_is_supported(devinfo)

  • _get_max_buf_size + _get_max_buf_list_len queries (and corresponding shipped samples under /opt/mellanox/doca/samples/doca_compress/). Do NOT blanket- refuse LZ4 — only outbound LZ4 encode is out of scope. The rule of thumb is: DEFLATE both directions; LZ4 decompress only; everything else (zstd / Snappy / brotli / …) → CPU. When the user only wants to move bytes without encoding them, use doca-dma/SKILL.md — pure mmap-to-mmap copy is DMA's job, not Compress's.

The size-threshold the cap query does NOT report. The capability queries below report the per-task maximum input size; they do NOT report the per-task minimum below which CPU compression beats accelerator submission. That floor is device-dependent and workload-dependent, and the agent must own it as a path-selection rule rather than read it from the device. A small smoke benchmark on representative inputs (CPU zlib elapsed vs doca-compress elapsed end-to-end including submit + progress + completion drain) is the only authoritative answer for "is offload worth it for inputs of size N" on a given host.

Capability discovery — the only rule. Before enabling either task or sizing any buffer, call the matching doca_compress_cap_* query against the active doca_devinfo:

Configuration shape. Mandatory configurations before doca_ctx_start(): enable at least one source-verified task configuration matching the selected DEFLATE encode, DEFLATE decode, LZ4 stream decode, or LZ4 block decode mode, and set the matching mmap permissions on both source and destination buffers per the matrix in ## Safety policy. Optional configurations (queue sizing, per-task set_conf callback budget) use the standard DOCA Core set_* family with defaults coming from the library; query the active capability values via doca_compress_cap_*.

Version compatibility

For the canonical DOCA version-detection chain, the four-way match rule, NGC container semantics, and the headers-win-over-docs rule, see doca-version/SKILL.md. The body lives there; this skill does not duplicate it.

The Compress-specific overlay is:

• Per-task support is device-bound, not version-bound. The agent must not infer "DOCA version X includes the compress task and the decompress task" from release notes alone; the runtime authority is doca_compress_cap_task_compress_deflate_is_supported(devinfo) and _decompress_deflate_is_supported(devinfo) against the active device. Per the cross-cutting cap-query rule in doca-version CAPABILITIES.md ## Observability (doca-version/CAPABILITIES.md#observability), the cap query is the runtime authority — never quote either task as available from agent memory.

• doca-compress.pc plus doca-common.pc must both match doca_caps --version at the four-way-match check (per doca-version CAPABILITIES.md ## Version compatibility (doca-version/CAPABILITIES.md#version-compatibility)). A common partial-install pattern on hosts where the Compress package was installed separately from the rest of DOCA is that doca-compress.pc reports release X while doca-common.pc reports release Y; route to doca-version TASKS.md ## debug (doca-version/TASKS.md#debug) layer 2 before any Compress-layer diagnosis.

• Per-task max buffer size is device-bound, not version-bound. The _get_max_buf_size queries are the runtime authority; the matching version-matrix.json row records the promise. If the two disagree, the cap-query value wins, per the headers-win-over-docs rule in doca-version/SKILL.md.

Error taxonomy

Compress-specific overlays on the cross-library DOCA_ERROR_* taxonomy. The cross-library taxonomy itself lives in doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy); the rows below are the Compress surface meaning that the agent must disambiguate before falling back to the cross-library response.

The agent's rule: never recommend a retry loop on DOCA_ERROR_* without first identifying which of the rows above is the cause. _AGAIN is the only one that wants a retry (after doca_pe_progress()); the others want investigation, not retry.

Observability

DOCA Compress observability is event-driven, not poll-driven. Every submitted task produces a completion event on the DOCA Core progress engine; there are no Compress-specific counters the way Flow has per-pipe counters.

Three primary signals the agent should reach for:

1. Task completion events on the PE. Every submitted Compress task (compress or decompress) produces a completion event when it finishes (or errors). The completion carries the doca_error_t if it failed AND the actual output size produced; the agent must inspect the per-task completion, not the doca_task_submit() return value alone. Submitted but no completion is almost always a missed doca_pe_progress() call in the user's main loop.

2. Capability snapshot at configure time. The output of every doca_compress_cap_* query is a snapshot of what the device's accelerator said was possible before any task was submitted. Save it as the baseline; if a task later returns DOCA_ERROR_NOT_SUPPORTED or DOCA_ERROR_INVALID_VALUE the diff against this snapshot is the bug.

3. Round-trip validation on a known input. The cheapest correctness signal is to compress a small fixed input and decompress the output (either on the accelerator if both tasks are enabled, or against a published CPU decoder like zlib) and compare to the original byte-for-byte. If the round-trip does not match, the configuration is wrong; do not move on to bulk input. For decompression-only consumers, the equivalent smoke is decompressing a small DEFLATE blob produced by a published CPU encoder and comparing to the original input.

For cross-cutting observability primitives (--sdk-log-level, the doca-<lib>-trace build flavor, the DOCA_LOG_LEVEL env var) see doca-debug CAPABILITIES.md ## Observability (doca-debug/CAPABILITIES.md#observability). For the install-tree observability (logger names, package layout) defer to doca-public-knowledge-map/SKILL.md.

Safety policy

Overlay on the bundle-wide hardware-safety meta-policy. The rules below are this skill's per-artifact overlay on the cross-cutting rules in doca-hardware-safety CAPABILITIES.md ## Safety policy (doca-hardware-safety/CAPABILITIES.md#safety-policy) (specifically ### Per-artifact overlay pattern (doca-hardware-safety/CAPABILITIES.md#per-artifact-overlay-pattern)). When the two layers disagree, the stricter wins; when either layer says STOP, the agent stops.

DOCA Compress's safety surface is buffer-permission discipline plus path-selection discipline. The library reads from a source buffer and writes encoded (or decoded) output into a destination buffer; an incorrect permission flag on either side surfaces as a submit-time error rather than silent corruption — but only when the matrix is correct. The agent's job is to verify both permission and the should I use the accelerator at all question before any submission.

The permission matrix the agent must walk for any new Compress setup:

Path-selection — when NOT to use doca-compress. Equally important:

• Tiny one-shot inputs. A single 64-byte compress on the CPU is faster than the DMA-to-accelerator setup. The agent should recommend doca-compress only when the input is bulk (rule of thumb: ≥ a few KiB, typically ≥ 4 KiB and upwards) or repeated, or the input is already pinned in doca_mmap memory because another DOCA library produced it.

• Non-DEFLATE / non-LZ4 algorithm. The public task names commit doca-compress to DEFLATE (the same algorithm underlying zlib and gzip) on the encode side (doca_compress_task_compress_deflate) and to DEFLATE + LZ4 stream / LZ4 block on the decode side (doca_compress_task_decompress_deflate, doca_compress_task_decompress_lz4_stream, doca_compress_task_decompress_lz4_block). If the user needs a different algorithm (zstd, Snappy, brotli, …, or LZ4 on the encode side), the answer is to use a CPU library that implements it, not to invent a DOCA workaround.

• Pure copy, no compression. If the user only wants to move bytes between two doca_mmap regions without encoding them, doca-dma/SKILL.md is the right library — its memcpy task is purpose-built for that pattern. Layering doca-compress on top wastes accelerator cycles on encoding output the user does not want.

The mmap must stay valid until the Compress context is destroyed. Destroying either the source or destination mmap before doca_ctx_destroy() is a use-after-free on the library's bookkeeping; symptoms include DOCA_ERROR_BAD_STATE from subsequent calls and undefined behavior on outstanding tasks.

Validate against a round-trip BEFORE bulk submission. A single round-trip smoke (compress one short input and decompress the output, or decompress a single small published DEFLATE fixture) catches algorithm mis-selection, endian assumptions, and buffer-sizing bugs before they corrupt a multi-GiB result.

Deferred topic boundaries

This skill scopes itself to the DOCA Compress library. Adjacent topics the agent will get asked but should route elsewhere:

• General compression algorithm background (entropy coding, why DEFLATE produces sub-optimal ratios on already-compressed inputs, when zstd is preferred to gzip, compression-level trade-offs in CPU libraries) — outside this skill. Route to upstream zlib / DEFLATE specification documentation; this skill assumes the user already knows they want DEFLATE and is asking how to express it through the DOCA Compress API.

• DOCA Core context and progress engine internals — owned by doca-programming-guide/SKILL.md. This skill uses the Core context lifecycle; it does not redefine it.

• Cross-cutting DOCA_ERROR_* taxonomy — owned by doca-programming-guide CAPABILITIES.md ## Error taxonomy (doca-programming-guide/CAPABILITIES.md#error-taxonomy). This skill adds the Compress overlay, not the taxonomy itself.

• Pure mmap-to-mmap copies — owned by doca-dma/SKILL.md. When the user wants bytes moved without encoding, DMA is the answer; Compress is not.

• Other DOCA accelerator libraries (DOCA SHA, DOCA AES-GCM, DOCA Erasure Coding, …) — separate libraries with their own skills. Path-selection guidance in ## Capabilities and modes names them; the deep per-library substance lives in the matching skill.

• Cross-library doca_caps invocation patterns — owned by the cross-library doca-caps/SKILL.md tool skill. This skill references the Compress capability query family (doca_compress_cap_*), which is per-library; the cross-library capability snapshot tool (doca_caps --list-devs) is a separate surface.