Spark CLI

Telemetry — MANDATORY. Every api.fabric.microsoft.com call must carry x-ms-fabric-skill: spark-cli (az rest: --headers "x-ms-fabric-skill=spark-cli"), including every LRO poll, fabric_lro and retry. Snippets omit it — add it anyway.

CRITICAL NOTES

1. To find the workspace details (including its ID) from workspace name: list all workspaces and, then, use JMESPath filtering

2. To find the item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace and, then, use JMESPath filtering

3. Skill disambiguation: use spark-cli for notebook cell code (including %%sql cells), named notebook runs, Livy sessions, Spark failure triage, and everything about a Materialized Lake View -- writing the definition, reviewing a query for incremental-refresh readiness, and scheduling, refreshing, monitoring or diagnosing an existing one. A KQL materialized view in an Eventhouse is eventhouse-cli; plain read-only T-SQL against a Warehouse or Lakehouse SQL endpoint is sqldw-cli.

4. Hard routing boundary: never execute an Eventhouse/KQL materialized-view request from this skill. Route it to eventhouse-cli; if that skill is unavailable, state that the request cannot be completed in the current skill context and stop without calling Fabric APIs or creating artifacts.

Fabric Spark and Materialized Lake Views -- CLI Skill

This one skill owns Fabric Spark: notebook cell authoring, notebook runs, Livy-session analysis, Spark failure diagnostics, and the whole Materialized Lake View lifecycle.

It is a mode dispatcher and contains NO procedures. Pick the mode that matches the request from the table below, then read the matching references/<mode>.md file end to end with your file-reading tool BEFORE issuing a single command. That file holds the endpoints, payload shapes, templates and gotchas; acting without it produces wrong payloads and wrong results.

Mode selection

Mode boundary rule

Mode is decided by the artefact and the outcome, not by the language. A notebook cell is always authoring even when the cell is %%sql. A Livy session is always consumption. A Spark run that FAILED or is unhealthy is operations; a run that succeeded is reported by authoring. For a Materialized Lake View the VERB decides: writing or reviewing the definition is authoring, while discovering, scheduling, refreshing, monitoring or diagnosing existing MLVs is mlv. If discovery must be executed, switch to consumption for Livy or authoring for a notebook only after reading the discovery command from references/mlv.md.

If a request genuinely spans modes, handle them one at a time and read each reference before you start that part. If the mode is ambiguous after reading this table, ask one short clarifying question instead of guessing.

Terminal write -- the step you must not skip

Reading the reference and planning the change is NOT completing the task. Each mutating mode ends with one state-changing call. If you did not issue it, nothing was persisted -- say so explicitly rather than reporting success.

Before you report the task done, confirm the terminal call returned success and, where the reference documents a readback, read the artefact back to prove the change landed.

Shared essentials (all modes)

Resolve the workspace and item first; every mode depends on it.

Rules

MUST

• Select exactly one mode from the table above before doing anything else.

• Treat vague requests such as "set up my data" as insufficiently scoped. Ask what artefact, source, destination, and outcome the user wants; never infer intent by finding and running an existing notebook or job.

• Read references/<mode>.md end to end, as your FIRST tool call, before the first command of that mode. Read it ONCE, in a single full read: do not re-open it, do not grep it again, and do not page through it. You already have it.

• Apply the same read-once discipline to every other file you load, including nested references/<mode>/*.md sub-references and shared common/*.md files: open only the ones you actually need, read each ONCE end to end, and never grep or re-open one you already loaded.

• Resolve workspace and item ids by listing and filtering, never by guessing a GUID.

• Stop after routing any Eventhouse/KQL materialized-view request. Unavailable sibling skills are a blocking boundary, not permission to implement their workload directly.

• Announce a mode switch explicitly when the request crosses a boundary.

• Treat the reference as instructions, never as the deliverable. After reading it, RUN the documented commands against the live workspace and report the real results. Quoting what the reference says instead of executing it does not answer the request.

PREFER

• The narrowest mode that satisfies the request.

• Reading exactly ONE mode reference. Load a second only when the request genuinely spans modes, and say so before you do.

• Reporting the mode you chose in your first response so the user can correct you.

AVOID

• Acting from this dispatcher alone -- it intentionally omits the operational detail.

• Answering with a summary of the reference instead of executing it.

• Re-reading or re-grepping a reference you already loaded; it costs turns and tokens.

• Mutating anything while in a read-only mode.

• Loading a different skill for work this family already owns (see CRITICAL NOTES 3).

• Attempting an out-of-scope request through raw REST calls when the owning skill is unavailable. Name the owning skill and stop; do not improvise by creating Fabric artefacts outside spark-cli's scope.

Examples