K6 Cloud Investigate Test
Grafana LabsOfficialPublished by Grafana Labs
- Skill
- Workflow
- Cloud and DevOps
- Apache-2.0
- No issues found
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a systematic approach to investigating Grafana Cloud k6 performance tests. It guides users through identifying test runs, fetching metadata and scripts, analyzing pass/fail status across multiple layers, and extracting metrics and logs. It emphasizes safe practices, such as avoiding accidental script overwrites, and provides a framework for diagnosing failures by correlating run-level results, in-script checks, and infrastructure metrics.
What it does
- Identifies and retrieves metadata for specific k6 cloud tests and runs.
- Lists and paginates run history to identify trends or specific anomalies.
- Determines pass/fail status by evaluating run-level results, status, and per-check metrics.
- Fetches and summarizes raw metric time-series and log lines for deep-dive analysis.
- Provides a safe, step-by-step procedure for editing test scripts.
- Generates structured reports for single-run investigations or multi-run comparisons.
When to use it
- When you need to understand why a specific k6 test run failed.
- When you want to compare performance metrics across multiple test runs.
- When you need to verify if a test is healthy or if thresholds are configured correctly.
- When you need to extract logs or metrics for a past test run.
- When you need to safely update a test script to fix a failing threshold.
Advantages
- Prevents accidental script overwrites by enforcing a read-before-write policy.
- Provides a clear, multi-layered approach to distinguishing between script issues and platform aborts.
- Ensures accurate data analysis by verifying date alignment and handling pagination correctly.
- Standardizes the investigation process with clear reporting templates.
- Helps identify the root cause of 'zero-observation' threshold traps.
Usage examples
- “Why did my k6 test run 12345 fail?”
- The agent will fetch the run metadata, check the logs for errors, analyze the per-check success ratio, and report whether the failure was due to a threshold breach or a platform-side issue.
- “Show me the metrics for the latest run of my search test.”
- The agent will identify the most recent run for the specified test, retrieve the relevant performance metrics, and present a summary of key indicators like iteration duration and check success rates.
- “Is my test healthy? Here is the link: /a/k6-app/tests/555”
- The agent will list the recent run history, check the pass/fail status of the latest runs, and provide a summary of the test's overall health based on the retrieved data.
- “I need to add a threshold to my k6 script to catch navigation failures.”
- The agent will guide you through the safe-edit workflow: fetching the current script, creating a backup, applying the threshold change, and verifying the update.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/grafana/k6-cloud-investigate-test/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip k6-cloud-investigate-test.zip -d .agents/skills
Documentation by Grafana Labs
k6 Cloud test investigator
Structured 9-step workflow for investigating a Grafana Cloud k6 test or run.
This skill is workflow-only. All API calls go through gcx api against the plugin proxy, using v6 for REST and v5 for metrics. For the underlying mechanics (gcx auth, path conventions, endpoint discovery, log queries, script editing, threshold semantics, gotchas), see the k6-manage skill — every step below references it.
The content unique to this skill is:
the ordered investigation flow (Steps 1-9 below)
the date-alignment check ("last 7 days" ≠ "last 7 runs")
the 3-layer pass/fail determination (
resultvsstatusvs per-checkchecksquery)a worked example with realistic numbers (
references/worked-example.md)
Core principles
Read before write. Always GET the script before any PUT.
gcx k6 load-tests update-scriptis a write that replaces the live script with whatever file you pass — running it "just to see the URL it hits" has cost users their production scripts. If you need to learn URLs, run any non-mutating command with-vvv --log-http-payloadinstead.Paginate when enumerating runs. The
/test_runsendpoint caps at 1000 rows andgcx k6 runs list --limit 0does not auto-follow@nextLink. Use thegcx apiloop documented ink6-manage§3.Verify date framing. When the user says "last 7 days", "this week", "recent runs" — confirm the most-recent run's
createdactually falls in that window. Surface the gap if not.check()doesn't fail runs; onlythresholdsdo. And thresholds with zero observations are reported as ✓ pass. See "Threshold semantics" below for the full deep dive; the per-checkchecksmetric query in Step 5 catches both cases.
Prerequisites
gcx installed and authenticated against the user's stack. See k6-manage §1. Verify with:
gcx --context <stack> config check # expect "✔ Connectivity: online"Investigation workflow
Step 1: Identify the test and target run(s)
From the user's URL:
/a/k6-app/tests/<id>→ load test (parent of many runs)/a/k6-app/runs/<id>→ a specific run
To go run → test: fetch the run via gcx api, see k6-manage §2 for the path-shape rules:
gcx --context <stack> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<run_id>and read .test_id from the response. To list runs for a test, see Step 3.
Step 2: Fetch test metadata and script
gcx --context <stack> k6 load-tests get <test_id> -o jsonFor the script, follow the GET half of the safe-edit recipe in k6-manage §5 — save a backup if you'll be editing later.
Two script endpoints exist, and the difference matters for investigation. k6-manage §5 documents both: the current load-test script and the per-run snapshot that was actually executed. They drift apart whenever the script is edited after a run. Whenever the question involves "what changed", "why did this run fail", or you're examining a run more than a few days old, also fetch the run-bundled snapshot(s) via k6-manage §5's run-script endpoint and diff against the current load-test script (or against another run's snapshot). The current load-test script is the wrong artifact to reason about a past run.
Step 3: List runs WITH pagination
Use the gcx api + @nextLink loop pattern documented in k6-manage §3 against /cloud/v6/load_tests/<test_id>/test_runs. After collecting all_runs:
print(f"Total: {len(all_runs)}")
runs_sorted = sorted(all_runs, key=lambda r: r['created'], reverse=True)
for r in runs_sorted[:10]:
print(f" {r['created']:30s} id={r['id']:>8} status={r['status']:<10} result={r.get('result','?')}")Report to user: total run count, date range, latest run date. If "latest run" is more than a day old, call it out — they may believe the schedule is firing when it isn't.
Step 4: Verify date alignment with user's intent
If the user asked for "last 7 days" / "this week" / "recent": filter by date range, not by row count — "7 most recent runs" could span a day or a year depending on how often the test runs.
last7 = [r for r in all_runs if r['created'] >= '<today_minus_7_days_iso>']If len(last7) == 0: surface this to the user immediately. Don't proceed with stale data.
Step 5: Determine pass/fail status
For each run examine three independent layers:
| Layer | Field | Meaning |
|---|---|---|
| Run-level outcome | result (passed / failed / error / aborted) | Whether thresholds breached |
| Run-level status | status (completed / aborted) | Whether the run finished orderly |
| In-script checks | v5 checks metric, aggregated by the check label | Per-check success rate |
For a run that the user thinks is "failing" but reports result: passed: check the third layer. Common pattern: every iteration's check() returns false but the run still "passes" because no threshold is defined on checks. See "Threshold semantics" below for the full deep dive (zero-observation trap, abortOnFail cloud delay, operator support).
Query the per-check breakdown via v5 (see k6-manage/references/metrics.md §7 for query_aggregate_k6 shape):
gcx --context <stack> api \
"/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/query_aggregate_k6(query='ratio by (check)',metric='checks')"Each result entry in the response carries a check label with the check name and a single value: the success ratio (0.0–1.0). For raw success/fail counts, also query increase_nz by (check) (successes) and increase_z by (check) (failures).
Step 6: Fetch metrics for a run
Use the v5 metrics endpoints documented in k6-manage/references/metrics.md. Typical workflow:
# 6a. List metrics available for the run (metrics.md §1)
gcx --context <stack> api /api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/metrics
# 6b. List labels for a metric to know what's available to filter/group by (metrics.md §4)
gcx --context <stack> api \
"/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/labels?match[]=http_req_duration"
# 6c. Time-series — pick a query method that matches the metric's type (metrics.md §6)
gcx --context <stack> api \
"/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/query_range_k6(query='histogram_quantile(0.95) by (name,status)',metric='http_req_duration',step=10)"
# 6d. Scalar aggregate over the whole run (metrics.md §7)
gcx --context <stack> api \
"/api/plugins/k6-app/resources/cloud/cloud/v5/test_runs/<run_id>/query_aggregate_k6(query='increase',metric='http_reqs')"Picking the right query method per metric type matters — see the "Query methods" tables in k6-manage/references/metrics.md. For browser tests, the URL/status breakdown comes from labels + label/{name}/values (metrics.md §4/§5), not from a separate tags endpoint.
Step 7: Fetch logs for a run
Follow the Loki recipe in k6-manage §4 (gcx api against /api/plugins/k6-app/resources/logs/..., {test_run_id="<id>"} selector, mandatory X-K6TestRun-Id header, scope start/end to run.created/run.ended). Save the response to /tmp/run_<id>_logs.json and summarise from the file as described there.
Step 8: (If asked) edit the script safely
Use the full safe-edit recipe in k6-manage §5 (GET → backup → edit → k6 inspect → 1-iter smoke → PUT with Content-Type: application/octet-stream → sha256 verify).
When the edit involves thresholds, also read "Threshold semantics" below for the zero-observation trap, the abortOnFail cloud delay, and operator support (==/!= work despite docs).
Step 9: Report
Pick a template based on the question shape.
For single-run investigations ("what's going on with run X"):
Test: <name> (id <test_id>)
Run: <run_id> @ <created> → <ended>, load_zone=<zone>, result=<result>
Logs (<n> streams, <m> lines):
- <stream summary>
Metrics (key indicators):
- <metric>: <method> = <value> (n=<count>)
...
Per-check breakdown:
- <check name>: <success_rate> (succ=<n>, fail=<n>)
...
Diagnosis: <one-paragraph>For multi-run comparisons ("what's different between passing and failing runs", "did this change cause the failures"):
Question: <one-line restatement of what the user is trying to attribute>
Run timeline (relevant window):
| Run ID | Created (UTC) | Result | exec_duration | processing_duration | k6 build | error code | key check ratios |
|---|---|---|---|---|---|---|---|
| ... | ... | passed | 60s | 195s | <build> | — | 1.0/1.0/1.0 |
| ... | ... | failed | 27s | 193s | <build> | — | 0.0/n=0/n=0 |
| ... | ... | error | 51s | 3601s aborted | <build> | 8016 | 1.0/1.0/1.0 |
Differences that matter:
- <field>: <value-in-passing> vs <value-in-failing> — <interpretation>
...
Script diff (if relevant): <bundled-script diff between a representative passing and failing run, summarised>
Diagnosis: <one-paragraph attributing the change to test-side, SUT-side, or platform-side, with the supporting evidence>The table makes side-by-side anomalies obvious (e.g. a column that's identical across passing and failing rules out that dimension as the cause; a column that flips between them is your candidate).
If the user asked for raw data dumps, also save them to /tmp/k6inv/run<id>/ and tell them the paths.
Threshold semantics
Thresholds, not check() calls, are what determine a run's result. The behaviour has several non-obvious corners that mislead first-time investigators, so they're collected here. Step 5 ("Determine pass/fail status") and Step 8 ("edit the script safely") both depend on this section.
What result means
result value | Meaning |
|---|---|
passed | All thresholds passed (or none defined) |
failed | At least one threshold failed |
error | Either the script crashed before finishing (e.g. browser wouldn't launch) or k6 Cloud aborted the run platform-side. To tell which, check status_history[*].extra.code on the run — a non-null code indicates a platform abort (e.g. 8016 = "Test run max lifetime exceeded" during processing_metrics). Platform aborts are not your code's fault even though they surface as error. |
aborted | User or system aborted the run |
check() calls do not affect result directly. They emit observations into the built-in checks rate metric, which a threshold may then evaluate. If no threshold references checks, failing checks are invisible to the run-level result.
The "zero observations = pass" trap
k6 reports a threshold as ✓ pass when the underlying metric has zero samples — even if the threshold expression would otherwise evaluate to false:
checks{check:response is 200}
✓ 'rate==1.0' rate=0.00% ← ✓ pass!?This bites when the script throws before reaching the relevant check() call. Force a check observation in a catch block to make the threshold see something:
try {
const r = await page.goto(URL);
check(r, { "response is 200": x => x.status() === 200 });
// ... rest of iteration body
check(true, { "script completed without exception": () => true });
} catch (e) {
console.error(e);
check(null, { "script completed without exception": () => false });
} finally {
await page.close();
}Then add 'checks{check:script completed without exception}': ['rate==1.0'] to thresholds. This catches both navigation failures AND later-stage exceptions.
abortOnFail cloud delay
When k6 runs in the cloud, thresholds are evaluated every 60 seconds. The
abortOnFailfeature may be delayed by up to 60 seconds.
For runs shorter than 60 s, abortOnFail may not fire before iterations complete naturally. The threshold is still evaluated at end-of-run and result flips to failed — abort just doesn't save execution time.
Operators
Docs show <, >, <=, >=. == and != also work even though they aren't in the docs. For rate metrics (value 0.00–1.00), rate==1.0 means "every observation was non-zero" and rate==0 means "all observations were zero."
Workflow-specific gotchas
For the canonical gotcha list (auth expiry, doubled cloud/cloud/, 415 on script PUT, Loki missing X-K6TestRun-Id, script PUT not bumping updated, gcx k6 runs list --limit 0 not following @nextLink), see k6-manage §7. The entries below are specific to this investigation workflow:
| Symptom | Cause | Fix |
|---|---|---|
| "Latest run was 6 months ago" but schedule says daily | You didn't paginate /test_runs | Use the @nextLink loop in k6-manage §3 (Step 3) |
| "Last 7 days" report contains only old runs | Filtered by row count, not date | Re-filter by created >= <iso_date> (Step 4) |
| Threshold reports ✓ pass but checks fail | Zero check observations; iteration aborted before check() ran | See "Threshold semantics" above |
result: error but the script logs/metrics look fine | Likely a platform abort (e.g. processing_metrics exceeded the 1h cap, code 8016). The execution itself completed normally. | Check status_history[*].extra.code on the run. Non-null platform code → not your code's fault. See "Threshold semantics" |
| Investigating a past run by reading the current load-test script | Script may have been edited since the run executed — what you're reading isn't what ran | GET the run's bundled script via the per-run endpoint (k6-manage §5) and diff against the current load-test script before drawing conclusions |
| Just overwrote the user's script | Invoked update-script to "see the URL" | Restore from the backup taken in Step 2. To learn URLs without writing, use -vvv --log-http-payload on any non-mutating command. |
CLI --iterations 1 breaks browser scenarios | Overrides scenario block entirely | Edit iterations: in-file with sed instead |
Reference
references/worked-example.md— a synthetic investigation with realistic findings, useful as a template for the Step 9 report.
Files
- ATTRIBUTION.mdAdded by AutoGineer889 B
- LICENSELicence file added by AutoGineer11.1 KiB
- SKILL.md14.9 KiB
- references/worked-example.md5.8 KiB
LICENSE
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Grafana Labs
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Needs: Needs Python
Security
No issues found by our static scan· No issues found
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Published by
- Grafana Labs
- Publisher
- Grafana Labs (Official)
- Source
- grafana/skills · skills/grafana-k6/k6-cloud-investigate-test · commit 8fb72a5
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by Grafana Labs or Grafana Labs.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download k6-cloud-investigate-test.zip