Elasticsearch Anomaly Detection Explainer
ElasticOfficialCreated by elastic
- Skill
- Tool integration
- Data and AI
- Apache-2.0
- No issues found
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a structured approach to interpreting Elasticsearch machine learning anomaly detection results. It helps users understand why specific anomaly scores were generated, how models behave, and how to troubleshoot unexpected results. By leveraging the ML REST API and standard search queries against anomaly result indices, the skill explains score renormalization, identifies contributing factors, and clarifies model bounds without requiring direct access to complex internal APIs.
What it does
- Explains the difference between initial_record_score and record_score due to renormalization.
- Identifies the root cause of anomalies using influencer analysis and score explanation factors.
- Visualizes model behavior by comparing actual values against learned upper and lower bounds.
- Interprets specific detector functions like low_count or high_mean to distinguish between outages and spikes.
- Retrieves and summarizes job configuration and health statistics to provide context for anomaly scores.
- Categorizes log patterns for jobs using categorization fields.
When to use it
- When you need to understand why an anomaly score is high or low.
- When investigating whether a high score indicates a genuine incident or a false positive.
- When you need to explain why a score changed over time due to renormalization.
- When troubleshooting unexpected anomaly results or model behavior.
- When identifying which entities (e.g., hosts, users) are the primary contributors to an anomaly.
Advantages
- Provides clear, evidence-based explanations using actual record data.
- Distinguishes between expected model behavior and potential configuration issues.
- Uses standardized search queries compatible with both self-managed clusters and Elastic Serverless.
- Reduces manual effort in correlating job configurations with anomaly records.
Usage examples
- “Why is my anomaly score only 15 when the spike looks huge?”
- The agent retrieves the record, identifies that initial_record_score was 92, and explains that renormalization occurred because a more extreme anomaly appeared later, which is healthy model behavior.
- “Traffic stopped and I got a high score. Is this a false positive?”
- The agent checks the detector function, confirms it is a low_count job, and explains that the high score is a legitimate detection of an absence anomaly (outage) rather than a false positive.
- “Which entities contributed most to the anomalies in my job last night?”
- The agent queries the influencer documents for the specified time range and returns a list of entities sorted by their influencer_score.
- “Show me the model bounds for this job.”
- The agent queries the model_plot results and provides a comparison of the actual values against the model_lower and model_upper bounds.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/elastic/elasticsearch-anomaly-detection-explainer/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip elasticsearch-anomaly-detection-explainer.zip -d .agents/skills
Documentation by elastic
Anomaly Detection Score Explainer
Explain anomaly scores, model behavior, and why results look the way they do. Use the ML REST API for job config and
the standard _search API against .ml-anomalies-* for results — no ES|QL, fully compatible with Elastic
Serverless. For job lifecycle (create, start, stop), use the elasticsearch-anomaly-detection skill.
<!-- begin-partial: preamble -->
Environment Configuration
This skill executes Elasticsearch operations through the elastic CLI. If the
elastic CLI is not installed, tell the user what it is needed for. Do
not guess credentials, call the HTTP API directly, or attempt other workarounds.
This skill references operations in HTTP-shorthand form (e.g., GET /, GET /_cat/indices, GET /{index}/_mapping,
GET /{index}/_settings/index.mode, POST /_query). The Operations table at the end of this document
maps each shorthand to the equivalent elastic CLI command — always use the CLI rather than calling the HTTP API
directly.
<!-- end-partial: preamble -->
Prerequisite: ML anomaly detection requires a Platinum-equivalent license on self-managed clusters. Serverless projects include ML. The caller needs
monitor_mlto read job config and anomaly results.Serverless note: The
_ml/.../results/*REST endpoints return HTTP 410 in Elastic Serverless. Always usePOST /.ml-anomalies-*/_searchfor result queries instead — fully supported everywhere this skill runs.
Process
Decide whether to fetch data or interpret what the user supplied. If the user embeds an anomaly record (or job config) in the prompt, interpret it directly using the domain knowledge below — do not call APIs to re-fetch fields already present. If the job ID, time range, or record is missing, retrieve it from the cluster.
The decision: proceed with judgment-only explanation when the record contains
record_score,initial_record_score,actual,typical, andfunction; otherwise fetch the missing pieces before explaining.Verify connectivity when calling the cluster. Call
GET /. If the call fails, stop and surface the connection error — do not guess endpoints or credentials.Resolve the job ID and load config. When the job ID is unknown, call
GET /_ml/anomaly_detectorsto list candidates. CallGET /_ml/anomaly_detectors/{job_id}for fullanalysis_config(bucket_span, detectors, custom_rules, use_null, model_plot_config) andGET /_ml/anomaly_detectors/{job_id}/_statsforstate,model_size_stats.memory_status, and data counts.The decision: confirm detector function and direction match the user's question before interpreting scores. A
low_countjob legitimately fires on drops; ahigh_countjob does not.Retrieve anomaly records for the time range. Call
POST /.ml-anomalies-*/_searchwithresult_type: record, the job ID, a timestamp range, and optionalrecord_scorefilter. Readinitial_record_score,record_score,actual,typical,function,multi_bucket_impact, andanomaly_score_explanation.Always show both
initial_record_scoreandrecord_score. The gap is the renormalization story.Classify the score pattern before speculating on causes.
initial_record_score>>record_score— Renormalization. A later, more extreme anomaly rescale this record downward. This is expected, healthy model behavior — not a broken model or reason to distrust the detection. Useinitial_record_scorefor alerting severity; show both scores and explain the gap explicitly.initial_record_score==record_score— No renormalization has occurred since detection.actual<<typicalwithlow_count,count, orlow_mean— Absence / drop anomaly. A high score is legitimate — the job detected an outage, pipeline stall, or service failure. This is not a false positive. Recommend incident investigation, not score tuning.actual>>typicalwithhigh_countorhigh_mean— Spike anomaly; confirm withsingle_bucket_impact.
Only cite
anomaly_score_explanationfactors present in the record. Ifhigh_variance_penaltyisfalse, do not blame variance. If a factor is absent, note that it was not returned — do not invent it.Quantify renormalization across the job (optional). Re-query
POST /.ml-anomalies-*/_searchfor records in the time range sorted bytimestampascending. Computescore_drift = initial_record_score − record_scoreper record and filter to|score_drift| ≥ 20. Large negative drift (initial >> record) confirms renormalization after a more extreme anomaly appeared later.Add context when the user asks "what caused this?" or "why so low/high?"
Model bounds — If
model_plot_config.enabledis true, callPOST /.ml-anomalies-*/_searchwithresult_type: model_plotfor the same job and time range. Compareactualtomodel_lower/model_upper.Influencers — Call
POST /.ml-anomalies-*/_searchwithresult_type: influencerfor the bucket time range; sort byinfluencer_scoredescending.Categorization jobs — Call
POST /.ml-anomalies-*/_searchwithresult_type: category_definitionto list learned log patterns (terms,regex,examplespercategory_id).
For aggregations, cross-job queries, bucket-level results, or custom filters beyond score and time, see references/explainer-reference.md.
Common multi-step workflows
| Task | Steps (in order) |
|---|---|
| Explain a specific anomaly | job config → records (job + exact time) → show initial_record_score vs record_score + score factors. |
| Why is my score low? | job config → records → renormalization check → model plot (if enabled) → explain score factors. |
| Why is my score high? | job config → records → check function direction, insufficient history, use_null, cardinality. |
| Renormalization drift | records (timestamp sort) → compute score_drift → list records where initial >> record. |
| Which entities contributed? | influencers (job + time range) → sort by influencer_score. |
| Visualize model bounds | model plot (job + time range) → compare model_lower/model_upper vs actual. |
| Categorization job patterns | category_definition (job_id) → terms, regex, examples per category. |
Critical principles
Retrieve the record first (or use the one the user supplied). Never explain scores without
initial_record_score,record_score,actual,typical, andfunction.Renormalization is healthy. When
initial_record_score >> record_score, a more extreme anomaly appeared later and lowered this score — expected behavior, not a model failure.Direction matters.
low_countfires when values drop;high_countfires on spikes. A high score on a traffic stop withlow_countis correct detection, not a false positive.Explain factors before speculating. Read
anomaly_score_explanationfrom the record. Only address factors that are present and relevant.Job config is essential.
bucket_span, detector function,custom_rules,use_null, and memory status all affect scores. Inspect job config when a score is surprising.Model plot is the most visual explanation. When enabled, show model bounds to illustrate where the actual value falls relative to the expected range.
For job health ("missing documents", "memory limit", "datafeed not running") use the
elasticsearch-anomaly-detectionskill.
Domain knowledge
Score types
| Term | Meaning |
|---|---|
| record_score | Normalized 0–100 for a single anomaly record; updated by renormalization. >75 critical. |
| initial_record_score | Score assigned at detection time, before renormalization. Use for alerting. |
| anomaly_score | Bucket-level severity aggregated across all detectors in a job. |
| influencer_score | How unusual a specific entity (host, user, service) is in a bucket; high = likely cause. |
| multi_bucket_impact | 0–5; how much sustained, multi-bucket behavior raised the score. ≥3 = behavioral shift. |
anomaly_score_explanation factors
The anomaly_score_explanation field on each record breaks the score into components:
| Factor | Direction | Meaning |
|---|---|---|
| anomaly_length | Raises | Number of consecutive buckets the anomaly spans. Longer → higher score. |
| single_bucket_impact | Raises | Extremity of this single bucket. Lower probability → higher impact. |
| multi_bucket_impact | Raises | Contribution of sustained multi-bucket pattern. |
| anomaly_characteristics_impact | Raises | Whether the anomaly is a mean shift vs. variance change. |
| high_variance_penalty | Lowers | Noisy data or early training → wide confidence bounds → score reduced. |
| incomplete_bucket_penalty | Lowers | Bucket had less data than expected (delayed data, sparse events). |
Why a score might be unexpectedly low
high_variance_penalty: The metric is historically noisy — wide confidence bounds absorb the spike.
Renormalization: A more extreme anomaly appeared later and pushed this score down (
initial_record_score>>record_score).Insufficient training history: Need ≥3 weeks for weekly seasonality, ≥2 full cycles for any detected period.
bucket_span too large: Short-duration spikes get smoothed. Use a smaller
bucket_spanfor high-frequency events.Detector function mismatch:
meanvshigh_mean,countvshigh_count— only one direction fires.incomplete_bucket_penalty: Bucket received less data than expected (ingest latency or gaps).
custom_rules: A detector filter may be suppressing the anomaly.
Why a score might be unexpectedly high
Insufficient history: Model hasn't learned the normal pattern yet — early anomalies are unreliable.
Model split thin: High-cardinality
partition_fieldorby_field→ very few points per entity → unreliable probabilities.use_null: If
use_null: true, missing entities produce "null" anomalies that may not be meaningful.Absence / drop detection: With
low_countorlow_mean,actual << typicalproduces a legitimately high score — treat as a real incident, not a false positive.
Model behavior concepts
| Concept | Meaning |
|---|---|
| actual | Observed value. typical is what the model expected. The direction matters. |
| Absence anomaly | actual << typical with count, low_count, or low_mean → outage, pipeline stop, service failure. |
| by_field | Independent baseline per entity (e.g., per host). Each entity compared to its own history. |
| over_field | Population analysis — entity compared to its peer group in the same bucket, not its own history. |
| partition_field | Fully independent sub-models with separate score normalization per partition. |
Model plot and categories
Model plot: Shows the model's learned upper and lower bounds at each time point. If
actualis within bounds, no anomaly; if outside, the score depends on the distance from bounds. Only available whenmodel_plot_configis enabled on the job. Query viaPOST /.ml-anomalies-*/_searchwithresult_type: model_plot.Categories: For jobs with a
categorization_field_name, queryresult_type: category_definitionto show log message patterns (terms, regex, examples percategory_id). Anomaly records useby_field_value = <category_id>.
Score troubleshooting protocol
List jobs — Call
GET /_ml/anomaly_detectorswhen the job ID is unknown.Get job config and stats — Call
GET /_ml/anomaly_detectors/{job_id}andGET /_ml/anomaly_detectors/{job_id}/_stats. Verifybucket_span, detector function,custom_rules,use_null, job state (opened/closed/failed), andmodel_size_stats.memory_status.Retrieve the record — Call
POST /.ml-anomalies-*/_searchwithresult_type: record, the job ID, time range, and optional minimumrecord_score. Inspectinitial_record_score,record_score,actual,typical,function,multi_bucket_impact, andanomaly_score_explanation.Check renormalization — Compare
initial_record_scorevsrecord_score. If initial >> record, re-query records sorted by timestamp and computescore_driftto quantify renormalization across the job.Visualize model bounds — If
model_plot_configis enabled, queryresult_type: model_plotand show where the actual value fell relative tomodel_lowerandmodel_upper.Influencers — Query
result_type: influencerfor the anomaly bucket time range; sort byinfluencer_score.Explain factors — From the record's
anomaly_score_explanation, address each present relevant factor:high_variance_penalty,incomplete_bucket_penalty,anomaly_length,single_bucket_impact,multi_bucket_impact. Do not cite factors absent from the record.
Examples
"Why is my anomaly score only 15 when the spike looks huge?" → Check renormalization:
initial_record_score(~92) >>record_score(~15). The spike was real; the current score was rescaled down. Use the initial score for alerting."Traffic stopped and I got a HIGH score — false positive?" → No.
low_countwithactualfar belowtypicalis legitimate absence detection. Investigate the outage."Which entities contributed most to the anomalies in job X last night?" → Query influencers for the time range.
"Show me the model bounds for this job." → Query model plot when
model_plot_configis enabled."List records where the score was renormalized down a lot." → Records sorted by timestamp; filter large
initial_record_score − record_score.
Guidelines
Report only what the API or the user-supplied record contains; do not invent scores, timestamps, entity values, or explanation factors.
Always show both
initial_record_scoreandrecord_scorewhen explaining a record; state explicitly whether renormalization occurred.When a score factor is missing from the record, do not assert it; note that the field was not returned.
Do not attribute low scores to
high_variance_penaltyorincomplete_bucket_penaltywhen those flags arefalseor absent in the record.For investigation ("what caused this?", "which service is responsible?") query influencers or construct cross-job searches via references/explainer-reference.md.
For job health use the
elasticsearch-anomaly-detectionskill.
Operations
| HTTP API (shorthand) | elastic CLI command |
|---|---|
GET / | elastic es info |
GET /_ml/anomaly_detectors | elastic es ml get-jobs |
GET /_ml/anomaly_detectors/{job_id} | elastic es ml get-jobs --job-id '<job_id>' |
GET /_ml/anomaly_detectors/{job_id}/_stats | elastic es ml get-job-stats --job-id '<job_id>' |
POST /.ml-anomalies-*/_search | elastic es search --index '.ml-anomalies-*' --input-file '<search-body.json>' |
Query body shapes for each result_type (record, influencer, model_plot, category_definition) are documented in
references/explainer-reference.md.
Files
- ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
- LICENSELicence file added by AutoGineer11.1 KiB
- SKILL.md17.5 KiB
- references/explainer-reference.md6.7 KiB
LICENSE
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2026 Elasticsearch BV
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
The author says: Requires Elasticsearch 8.x+ or Elastic Cloud Serverless with ML anomaly detection. Uses the ML REST API and the standard _search API against .ml-anomalies-* — no ES|QL. User needs monitor_ml privilege for ML APIs.
Security
No issues found by our static scan· No issues found
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- elastic
- Publisher
- Elastic (Official)
- Source
- elastic/agent-skills · skills/elasticsearch/elasticsearch-anomaly-detection-explainer · commit 09d879d
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
- Also distributed in
- elastic/agent-skills
AutoGineer is not affiliated with or endorsed by elastic or Elastic.
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download elasticsearch-anomaly-detection-explainer.zip