AWS AI ML

AWSOfficialPublished by AWS

Skill
Tool integration
Cloud and DevOps
Apache-2.0
Low risk
Has scripts

From the author

Selects, deploys, and customizes AI models on Amazon SageMaker. Training or Processing jobs, fine-tuning (SFT/DPO/RLVR/RLAIF), model selection, dataset preparation, evaluation, SageMaker or Bedrock deployment, inference optimization and endpoint diagnostics. Covers the full lifecycle from planning through production. Use when fine-tuning models on SageMaker, choosing which base model to customize, fine-tune, or deploy from SageMaker JumpStart or Hub, SageMakerPublicHub, or the SageMaker public model catalog, transforming or validating training data, evaluating model quality, deploying or optimizing endpoints, configuring IAM/S3 for training, or managing SageMaker Managed MLflow. Use for endpoint health, failures, latency, logs, metrics, errors. Covers Serverless Model Customization, Nova and OSS deployment paths, and PySDK v3. NOT for Ground Truth labeling, Feature Store, or general-purpose AWS infrastructure.

— the description by AWS

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured framework for managing the Amazon SageMaker model lifecycle. It covers planning, dataset preparation, fine-tuning, model evaluation, and endpoint deployment. The skill assists users in navigating SageMaker-specific workflows, including selecting base models from catalogs, configuring training and processing jobs, and diagnosing endpoint health. It supports various fine-tuning techniques and deployment pathways, ensuring users follow validated procedures for their machine learning projects on AWS.

What it does

Guides users through the end-to-end lifecycle of SageMaker model customization.
Assists in selecting and evaluating base models from SageMaker JumpStart and public hubs.
Provides workflows for validating and transforming training and evaluation datasets.
Supports configuration of fine-tuning techniques like SFT, DPO, RLVR, and RLAIF.
Facilitates the deployment of models to SageMaker endpoints or Bedrock.
Diagnoses endpoint issues including latency, errors, and container logs.

When to use it

When planning a new model customization or deployment project on SageMaker.
When you need to validate or transform datasets for training or evaluation.
When selecting a base model or choosing a fine-tuning technique.
When you need to configure training, processing, or evaluation jobs.
When deploying a model to a SageMaker endpoint or troubleshooting an existing one.
When setting up or managing a SageMaker Managed MLflow environment.

Advantages

Provides structured, step-by-step workflows for complex SageMaker tasks.
Ensures dataset formatting meets specific SageMaker requirements.
Offers clear routing to relevant documentation based on user intent.
Helps resolve configuration issues for IAM, S3, and SDK settings.
Simplifies the diagnosis of endpoint performance and failure logs.

Usage examples

“I need to fine-tune a model on SageMaker. Where do I start?”: The agent will initiate the planning workflow, prompting you to define your use case and requirements before guiding you through model selection and dataset preparation.
“My SageMaker endpoint is returning errors. Can you help?”: The agent will route you to the endpoint diagnostics workflow to collect logs, check instance status, and analyze latency or failure metrics.
“How do I convert my CSV training data for SageMaker?”: The agent will guide you through the dataset transformation workflow, recommending the appropriate tools to convert your data into the required JSONL format.
“Which base model should I use for a text summarization task?”: The agent will trigger the model selection workflow to help you evaluate available models and resolve the correct Hub model ID for your specific requirements.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/aws/aws-ai-ml/download
Unzip it into the cross-client skills folder of your project:
```
mkdir -p .agents/skills && unzip aws-ai-ml.zip -d .agents/skills
```

Documentation by AWS

AWS AI/ML Model Customization

Domain expertise for fine-tuning and deploying models on Amazon SageMaker. Covers the full model customization lifecycle from planning through production deployment.

Routing

Match the user's intent to the appropriate reference folder and load only that content.

User intent	Reference	When to use
Plan a model customization project, discover scope of work, resume or modify a plan	references/planning/	User's request relates to model customization or deployment (fine-tuning, training, building, customizing, reviewing data, deploying or standing up a model — including selecting or deploying an off-the-shelf or base model with no training — or getting advice on approach). Always co-activate with other intents to discover full scope. Load this reference FIRST when the request matches multiple rows in this table — read its plan templates before routing to a single-action reference.
Define the business problem, success criteria, or use case spec	references/use-case-specification/	User says "define my use case", "capture requirements", "what should I decide up front", or as default first step in any plan. Skip only if user explicitly declines.
Select or change a base model	references/model-selection/	User asks which model to use, mentions a model name or family, or wants to evaluate what's available. Always activate model-selection even for known model names because the exact Hub model ID must be resolved. Recommended: route to use-case-specification first to capture requirements — this produces better filtering results. Routing to use-case-specification first is not required if user provides a specific model name/ID or declines. If intent is ambiguous (fine-tune vs deploy as-is), model-selection MUST confirm which path before proceeding. Base model filtering for deployment MUST go through select-for-deployment.md and its scripts for any final recommendation.
Choose a fine-tuning technique (SFT, DPO, RLVR, RLAIF)	references/finetuning-technique/	User has decided to fine-tune and needs to choose a technique, or technique needs validation against the selected model's recipes. Requires a base model to be selected first.
Validate dataset quality and format	references/dataset-evaluation/	User says "is my dataset okay", "check my training data", "I have my own data", or before starting any fine-tuning job.
Transform or convert a dataset between formats	references/dataset-transformation/	User says "transform", "convert", "reformat", or dataset schema needs to change. Always use this rather than writing inline transformation code.
Generate fine-tuning code and start training	references/finetuning/	User says "start training", "fine-tune my model", "I'm ready to train", or plan reaches the finetuning step. Supports SFT, DPO, RLVR, RLAIF trainers.
Train a model with a SageMaker Training job	references/training-jobs/	Use when the goal is to fit or update model parameters and produce model artifacts or checkpoints. Covers recipes, JumpStart traditional ML, custom training scripts or containers, distributed training, and training-job diagnosis.
Process data with a SageMaker Processing job	references/processing-jobs/	Use when the goal is preprocessing, feature engineering, evaluation, transformation, or ETL without model training as the primary operation. Covers scripts, framework processors, Spark, custom containers, Athena, Redshift, and processing-job diagnosis.
Evaluate or benchmark a trained model	references/model-evaluation/	User says "evaluate my model", "run a benchmark", "test model performance", "compare models". Supports LLM-as-Judge and Custom Scorer.
Deploy, benchmark, or optimize a model on an endpoint or Bedrock	references/model-deployment/	User says "deploy my model", "create an endpoint", or "make it available" (plain deploy) — or, for the inference-optimization sub-workflows on SageMaker Real-Time Endpoints only, "benchmark my endpoint" / "compare benchmark runs" (benchmarking), or states a performance/cost/latency/throughput goal for a new deployment such as "find the cheapest instance" (recommendations). Handles Nova vs OSS deployment pathways.
Set up IAM roles, S3 buckets, SDK configuration	references/sdk-getting-started/	User says "set up", "getting started", "check my environment", "configure SDK", or as first step in any plan involving SageMaker training/evaluation/deployment.
Manage project directory and artifacts	references/directory-management/	Starting a new project, resuming existing one, or when PLAN.md needs to be associated with a project directory.
Set up, update, or delete a SageMaker Managed MLflow app	references/manage-mlflow/	User says "set up MLflow", "create MLflow app", "update my MLflow app", "delete my MLflow app", "I need an MLflow server", asks "what is SageMaker MLflow", or a workflow needs an MLflow backend and none is connected.
Diagnose a failing or unhealthy SageMaker endpoint	references/endpoint-diagnostics/	User reports endpoint errors, latency, inference failures, or a deployment that failed. "What's the status of my endpoint?", "Is my endpoint erroring?", "My endpoint failed — why?", "How many instances are running behind my endpoint?", "Is the latency my model or SageMaker?", "Show me the container logs for my endpoint." NOT for training-job issues, endpoint deletion, scaling changes, or new deployments.

Rules

Service boundary. This skill covers SageMaker, including deployment of a customized model to Bedrock. Bedrock-native work (invoking Bedrock's managed foundation models, Bedrock Agents, Knowledge Bases, Guardrails, AgentCore) is supported by AWS but has no validated workflow in this skill: let the user know and proceed with best-effort guidance using general AWS knowledge. If the user asks for a model on AWS without naming a service, compare SageMaker and Bedrock or ask one service-choice question. Do not select a service by default. The two services have different model catalogs, deployment paths, and pricing, and a default selection can send the user into an incorrect workflow.
Progressive disclosure. Load only the reference folder relevant to the current user intent. Do not load all references at once.
Best-effort help. If the user's request falls outside this skill's references, do not dead-end the conversation. Help them using general AWS knowledge and documentation, and inform the user that the guidance is not covered by this skill's validated workflows.
Usage attribution. Before running any AWS CLI command or packaged script, set export AWS_SDK_UA_APP_ID=AWSSkill-SageMaker.

Files

ATTRIBUTION.mdAdded by AutoGineer883 B
LICENSELicence file added by AutoGineer9.9 KiB
NOTICELicence file added by AutoGineer67 B
SKILL.md8.1 KiB
references/dataset-evaluation/overview.md5.4 KiB
references/dataset-evaluation/references/custom-scorer-evaluation-dataset-formats.md3.6 KiB
references/dataset-evaluation/references/strategy_data_requirements.md4.8 KiB
references/dataset-evaluation/scripts/format_detector.pyScript26.7 KiB
references/dataset-transformation/code_templates/transformation.pyScript1.5 KiB
references/dataset-transformation/overview.md13.3 KiB
references/dataset-transformation/references/code_output_guide.md2.8 KiB
references/dataset-transformation/references/dataset_transformation_code.md5.2 KiB
references/dataset-transformation/references/sagemaker_dataset_formats.md2.9 KiB
references/dataset-transformation/scripts/transformation_tools.pyScript4.8 KiB
references/directory-management/overview.md1.9 KiB
references/endpoint-diagnostics/overview.md3.5 KiB
references/endpoint-diagnostics/references/minimum_iam_policy.md1.2 KiB
references/endpoint-diagnostics/scripts/collect_diagnostics.pyScript12.8 KiB
references/finetuning-technique/overview.md2.1 KiB
references/finetuning-technique/references/finetune_technique_selection_guide.md2.3 KiB
references/finetuning-technique/scripts/get_recipes.pyScript817 B
references/finetuning/code_templates/dpo.pyScript6.4 KiB
references/finetuning/code_templates/rlaif_builtin.pyScript5.8 KiB
references/finetuning/code_templates/rlaif_custom_prompt.pyScript6.3 KiB
references/finetuning/code_templates/rlvr.pyScript6.5 KiB
references/finetuning/code_templates/sft.pyScript6.1 KiB
references/finetuning/overview.md11.4 KiB
references/finetuning/references/code_output_guide.md3.2 KiB
references/finetuning/references/continuous_customization.md8.4 KiB
references/finetuning/references/eula_links.md7.3 KiB
references/finetuning/references/rlaif_guide.md3.8 KiB
references/finetuning/references/rlvr_reward_function.md10.0 KiB
references/finetuning/scripts/mlflow_reference.pyScript841 B
references/finetuning/templates/nova_rlvr_reward_function_source_template.pyScript12.8 KiB
references/finetuning/templates/rlvr_reward_function_source_template.pyScript9.1 KiB
references/manage-mlflow/overview.md4.8 KiB
references/manage-mlflow/references/app-creation-workflow.md3.2 KiB
references/manage-mlflow/references/app-deletion-workflow.md2.3 KiB
references/manage-mlflow/references/app-discovery-workflow.md1.5 KiB
references/manage-mlflow/references/app-update-workflow.md1.7 KiB
references/manage-mlflow/references/sm-mlflow-guide.md1.3 KiB
references/model-deployment/code_templates/deploy-jumpstart-sagemaker.pyScript7.5 KiB
references/model-deployment/code_templates/deploy-nova-bedrock.pyScript2.0 KiB
references/model-deployment/code_templates/deploy-nova-sagemaker.pyScript1.9 KiB
references/model-deployment/code_templates/deploy-oss-bedrock.pyScript4.0 KiB
references/model-deployment/code_templates/deploy-oss-sagemaker.pyScript2.6 KiB
references/model-deployment/code_templates/optimize-benchmark.pyScript4.6 KiB
references/model-deployment/code_templates/optimize-huggingface-to-s3.pyScript2.0 KiB
references/model-deployment/code_templates/optimize-recommendation.pyScript6.4 KiB
references/model-deployment/overview.md16.6 KiB
references/model-deployment/references/code_output_guide.md3.2 KiB
references/model-deployment/references/deploy-jumpstart-sagemaker.md14.3 KiB
references/model-deployment/references/deploy-nova-bedrock.md4.4 KiB
references/model-deployment/references/deploy-nova-sagemaker.md5.8 KiB
references/model-deployment/references/deploy-oss-bedrock.md5.2 KiB
references/model-deployment/references/deploy-oss-sagemaker.md12.9 KiB
references/model-deployment/references/model-licenses.md11.9 KiB
references/model-deployment/references/optimization/benchmark-workflow.md12.7 KiB
references/model-deployment/references/optimization/benchmarking-guidance.md8.9 KiB
references/model-deployment/references/optimization/huggingface-to-s3.md3.1 KiB
references/model-deployment/references/optimization/interpreting-results.md8.5 KiB
references/model-deployment/references/optimization/recommendation-deploy.md1.5 KiB
references/model-deployment/references/optimization/recommendation-options.md3.1 KiB
references/model-deployment/references/optimization/recommendation-workflow.md10.3 KiB
references/model-evaluation/code_templates/custom_scorer_evaluator.pyScript2.5 KiB
references/model-evaluation/code_templates/llmaaj_evaluator.pyScript2.8 KiB
references/model-evaluation/overview.md5.2 KiB
references/model-evaluation/references/code_output_guide.md3.3 KiB
references/model-evaluation/references/create-reward-function.md3.0 KiB
references/model-evaluation/references/custom-lambda-scorer.md4.4 KiB
references/model-evaluation/references/custom-scorer-evaluation.md10.5 KiB
references/model-evaluation/references/evaluation-type-guide.md10.1 KiB
references/model-evaluation/references/llmaaj-builtin-evaluation.md5.0 KiB
references/model-evaluation/references/llmaaj-custom-evaluation.md2.5 KiB
references/model-evaluation/references/llmaaj-evaluation.md14.6 KiB
references/model-evaluation/references/supported-judge-models.md2.9 KiB
references/model-evaluation/scripts/nova_reward_function_source_template.pyScript13.1 KiB
references/model-evaluation/scripts/reward_function_source_template.pyScript9.2 KiB
references/model-evaluation/scripts/validate_custom_metrics.pyScript3.5 KiB
references/model-selection/overview.md10.3 KiB
references/model-selection/references/benchmarks/agentic-index.md2.8 KiB
references/model-selection/references/benchmarks/coding-index.md2.8 KiB
references/model-selection/references/benchmarks/gpqa.md2.8 KiB
references/model-selection/references/benchmarks/hle.md2.8 KiB
references/model-selection/references/benchmarks/ifbench.md2.8 KiB
references/model-selection/references/benchmarks/intelligence-index.md2.8 KiB
references/model-selection/references/benchmarks/mmmu-pro.md3.0 KiB
references/model-selection/references/benchmarks/tau2.md2.8 KiB
references/model-selection/references/model-licenses.md7.3 KiB
references/model-selection/references/model-selection.md8.2 KiB
references/model-selection/references/select-for-deployment.md19.8 KiB
references/model-selection/scripts/filter_deployable_models.pyScript10.1 KiB
references/model-selection/scripts/get_deployable_models.pyScript21.7 KiB
references/model-selection/scripts/get_model_names.pyScript1.4 KiB
references/model-selection/scripts/search_deployable_models.pyScript34.9 KiB
references/planning/overview.md10.3 KiB
references/planning/references/deploy-base-model-plan.md699 B
references/planning/references/evaluate-first-plan.md1.9 KiB
references/planning/references/input-output-contracts.md3.1 KiB
references/planning/references/model-customization-plan.md1.5 KiB
references/planning/references/region-availability.md1.2 KiB
references/planning/references/skill-routing-constraints.md2.7 KiB
references/processing-jobs/code_templates/dataset_definition_processing.pyScript4.1 KiB
references/processing-jobs/code_templates/framework_processing.pyScript3.0 KiB
references/processing-jobs/code_templates/script_processing.pyScript2.9 KiB
references/processing-jobs/code_templates/spark_processing.pyScript3.7 KiB
references/processing-jobs/overview.md6.7 KiB
references/processing-jobs/references/configuration.md4.6 KiB
references/processing-jobs/references/troubleshooting.md2.4 KiB
references/sdk-getting-started/overview.md643 B
references/sdk-getting-started/references/execution-role-setup.md2.4 KiB
references/sdk-getting-started/references/sagemaker-python-sdk-setup.md2.1 KiB
references/training-jobs/code_templates/custom_training.pyScript6.2 KiB
references/training-jobs/code_templates/jumpstart_training.pyScript5.6 KiB
references/training-jobs/code_templates/recipe_training.pyScript5.7 KiB
references/training-jobs/overview.md9.1 KiB
references/training-jobs/references/configuration.md4.0 KiB
references/training-jobs/references/troubleshooting.md4.4 KiB
references/use-case-specification/overview.md2.0 KiB
references/use-case-specification/references/spec-for-deployment.md4.0 KiB
references/use-case-specification/references/spec-for-finetuning.md2.1 KiB

references/model-selection/references/select-for-deployment.md

Select for Deployment

Select a base model to deploy by filtering the catalog against the user's use_case_spec.md.

Prerequisites

A use_case_spec.md file exists with a Deployment Constraints section. If not, activate the use-case-specification skill first.
Public-Hub Search requires the boto3 and botocore versions enforced by search_deployable_models.py's MINIMUM_SDK_VERSION. SageMaker PySDK v3 does not expose Search(Resource="HubContent"), so the public path intentionally uses the version-gated boto3 client. An older client triggers the visible List-and-filter fallback described in Step 3.

Step 1: Discover Available Filter Values

Branch on the selected Hub.

SageMakerPublicHub

Create one unique, short-lived catalog snapshot while discovering values:

python3 model-selection/scripts/get_deployable_models.py \
    SageMakerPublicHub --list-values --create-snapshot <region>

The command creates a unique private temporary file and returns both available_values and snapshot_file in one JSON object. It removes the file automatically if snapshot creation fails. Record the exact snapshot_file value in conversation context; later commands MUST use that literal path.

The snapshot is a per-selection-session cache of compact, normalized ListHubContents summaries. It provides current exact keyword spellings, raw size values, and OriginalCreationTime without a second catalog call after confirmation. It does not contain credentials, model documents, or markdown. Reuse it only while the Hub and region remain unchanged, and delete it as described under Snapshot cleanup.

Private Hub

Run the existing discovery command without a snapshot:

python3 model-selection/scripts/get_deployable_models.py <hub-name> --list-values <region>

Both commands return a JSON object showing all unique values available for each filterable field. For example:

{
  "tasks": ["text generation", "image classification", "translation", ...],
  "sizes": ["<=10b", "11b-70b", "71b-100b", ">100b", "unknown"],
  "data_types": ["text", "vision", "audio", "multimodal", "tabular"],
  ...
}

Step 2: Map Soft Constraints to Hard Filters

Read use_case_spec.md and extract the Deployment Constraints section. These are natural language preferences from the user.

ENFORCEMENT — read before mapping. Apply these rules in order:
Use ONLY the tables below. Do not infer filter values from your own knowledge of model sizes, tasks, or context lengths, and do not substitute "close enough" values. The tables are the single source of truth, so the SAME user words always produce the SAME filter set every run.
Unmapped term → ASK. If a constraint term is not listed below, ask the user a clarifying question rather than guessing.
Ambiguous term → offer options. If a term is ambiguous (e.g. "qa" = question answering or quality assurance), ask and offer the mapped categories to choose from (e.g. "task options: chatbot/assistant, summarization, classification, translation — which fits?").
"any" / "don't care" → no filter for that field.
Pass each value in a cell as its own argument with the same key; the filter OR-s values within a key.

2a. Size — soft term → exact `size` value

get_deployable_models.py normalizes every model into one of five canonical size buckets (<=10b / 11b-70b / 71b-100b / >100b / unknown). The four below are the selectable values; unknown means the model's size could not be determined — it is excluded by any size filter and has no soft-term mapping.

If the user says…	Use exactly
small, tiny, lightweight, cheap, low-cost, smallest	`size:<=10b`
medium, mid-size, moderate	`size:11b-70b`
large	`size:71b-100b`
very large, huge, biggest, largest	`size:>100b`

2b. Task — soft term → exact `task` value(s)

Task tags in the hub overlap and are matched by substring, so use these exact OR-lists (do not shorten them — e.g. bare task:classification wrongly matches image/tabular/token classification).

If the user wants…	Use exactly
chatbot, chat, assistant, conversational, customer support	`task:text generation` `task:generation-text`
summarize, summarization, summaries	`task:text generation` `task:generation-text`
classify, classification, categorize	`task:text classification` `task:classification-text`
translate, translation	`task:translation`
code, coding, programming	`task:text generation` `task:generation-text` — then tell the user that code-focused models are not separately tagged; to find them, look for the `codellama`, `qwen-coder`, `starcoder`, or `granite-code` families by name

Note on summarization: the hub's dedicated text summarization / summarization-text tags are almost entirely older short-context (≤4k) seq2seq models (BART/Pegasus/DistilBART) that mostly lack a context_window value — so filtering on them (especially combined with a long-context constraint) returns few or zero usable models. Capable long-document summarizers are general instruction-tuned LLMs tagged text generation / generation-text, which is why "summarize" maps there.

2c. Context window — soft term → exact `context_window` value(s)

If the user says…	Use exactly
short, small context	`context_window:<4k` `context_window:4k-8k`
medium context	`context_window:8k-32k`
long, long documents, long context	`context_window:32k-128k` `context_window:>128k`
very long, largest context	`context_window:>128k`

Unlike size, context_window is NOT normalized by the script — it is stored and matched verbatim, and the table above assumes the hub's buckets are exactly <4k / 4k-8k / 8k-32k / 32k-128k / >128k. Before applying a context_window filter, confirm the value(s) appear in the context_windows list from the Step 1 --list-values output. If the hub's buckets differ (e.g. a new value like 16k), use the matching value(s) from that list instead of a value from this table — do NOT filter on a bucket that isn't in the catalog, or the result will be silently empty.

2d. Other fields — exact mappings

Field	If the user says…	Use exactly
Data type	text	`data_type:text`
	image, vision	`data_type:vision`
	audio, speech	`data_type:audio`
	multimodal	`data_type:multimodal`
	tabular	`data_type:tabular`
Model type	open source, open weights	`model_type:open_weights`
	proprietary, closed	`model_type:proprietary`
Deployment target	Bedrock	`bedrock:true`
	SageMaker, either, unspecified	(no filter)

2e. Concrete fields — user names the value directly

For license, language, provider, model name, and modalities, do not use a lookup table — the user names a concrete value. Match it against the Step 1 --list-values output using substring matching. The public Search adapter repeats this lookup against the session snapshot and sends every matching exact value; for example, license:apache searches both @license:Apache 2.0 and @license:Apache-2.0. This explicit expansion is required until Hub license metadata uses one standardized spelling. If the named value is absent from the catalog, tell the user rather than substituting a different one.

The filter keys you pass are the same for every field. Internally the adapter sends data_type, input_modality, output_modality, model_type, bedrock, size, and context_window as exact Metadata.* filters over the model document, and the other keys as HubContentSearchKeywords tags. A size or context_window filter can return a few models the private-Hub path would not: the Hub writes at most 50 keywords per model, and a model with many languages can run out of slots before its size or context window is written. The document still carries the value, so Search finds the model and the script fills in size_raw and context_window from the document. Present those models exactly like the others.

General rules:

If the user said "any" or "don't care" for a field, do not create a filter for it.
If a term is not covered by the tables above, is ambiguous (like "qa"), or is not a concrete value the user named, ASK a clarifying question — and list the relevant mapped options so the user can choose (for tasks: chatbot/assistant, summarization, classification, translation; for size: small, medium, large, very large). Do not guess.

Present your resolved mapping to the user for confirmation before filtering:

"Based on your preferences, I'll filter with these criteria: [list each mapping]. Does this look right?"

⏸ Wait for user confirmation. If they disagree, adjust — but only to other values that exist in the tables above or the Step 1 catalog values.

Step 3: Retrieve and Filter Candidates

Branch on the selected Hub after the user confirms the complete filter mapping.

SageMakerPublicHub: Search

Run Search with the literal snapshot path recorded in Step 1:

python3 model-selection/scripts/search_deployable_models.py \
    SageMakerPublicHub --region <region> \
    --snapshot-file <recorded-snapshot-path> \
    <filter1> <filter2> ...

The adapter uses Search for candidate membership, restores each result's cached OriginalCreationTime, and applies the existing deterministic order. It never substitutes Search CreationTime, because that timestamp belongs to the indexed version and can move when a new version becomes latestSupported.

Inspect these output fields before presenting candidates:

retrieval_backend: "search" means Search returned the candidate set.
retrieval_backend: "list_fallback" means an explicit SDK, permission, service, transport, or Search-response failure caused the adapter to apply the complete confirmed filter set to the cached List catalog. The adapter performs this fallback internally from the existing snapshot. Do not call get_deployable_models.py or filter_deployable_models.py after a Search failure, and do not make another List request. Tell the user: "SageMaker Search was unavailable, so I used the existing complete Hub catalog filtering path for this selection."
A successful Search with matched: 0 is a real empty result and does not invoke fallback.
Present every entry in warnings. In particular, if a Search result lacks cached OriginalCreationTime, explain that it was placed after dated models and ordered by name.

Never merge Search and List results. One invocation uses either the Search result or the complete local fallback.

Private Hub: existing local filtering

Run the unchanged List-and-filter path:

python3 model-selection/scripts/get_deployable_models.py \
    <hub-name> <region> > /tmp/deployable_models.json

python3 model-selection/scripts/filter_deployable_models.py \
    /tmp/deployable_models.json <filter1> <filter2> ...

For either Hub, report:

"Based on your constraints, I applied these filters: [list each]. This narrowed [total_models] models down to [matched] candidates."

Step 4: Handle Results

If results are non-empty (≤20 models)

The candidate script returns models in a fixed, deterministic order: newest first (by original creation date, descending), with model name (A→Z) breaking ties and any models lacking a creation date placed last (also A→Z). Present the models in exactly the order the script returned them — do NOT re-sort, re-group, or reorder by size, name, license, or relevance. This guarantees the same constraints always produce the same ordered list.

Present all matching models. Cross-reference with references/model-licenses.md for license links where available. Display each model on its own line, preserving script order:

<model name> — <size_raw> | <license> | Bedrock: ✓/✗

For the size column use the model's size_raw field (the actual parameter count, e.g. 7b) when present — it is more informative than the normalized size bucket. Fall back to size only if size_raw is absent. Omit fields that are unknown rather than showing blanks. Display every matching model — completeness is required.

Ask the user to select:

"Which model would you like to deploy?"

⏸ Wait for user selection. Once the user selects a model, proceed immediately to Step 5 (do not confirm or hand off yet).

If results are non-empty but large (>20 models)

Show the first 20 (the 20 newest, since the script returns newest-first — preserve script order), state how many total matched, and ask:

"There are [matched] models matching your constraints. Here are the first 20. Would you like to see more, or would you like to tighten your constraints to narrow further?"

If results are empty

Re-run the active Hub's candidate command with one constraint removed at a time to identify which filter is most restrictive. For SageMakerPublicHub, reuse the same snapshot and rerun search_deployable_models.py; for a private Hub, rerun filter_deployable_models.py.

Do not treat a successful zero-result Search as an availability failure and do not switch to List merely because the result is empty. Report:

"No models matched all your constraints. The most restrictive filter was [field]: [value], which eliminated [N] candidates. Would you like to relax this constraint?"

If the user agrees, adjust the filter and re-run from Step 3. If they want to change their use case spec, activate the use-case-specification skill, then re-run from Step 1.

Snapshot cleanup

For SageMakerPublicHub, keep the snapshot while the user may refine filters or ask to see more. When the user selects a model, cancels selection, changes Hub or region, or the workflow ends with an unrecoverable error, delete only the exact temporary file created in Step 1:

rm -f -- "<recorded-snapshot-path>"

If the Hub or region changes, delete the old snapshot and create a new one before filtering. The snapshot is selection-session state, not a shared or long-lived cache.

Step 5A: Select Instance Type

Once the user selects a model, use the AWS API describe-hub-content (SageMaker service) with:

HubName: the hub name from Step 1
HubContentType: "Model"
HubContentName: the selected model name

From the response, parse the HubContentDocument JSON and determine the instance type:

If `InferenceConfigs` exists (labeled configs):

The model has pre-configured hosting profiles optimized for different use cases. Each config name follows the pattern <use-case>_<optimization>, e.g.:

generate_best_price_performance — balanced cost/speed for text generation
summarize_lowest_latency — fastest for summarization tasks
interact_lowest_cost — cheapest for interactive/chatbot use
max_context_best_price_performance — best value for long-context inputs

Each config includes BenchmarkMetrics with latency and throughput per instance type.

Recommend a config based on the user's use case from use_case_spec.md:

Match the use-case prefix (generate, summarize, interact, max_context, modify) to the user's task.
Default to best_price_performance optimization unless the user expressed a preference for speed (lowest_latency) or cost (lowest_cost).
Use the DefaultInferenceInstanceType from the matched config component as the instance type.
Present your recommendation with the benchmark metrics:

"For your use case ([task]), I recommend the [config_name] hosting configuration on [instance_type] ([latency] latency, [throughput] throughput). There are [N] other configurations available if you'd like to optimize differently. Would you like to proceed with this, or see other options?"

⏸ Wait for user confirmation or selection.

If `InferenceConfigs` does NOT exist (no labeled configs):

The model has basic hosting info only. Collect ALL available instance types from both sources:

SupportedInferenceInstanceTypes — the top-level list
HostingInstanceTypeVariants.Variants — keys in this object are instance families or specific instance types (e.g., g5, ml.g5.2xlarge) that also support the model

Merge both into a single candidate set (union, deduplicated). The HostingInstanceTypeVariants often contains smaller/cheaper instances not listed in SupportedInferenceInstanceTypes.

From the merged set, select the cheapest instance type as the default recommendation. Consider both:

Instance family (cost tier): g4dn < g5 ≈ g6 < g6e < p4d < p5
Instance size (suffix): xlarge < 2xlarge < 4xlarge < 12xlarge < 24xlarge < 48xlarge

Pick the cheapest family available, then the smallest size within that family.

Important: A hosting config entry (with ResourceRequirements, ImageUri, etc.) is NOT required to recommend an instance. If an instance family appears in HostingInstanceTypeVariants with just an ImageUri, that is sufficient — the model can run on it. Do not bias toward instances that have a more detailed config. Always recommend the cheapest option regardless of whether it has a full config or just an image URI.

Present:

"For this model, I recommend starting with [cheapest_instance] (the most cost-effective supported instance). Other supported options from smallest to largest: [list]. Would you like to proceed with this, or choose a different instance?"

⏸ Wait for user confirmation.

Step 5B: Resolve Hosting Configuration

⚠️ Do NOT proceed to Step 6 until you have completed this step. The user confirming an instance type does NOT mean the step is done.

Now that the instance type is confirmed, resolve the hosting configuration:

Check RecipeCollection entries in the HubContentDocument. Within each recipe, look at HostingConfigs — an array of objects keyed by InstanceType.
Find an entry where InstanceType matches the confirmed instance (e.g., ml.g5.2xlarge).
If a match exists, it contains deployment-specific settings (EcrAddress, Environment, ComputeResourceRequirements, Profile). Record this as the hosting config.
If no match in RecipeCollection, check HostingInstanceTypeVariants.Variants for the instance family (e.g., g5). This may contain ImageUri or environment variable overrides.
If neither has a config for the selected instance, record that no hosting config was found.

Step 6: Hand Off

Emit a flat deployment config with exactly these three fields:

[ ] model_id — the selected Hub model ID.
[ ] instance_type — the user-confirmed instance type.
[ ] inference_config_name — the config name the user confirmed in Step 5A (the one you recommended, or a different one they chose), e.g. generate_best_price_performance, so deployment deploys THAT config instead of the SDK's top-ranked default. This is required whenever the model has labeled InferenceConfigs: because the instance type was derived from this config, omitting the name lets deployment pick a different (top-ranked) config for which the chosen instance may be unsupported. If the user switched to a different config, make sure the instance_type you hand off is that config's DefaultInferenceInstanceType (re-derive it), so the pair stays consistent. Set it to null ONLY on the Step 5A "no labeled configs" path.

Do NOT hand off the image URI / environment / ComputeResourceRequirements resolved in Step 5B — the serving container and environment are re-resolved from inference_config_name, so Step 5B stays a validation step (confirm a hosting config exists for the chosen instance) and its output is not part of the hand-off. Do not emit the role ARN, region, or endpoint/model naming either — those are set at deployment time, not here.

If any of the three fields is missing, go back and resolve it before proceeding.

Return to Step 4 of the main SKILL.md (Confirm Selection) with these outputs.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs Python

Security

What our scanner noticed· Low risk

Instructions that use the network
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

AWS

Publisher

AWS (Official)

Source

aws/agent-toolkit-for-aws · skills/core-skills/aws-ai-ml · commit 0a218f8

Licence

Apache License 2.0 (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
NOTICE (added by AutoGineer from NOTICE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

Also distributed in

aws/agent-toolkit-for-aws

AutoGineer is not affiliated with or endorsed by AWS or AWS.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download aws-ai-ml.zip