Compose UI Control

JetBrainsOfficialCreated by ForketyforkDistributed by JetBrains

Skill
Tool integration
Software development
MIT
Low risk
Has scripts

From the author

Control a running Compose Desktop application via HTTP. Use when you need to interact with UI elements, click buttons, enter text, wait for elements to appear, or capture screenshots in a Compose Desktop app that has compose-ui-test-server enabled.

— the description by Forketyfork

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The compose-ui-control skill provides a mechanism to interact with running Compose Desktop applications through an HTTP server. By integrating the library into a project, developers can enable an agent to perform UI operations such as clicking buttons, entering text, waiting for specific elements, and capturing screenshots. This allows for programmatic control of the application interface, facilitating automated testing or remote interaction workflows.

What it does

Enables HTTP-based control of Compose Desktop applications
Allows clicking UI elements via test tags or display text
Supports text input into fields with URL-encoded values
Provides synchronization primitives like waiting for elements or UI idle states
Captures screenshots of the application window
Integrates into existing Gradle-based Compose projects

When to use it

When you need to automate interactions with a Compose Desktop application
When building test suites that require runtime UI manipulation
When an AI agent needs to verify application state through visual or structural checks
When you need to drive an application's UI from an external script or agent

Advantages

Decouples UI interaction from the application's internal code
Uses standard HTTP requests for broad compatibility
Supports stable testing patterns with wait-for-idle and timeout mechanisms
Easy to integrate into existing Compose Desktop projects

Usage examples

“Click the login button in my Compose Desktop app.”: The agent sends a GET request to /onNodeWithTag/login_button/performClick to trigger the button action.
“Enter 'admin' into the username field.”: The agent sends a GET request to /onNodeWithTag/username_field/performTextInput?text=admin to populate the text field.
“Wait for the dashboard to load and take a screenshot.”: The agent calls /waitUntilExactlyOneExists/tag/dashboard?timeout=10000 followed by /captureScreenshot?path=dashboard.png.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/jetbrains/compose-ui-control/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip compose-ui-control.zip -d .agents/skills

Documentation by Forketyfork

compose-ui-test-server

A library that enables AI coding agents to control Compose Desktop applications at runtime via HTTP.

Repository: https://github.com/forketyfork/compose-ui-test-server
Maven Central: io.github.forketyfork:compose-ui-test-server
Current version: 0.2.0

Checking If Already Installed

Before setting up, check if the project already has the library:

grep -r "compose-ui-test-server\|composeuittest" --include="*.gradle*" --include="*.kt" .

Look for:

Dependency on io.github.forketyfork:compose-ui-test-server
Imports from io.github.forketyfork.composeuittest

If found, skip to Starting the Application.

Installing in a Compose Desktop Project

Step 1: Add the dependency

Find the app's build.gradle.kts and locate the desktop source set dependencies. Add compose-ui-test-server and compose.uiTest:

kotlin {
    sourceSets {
        val desktopMain by getting {
            dependencies {
                // Existing dependencies...

                // Add these two:
                implementation("io.github.forketyfork:compose-ui-test-server:0.2.0")
                @OptIn(org.jetbrains.compose.ExperimentalComposeLibrary::class)
                implementation(compose.uiTest)
            }
        }
    }
}

For projects using version catalogs, add to gradle/libs.versions.toml:

[libraries]
compose-ui-test-server = { module = "io.github.forketyfork:compose-ui-test-server", version = "0.2.0" }

Then reference in build.gradle.kts:

implementation(libs.compose.ui.test.server)

Step 2: Update the main function

Find the application's main() function (usually in Main.kt or similar). Replace the standard Compose Desktop launcher with runApplication:

Before (typical Compose Desktop main):

import androidx.compose.ui.window.Window
import androidx.compose.ui.window.application

fun main() = application {
    Window(onCloseRequest = ::exitApplication, title = "My App") {
        App()
    }
}

After (with agent control support):

import io.github.forketyfork.composeuittest.WindowConfig
import io.github.forketyfork.composeuittest.runApplication

fun main() =
    runApplication(
        windowConfig = WindowConfig(
            title = "My App",
            minimumWidth = 1024,
            minimumHeight = 768,
        ),
    ) {
        App()
    }

The app now runs normally by default, but supports agent control when launched with COMPOSE_UI_TEST_SERVER_ENABLED=true.

Step 3: Add test tags to UI elements

For agents to interact with specific UI elements, add test tags:

import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.testTag

Button(
    onClick = { /* ... */ },
    modifier = Modifier.testTag("login_button")
) {
    Text("Login")
}

TextField(
    value = username,
    onValueChange = { username = it },
    modifier = Modifier.testTag("username_field")
)

Starting the Application

# Normal mode (no server)
./gradlew run

# Agent-controlled mode (server enabled)
COMPOSE_UI_TEST_SERVER_ENABLED=true ./gradlew run

# With custom port (default is 54345)
COMPOSE_UI_TEST_SERVER_ENABLED=true COMPOSE_UI_TEST_SERVER_PORT=8080 ./gradlew run

Verifying the Server

Always check health first:

curl http://localhost:54345/health
# Expected: "OK"

Available Endpoints

Endpoint	Description
`GET /health`	Health check
`GET /onNodeWithTag/{tag}/performClick`	Click element by test tag
`GET /onNodeWithTag/{tag}/performTextInput?text=...`	Enter text (URL-encode the text!)
`GET /onNodeWithText/{text}/performClick`	Click element by display text
`GET /waitUntilExactlyOneExists/tag/{tag}?timeout=5000`	Wait for element by tag
`GET /waitUntilExactlyOneExists/text/{text}?exact=true&timeout=5000`	Wait for element by text
`GET /waitForIdle`	Wait for UI to stabilize
`GET /captureScreenshot?path=screenshot.png`	Capture screenshot

Workflow Pattern

Follow this sequence for reliable interactions:

# 1. Verify server is running
curl http://localhost:54345/health

# 2. Wait for UI to be ready
curl http://localhost:54345/waitForIdle

# 3. Perform action
curl "http://localhost:54345/onNodeWithTag/username/performTextInput?text=myuser"

# 4. Wait for UI to settle
curl http://localhost:54345/waitForIdle

# 5. Perform next action
curl http://localhost:54345/onNodeWithTag/login_button/performClick

# 6. Wait for result
curl "http://localhost:54345/waitUntilExactlyOneExists/tag/dashboard?timeout=10000"

# 7. Capture screenshot to verify
curl "http://localhost:54345/captureScreenshot?path=result.png"

Finding Test Tags

Search the codebase for existing test tags:

grep -r "testTag\|Modifier.testTag" --include="*.kt" .

Also check:

CLAUDE.md for documented test tags
Test files in src/*Test/ directories

Important Notes

Always URL-encode special characters in text: space→%20, @→%40, &→%26
Use waitForIdle between operations for stability
Check HTTP status codes: 200=success, 400=bad request, 500=error
Use appropriate timeouts for waits (default 5000ms may be too short)
Screenshots require absolute paths

Error Handling

If an endpoint returns an error:

Check the element exists (search for its test tag in code)
Ensure UI has finished loading (waitForIdle)
Verify the server is still running (/health)
Try with a longer timeout for wait operations

Files

.github/dependabot.yml342 B
.github/workflows/build.yml646 B
.github/workflows/junie.yml472 B
.github/workflows/release.yml1.6 KiB
.gitignore343 B
ATTRIBUTION.mdAdded by AutoGineer941 B
CLAUDE.md2.9 KiB
LICENSE1.0 KiB
README.md8.4 KiB
SKILL.md6.0 KiB
build.gradle.kts2.3 KiB
docs/PUBLICATION.md6.0 KiB
gradle/libs.versions.toml909 B
gradle/wrapper/gradle-wrapper.properties252 B
gradlewScript8.4 KiB
gradlew.batScript2.7 KiB
settings.gradle.kts145 B
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/ApplicationLauncher.ktScript2.7 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/ComposeUiTestServer.ktScript4.5 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/ComposeUiTestServerConfig.ktScript1.2 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/Extensions.ktScript3.1 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/TestEndpoint.ktScript1.2 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/endpoint/ClickEndpoints.ktScript2.0 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/endpoint/HealthEndpoint.ktScript611 B
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/endpoint/ScreenshotEndpoint.ktScript1.9 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/endpoint/TextInputEndpoint.ktScript1.6 KiB
src/jvmMain/kotlin/io/github/forketyfork/composeuittest/endpoint/WaitEndpoints.ktScript3.2 KiB

README.md

compose-ui-test-server

Image: Build status Image: MIT License Image: Kotlin Image: Release

A library that enables coding agents and automation tools to control Compose Desktop applications at runtime via HTTP. Agents can click buttons, enter text, wait for UI elements, and capture screenshots through simple REST API calls.

Use Case

This library is designed for AI coding agents (like Claude Code) to interact with running Compose Desktop applications. When enabled, the app exposes an HTTP server that agents can use to:

Navigate through the UI by clicking buttons and entering text
Wait for specific UI elements to appear
Capture screenshots to verify the current state
Perform automated workflows and testing scenarios

Features

HTTP-based control of running Compose Desktop apps
Zero-configuration launcher with automatic mode switching
Conditional startup via environment variable (disabled by default)
Built-in endpoints for common UI operations
Extensible with custom endpoints for app-specific shortcuts
Screenshot capture for visual verification

Installation

Add the dependency to your desktop source set in build.gradle.kts:

kotlin {
    sourceSets {
        @OptIn(org.jetbrains.compose.ExperimentalComposeLibrary::class)
        val desktopMain by getting {
            dependencies {
                implementation("io.github.forketyfork:compose-ui-test-server:0.2.0")

                // Required: Compose UI Test framework
                implementation(compose.uiTest)
            }
        }
    }
}

Quick Start

1. Use the Application Launcher

Replace your main() function with runApplication:

import io.github.forketyfork.composeuittest.WindowConfig
import io.github.forketyfork.composeuittest.runApplication

fun main() =
    runApplication(
        windowConfig = WindowConfig(
            title = "My App",
            minimumWidth = 1024,
            minimumHeight = 768,
        ),
    ) {
        App()
    }

That's it! Your app now supports agent control with zero additional configuration.

2. Add Test Tags to UI Elements

For agents to interact with UI elements, add test tags:

import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.testTag

Button(
    onClick = { /* ... */ },
    modifier = Modifier.testTag("login_button")
) {
    Text("Login")
}

TextField(
    value = username,
    onValueChange = { username = it },
    modifier = Modifier.testTag("username_field")
)

3. Run Your App

Normal mode (default):

./gradlew run

Agent-controlled mode:

COMPOSE_UI_TEST_SERVER_ENABLED=true ./gradlew run

When the environment variable is set, your app automatically starts with an HTTP server that agents can use to control the UI.

4. Control via HTTP

Once running in agent-controlled mode:

# Health check
curl http://localhost:54345/health

# Click a button by test tag
curl http://localhost:54345/onNodeWithTag/submit_button/performClick

# Enter text into a field
curl "http://localhost:54345/onNodeWithTag/username_field/performTextInput?text=myuser"

# Wait for an element to appear
curl "http://localhost:54345/waitUntilExactlyOneExists/tag/welcome_screen?timeout=5000"

# Capture a screenshot
curl "http://localhost:54345/captureScreenshot?path=current_state.png"

Available Endpoints

Endpoint	Description
`GET /health`	Health check - returns "OK"
`GET /onNodeWithTag/{tag}/performClick`	Click element by test tag
`GET /onNodeWithTag/{tag}/performTextInput?text=...`	Enter text into element
`GET /onNodeWithText/{text}/performClick`	Click element by display text
`GET /waitUntilExactlyOneExists/tag/{tag}?timeout=5000`	Wait for element by tag
`GET /waitUntilExactlyOneExists/text/{text}?exact=true&timeout=5000`	Wait for element by text
`GET /waitForIdle`	Wait for UI to become idle
`GET /captureScreenshot?path=screenshot.png`	Capture screenshot to file

Environment Variables

Variable	Description	Default
`COMPOSE_UI_TEST_SERVER_ENABLED`	Enable agent-controlled mode	`false`
`COMPOSE_UI_TEST_SERVER_PORT`	Server port	`54345`

Advanced Configuration

Server Configuration

For more control over the test server, pass a server configuration:

import io.github.forketyfork.composeuittest.ComposeUiTestServerConfig
import io.github.forketyfork.composeuittest.WindowConfig
import io.github.forketyfork.composeuittest.runApplication

fun main() =
    runApplication(
        windowConfig = WindowConfig(title = "My App"),
        serverConfig = ComposeUiTestServerConfig(
            port = 8080,
            host = "0.0.0.0",
            defaultScreenshotPath = "app_screenshot.png",
            defaultTimeout = 10_000L,
        ),
    ) {
        App()
    }

Custom Endpoints

Add app-specific shortcuts for common agent workflows:

import io.github.forketyfork.composeuittest.TestEndpoint
import androidx.compose.ui.test.ComposeUiTest
import androidx.compose.ui.test.onNodeWithTag
import androidx.compose.ui.test.performTextInput
import androidx.compose.ui.test.performClick
import io.ktor.server.routing.Route
import io.ktor.server.routing.get
import io.ktor.server.response.respondText

class LoginEndpoint(
    private val username: String,
    private val password: String
) : TestEndpoint {
    override fun Route.configure(composeTest: ComposeUiTest) {
        get("/shortcuts/login") {
            composeTest.onNodeWithTag("username").performTextInput(username)
            composeTest.onNodeWithTag("password").performTextInput(password)
            composeTest.onNodeWithTag("login_button").performClick()
            call.respondText("Login completed")
        }
    }
}

To use custom endpoints, use the lower-level API:

import androidx.compose.ui.test.runComposeUiTest
import androidx.compose.ui.window.Window
import androidx.compose.ui.window.application
import io.github.forketyfork.composeuittest.ComposeUiTestServer
import io.github.forketyfork.composeuittest.isTestServerEnabled

fun main() {
    if (isTestServerEnabled()) {
        runComposeUiTest {
            setContent { App() }
            ComposeUiTestServer(this)
                .registerEndpoint(LoginEndpoint("[email protected]", "password"))
                .start()
                .awaitTermination()
        }
    } else {
        application {
            Window(onCloseRequest = ::exitApplication, title = "My App") {
                App()
            }
        }
    }
}

Platform Support

Currently supports Desktop (JVM) only due to Skia dependencies for screenshot capture.

Claude Code Skill Installation

This library includes a skill that teaches Claude Code how to control Compose Desktop apps and set up new projects. Install it to enable automatic UI control capabilities.

Personal Installation (all your projects)

mkdir -p ~/.claude/skills/compose-ui-control
curl -o ~/.claude/skills/compose-ui-control/SKILL.md \
  https://raw.githubusercontent.com/forketyfork/compose-ui-test-server/main/SKILL.md

Project Installation (this project only)

mkdir -p .claude/skills/compose-ui-control
curl -o .claude/skills/compose-ui-control/SKILL.md \
  https://raw.githubusercontent.com/forketyfork/compose-ui-test-server/main/SKILL.md

Using the Skill

Once installed, Claude Code can:

Automatically use the skill when you ask it to interact with UI, click buttons, or test the app
Manually invoke it with /compose-ui-control
Set up new projects with agent control capabilities

Example prompts that trigger the skill:

"Click the login button in the app"
"Enter my username and password"
"Take a screenshot of the current state"
"Wait for the dashboard to appear"
"Add agent control to this Compose Desktop project"

Skill Reference

See SKILL.md for the full skill documentation that Claude Code uses.

License

MIT License - see LICENSE for details.

Requirements and compatibility

Written for: Claude Code

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Claude Code frontmatter, Needs network access

Security

What our scanner noticed· Low risk

Instructions that use the network
Scripts that your agent may run

This skill includes scripts that your agent may run. Read them before you use it.

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Forketyfork

Publisher

JetBrains (Official)

Distributed by

JetBrains

Source

JetBrains/skills · compose-ui-test-server · commit 3f53782

Licence

MIT License (redistribution allowed)

ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Forketyfork or JetBrains.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download compose-ui-control.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by Forketyfork

compose-ui-test-server

Checking If Already Installed

Installing in a Compose Desktop Project

Step 1: Add the dependency

Step 2: Update the main function

Step 3: Add test tags to UI elements

Starting the Application

Verifying the Server

Available Endpoints

Workflow Pattern

Finding Test Tags

Important Notes

Error Handling

Files

README.md

compose-ui-test-server

Use Case

Features

Installation

Quick Start

1. Use the Application Launcher

2. Add Test Tags to UI Elements

3. Run Your App

4. Control via HTTP

Available Endpoints

Environment Variables

Advanced Configuration

Server Configuration

Custom Endpoints

Platform Support

Claude Code Skill Installation

Personal Installation (all your projects)

Project Installation (this project only)

Using the Skill

Skill Reference

License

Requirements and compatibility

Security

Attribution and licence

Related skills

Migrate To Shoehorn

Tdd

Test Driven Development

Webapp Testing

Test Driven Development

Browser Testing With Devtools

Download ZIP