Display Glasses With Jetpack Compose Glimmer
Android (Google)OfficialCreated by Google LLCDistributed by Android (Google)
- Skill
- Tool integration
- Software development
- Apache-2.0
- No issues found
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides design principles, UI patterns, and development workflows for creating projected Android XR applications for display glasses. It focuses on the Jetpack Compose Glimmer toolkit, covering component usage, theming, input handling, and hardware integration for additive display environments.
What it does
- Defines Glimmer design principles and interaction models for display glasses.
- Provides guidance on setting up projected activities and managing device connectivity.
- Outlines UI component usage including cards, buttons, lists, and stacks.
- Details typography, color, and depth token usage for additive displays.
- Explains mapping hardware touchpad and gesture inputs to app actions.
- Offers patterns for voice input indicators and hardware access.
When to use it
- When developing Android XR apps specifically for display glasses.
- When implementing UI using the Jetpack Compose Glimmer toolkit.
- When designing interactions for one-dimensional hardware controls.
- When configuring apps to run as projected activities on connected eyewear.
Advantages
- Ensures consistent UI behavior across display glasses.
- Provides specific guidance for additive display rendering.
- Standardizes interaction patterns for limited input hardware.
- Facilitates proper integration with the Android XR projected context.
Usage examples
- “How do I set up a basic projected activity for display glasses?”
- The skill provides the AndroidManifest configuration and the necessary Activity setup, including the use of ProjectedDisplayController and GlimmerTheme.
- “What are the best practices for card layouts in Glimmer?”
- It explains how to use ActionCard for single actions and ButtonGroup with non-focusable cards to avoid focus contention.
- “How should I handle user input on display glasses?”
- The skill details mapping touchpad gestures like taps and swipes, and explains how to use System Back for navigation.
- “How do I implement a voice input indicator?”
- It provides a code pattern for using the VoiceInputIndicator component driven by SpeechRecognizer RMS changes.
Install
Download the skill as a ZIP:
https://autogineer.com/agent-skills/android/display-glasses-with-jetpack-compose-glimmer/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip display-glasses-with-jetpack-compose-glimmer.zip -d .agents/skills
Documentation by Google LLC
Glossary
| Term | Definition |
|---|---|
| Intelligent eyewear | Hands-free eyewear with speakers, camera, and microphone (audio-only or display glasses). |
| Display glasses | Eyewear with a private in-lens display for glanceable visuals that pair with audio. |
| Jetpack Compose Glimmer | Compose UI toolkit providing components, theming, and focus behavior for display glasses. |
| Projected activity | An Activity running on a phone host whose UI and input are projected to connected eyewear. |
| Projected device | An XR eyewear device connected to an Android host phone that projects app content to it. |
| GlimmerTheme | Root styling provider for GlimmerColors, GlimmerTypography, and GlimmerShapes. |
| Additive display | Display where black (#000000) renders as 100% transparent by adding light to the scene. |
| Augmented experiences | Lightweight, additive Android XR experiences that help users while they're on the go. |
| Visual angle | Perceived size in XR; minimum readable text is 0.6 degrees (about 18sp at 1 m). |
Prerequisites
Target
compileSdk37 or higher (compileSdk = 37orcompileSdk { version = release(37) }). UpdatecompileSdkto 37 if lower.When adding dependencies, follow the instructions in Create your first activity for intelligent eyewear:
androidx.xr.glimmer:glimmer(minimum1.0.0-alpha21, or catalog aliaslibs.androidx.glimmer)androidx.xr.glimmer:glimmer-google-fonts(minimum1.0.0-alpha21, or catalog aliaslibs.androidx.glimmer.google.fonts)androidx.xr.projected:projected(minimum1.0.0-alpha13, or catalog aliaslibs.androidx.xr.projectedorlibs.androidx.projected)Check
gradle/libs.versions.tomlfirst; if catalog entries exist, use them inapp/build.gradle.ktsand verify their versions meet the minimums. Never guess1.0.0-alpha01forglimmer-google-fonts.
Core constraints and limitations
Don't : Use
MaterialThemeor Material 3 components (androidx.compose.material3.*) inside Projected Activities or Glimmer UI.Do : Use
GlimmerThemeand Glimmer components (androidx.xr.glimmer.*).Do : Always pass
createGoogleSansFlexTypography()fromandroidx.xr.glimmer.googlefontsastypographyinGlimmerTheme.Specific hardware device models or unique capabilities aren't detailed.
Elevation : Material 3 shadow or elevation modifiers aren't supported. Use
ShadowScopeorDepthtokens from Jetpack Compose Glimmer.Typography limits: Minimum readable text size is 18sp (0.6°). Avoid "Thin" or "Hairline" weights, which shimmer on additive lenses.
1. Set up dependencies
Set up projected activity : When creating a projected activity, follow Create your first activity for intelligent eyewear and projectedcontext-source.md. Set
android:requiredDisplayCategorytoandroid.hardware.display.category.XR_PROJECTEDinAndroidManifest.xml:<activity android:name="com.example.xr.projected.ProjectedMainActivity" android:exported="true" android:requiredDisplayCategory="android.hardware.display.category.XR_PROJECTED"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.XR_PROJECTED_LAUNCHER"/> </intent-filter> </activity><br />
Mobile app integration : In the host phone app, add a button with
Text("Launch on Glasses"). ObserveProjectedContext.isProjectedDeviceConnected(context, coroutineContext)usingcollectAsStateWithLifecycle(initialValue = false)to set the button'senabledstate so it's disabled when glasses are disconnected. When clicked, launchGlassesMainActivityon the glasses display usingProjectedContext.createProjectedActivityOptions(context).toBundle()as shown in the following snippet:val options = ProjectedContext.createProjectedActivityOptions(context) val intent = Intent(context, ProjectedMainActivity::class.java) context.startActivity(intent, options.toBundle())<br />
UI library and font : When declaring the Glimmer, Google Fonts, and Projected artifacts in
app/build.gradle.kts, follow Declaring Jetpack Compose Glimmer Dependencies, and follow glimmersansflextypography-source.md when configuring Google Sans Flex usingcreateGoogleSansFlexTypography.Theming, initial focus, and black background : On additive displays, pure black (
Color.BlackorGlimmerTheme.colors.background) renders as 100% transparent. InonCreate(annotated with@OptIn(ExperimentalComposeUiApi::class)), setComposeUiFlags.isInitialFocusOnFocusableAvailable = true(androidx.compose.ui.ComposeUiFlags), wrap content inGlimmerTheme(typography = createGoogleSansFlexTypography()), and applyModifier.background(Color.Black)(orGlimmerTheme.colors.background) to the root container:@OptIn(ExperimentalProjectedApi::class) class ProjectedMainActivity : ComponentActivity() { private var displayController: ProjectedDisplayController? = null private var isVisualUiSupported by mutableStateOf(false) private var areVisualsOn by mutableStateOf(true) override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) lifecycle.addObserver(object : DefaultLifecycleObserver { override fun onDestroy(owner: LifecycleOwner) { displayController?.close() displayController = null } }) initializeGlassesFeatures() setContent { GlimmerTheme { HomeScreen( areVisualsOn = areVisualsOn, isVisualUiSupported = isVisualUiSupported, onClose = { finish() } ) } } } private fun initializeGlassesFeatures() { lifecycleScope.launch { // Check device capabilities val projectedDeviceController = ProjectedDeviceController.create(this@ProjectedMainActivity) isVisualUiSupported = projectedDeviceController.capabilities.contains(CAPABILITY_VISUAL_UI) val controller = ProjectedDisplayController.create(this@ProjectedMainActivity) displayController = controller val observer = GlassesLifecycleObserver( context = this@ProjectedMainActivity, controller = controller, onVisualsChanged = { visualsOn -> areVisualsOn = visualsOn } ) lifecycle.addObserver(observer) } } }<br />
Hardware capabilities, permissions, and access : When checking hardware capabilities, requesting permissions, or accessing glasses hardware, follow Check device capabilities at runtime, Request hardware permissions, and Use a projected context to access hardware.
2. Minimize and translate the UI
Build UI with Jetpack Compose Glimmer components and use depth for priority.
Design from the bottom up and always bottom-align UI (
Alignment.BottomCenter).One thing at a time : Show one primary piece of information at a time (for example, using
VerticalStack) and avoid multiple simultaneous cards.Color contrast : When choosing colors, follow material-hct-source.md to maintain at least a 70% tone difference between foreground and background in HCT.
3. Map input controls
Map tap and swipe interactions to the glasses touchpad. Inputs are one-dimensional; avoid nesting scrolling controls.
Use System Back to dismiss temporary states or detailed views.
When implementing focus navigation and hardware input handling, follow Focus in Jetpack Compose Glimmer and Hardware Controls for display glasses. Focus movement is continuous for scrollable containers and discrete for elements like button rows.
4. Build with Jetpack Compose Glimmer
Implementing Glimmer styles
When accessing theme tokens using GlimmerTheme, follow
glimmertheme-source.md.
| Category | Token | Value / Role |
|---|---|---|
| Color | primary / secondary | #9BBFFF / #4C88E9 (Focal colors) |
| Color | positive / negative | #7AD48F (Confirmation) / #F57082 (Alert) |
| Color | background | #000000 (Pure black, renders 100% transparent) |
| Color | surface | #262626 (Use SurfaceDefaults.border for borders) |
| Shape | Standard / Small | RoundedCornerShape(36.dp) / RoundedCornerShape(12.dp) |
Typography scale (Google Sans Flex) and depth levels
Use Google Sans Flex with variable axes ROND = 100f, wdth = 100f, opsz
= 9f, and wght by role:
| Style | Size / Line-Height | Weight |
|---|---|---|
| Title Large / Medium / Small | 30/36.sp, 24/32.sp, 20/28.sp | 725 |
| Body Large / Medium / Small | 30/36.sp, 24/32.sp, 20/28.sp | 520 |
| Caption | 18.sp / 28.sp | 650 |
When simulating depth with DepthEffect and DepthEffectLevels, follow
deptheffect-source.md and deptheffectlevels-source.md: use level1
for resting cards and background UI, and level2 for focused or pressed
interactive UI.
Implementing Jetpack Compose Glimmer components
Cards, ActionCards, ImageCards, and ButtonGroups
Cards (androidx.xr.glimmer.Card) group related content into a single focal
point. Never embed a Card within a ListItem. When implementing cards, follow
the instructions in Jetpack Compose Glimmer: Cards, card-source.md,
and card-samples-source.md.
Clickable versus non-clickable card : Pass
onClick = { ... }if the entireCardacts as a trigger; omitonClickif informational. Control focus on non-clickable cards withfocusable: Boolean = true(such asCard(focusable = false)).Single primary action (
ActionCard) : When a card has a single primary button action anchored to its bottom edge, useandroidx.xr.glimmer.ActionCardinstead of placing aButtoninsideCard. Itsactionbutton takes focus directly:ActionCard( action = { Button(onClick = {}, trailingIcon = { Icon(FavoriteIcon, "Localized description") }) { Text("Send") } } ) { Text("This is a card with a title, leading icon, header image, and action") }<br />
Multiple actions (
CardandButtonGroup)---avoiding focus contention : When pairing a card with multiple action buttons, useandroidx.xr.glimmer.ButtonGroup(with optionalrememberButtonGroupState). Never nestButtonGroupinsideCard { ... }. Place a non-focusableCard(focusable = false)(withoutonClick) aboveButtonGroupas a sibling in aColumnso the card and buttons don't compete for focus:ButtonGroup(modifier = Modifier.fillMaxWidth()) { Button(onClick = {}) { Text("Button 1") } Button(onClick = {}) { Text("Button 2") } Button(onClick = {}) { Text("Button 3") } Button(onClick = {}) { Text("Button 4") } Button(onClick = {}) { Text("Button 5") } }<br />
Image cards : Use
ImageCard,LeadingImageCard, orTrailingImageCardto display prominent imagery alongside card text.
Buttons and ToggleButtons
When implementing buttons or toggle buttons, follow the instructions in Jetpack Compose Glimmer: Buttons, button-source.md, and button-samples-source.md.
Standard and icon buttons : Use
Buttonfor text actions (with optionalleadingIcon/trailingIcon) andIconButtonfor icon-only triggers (such as media next-track or previous-track buttons).Toggle buttons (
ToggleButtonandIconToggleButton) : UseToggleButtonorIconToggleButtonfor stateful on/off or play/pause controls that morph based onchecked:@Composable fun ToggleButtonWithLeadingIconSample() { var checked by remember { mutableStateOf(false) } ToggleButton( checked = checked, leadingIcon = { Icon( if (checked) FavoriteIcon else OutlinedFavoriteIcon, contentDescription = "Toggle favorite" ) }, onCheckedChange = { checked = it } ) { Text(if (checked) "On" else "Off") } }<br />
Title chips
TitleChip (androidx.xr.glimmer.TitleChip) is a non-interactive pill header
(three words or fewer, single line). When implementing title chips, follow the
instructions in Jetpack Compose Glimmer: Title Chips,
titlechip-source.md, and titlechip-samples-source.md.
Spatial spacing : When placing a standalone
TitleChipabove aCardorActionCard, center theTitleChiphorizontally and separate it from the card usingTitleChipDefaults.associatedContentSpacing(usingSpacerorArrangement.spacedBy(TitleChipDefaults.associatedContentSpacing)):@Composable fun TitleChipWithCardSample() { Column(horizontalAlignment = Alignment.CenterHorizontally) { TitleChip { Text("Title Chip") } Spacer(Modifier.height(TitleChipDefaults.associatedContentSpacing)) Card( title = { Text("Title") }, subtitle = { Text("Subtitle") }, leadingIcon = { Icon(FavoriteIcon, "Localized description") }, ) { Text("Card Content") } } }<br />
Icons
When adding icons or icon buttons, follow the instructions in Jetpack Compose Glimmer: Icons, icon-source.md, iconbutton-source.md, and iconsizes-source.md.
Sizing and interactivity : Size icons with
GlimmerTheme.iconSizes(small,medium,large). UseIconButtonorIconToggleButtonfor triggers andIconfor status indicators.Contrast and library : Never use pure black (
Color.Black) for icon tints; black renders as transparent. Use Material Symbols (600 weight, Rounded, Unfilled). Ifandroidx.compose.material:material-icons-*isn't inapp/build.gradle.kts, build vectors withImageVector.BuilderorpainterResourceinstead of importingandroidx.compose.material.icons.*.
Lists
Use GlimmerLazyColumn (androidx.xr.glimmer.list.GlimmerLazyColumn) for
homogeneous scrollable items. When building scrollable lists and list items,
follow the API definitions in list-source.md, listitem-source.md,
and liststate-source.md.
Always use
GlimmerLazyColumn: Never use foundationLazyColumn. ImportGlimmerLazyColumn,rememberGlimmerLazyListState, anditemsfromandroidx.xr.glimmer.list(itemis a scope member, not imported). For programmatic scrolling to the first item, hoistval listState = rememberGlimmerLazyListState()and calllistState.scrollToItem(0)orlistState.animateScrollToItem(0).Integrated title, spacing, and rows : Pass headers using
title = { TitleChip { Text("...") } }directly onGlimmerLazyColumn(not outside it), setverticalArrangement = Arrangement.spacedBy(20.dp), and render rows withandroidx.xr.glimmer.ListItem(onClick = { ... }, supportingLabel = { Text(...) }) { Text(...) }(pass the primary label as the trailingcontentlambda, notheadlineContent, and never nest aCardinsideListItem):@Composable fun GlimmerLazyColumnWithTitleChipSample() { val ingredientItems = listOf("Milk", "Flour", "Egg", "Salt", "Apples", "Butter", "Vanilla", "Sugar", "Cinnamon") GlimmerLazyColumn(title = { TitleChip { Text("Ingredients") } }) { items(ingredientItems) { text -> ListItem { Text(text) } } } }<br />
Stacks
Use VerticalStack (androidx.xr.glimmer.stack.VerticalStack) to display
heterogeneous cards one at a time (such as calendar, transit, and weather
cards). Never use a TitleChip with VerticalStack. When implementing a
VerticalStack, follow the API definitions in stack-source.md,
stackstate-source.md, and stackitemscope-source.md.
Bottom alignment, height (+66.dp), and
itemDecoration: AlignVerticalStacktoAlignment.BottomCenter, set its height to66.dptaller than the tallest card (maxCardHeight + 66.dp), and applyModifier.itemDecoration(CardDefaults.shape)to theCardinside everyitem { ... }block so background cards are clipped and masked:@Composable fun VerticalStackSample() { VerticalStack { item { Card { Text("Top Card") } } item { Card { Text("Second Card") } } item { Card { Text("Third Card") } } } }<br />
Horizontal pagers and animated text
Use GlimmerHorizontalPager
(androidx.xr.glimmer.pager.GlimmerHorizontalPager) with
rememberGlimmerPagerState(pageCount = { ... }) for horizontal card
walkthroughs (never foundation HorizontalPager).
Smooth text motion : Inside pager pages, configure Glimmer
Text(androidx.xr.glimmer.Text, neverandroidx.compose.material3.Text) withtextMotion = TextMotion.Animated(using.copy(textMotion = TextMotion.Animated)) for smooth rendering during snap transitions:// Hoist the pager state, specifying the total page count with a lambda. val pagerState = rememberGlimmerPagerState(pageCount = { 10 }) GlimmerHorizontalPager( state = pagerState, modifier = Modifier.fillMaxSize(), ) { page -> // Use Glimmer components like Card and Text for optimized glasses styling. Card(modifier = Modifier.fillMaxWidth()) { Text( text = "Page: $page", // Recommended: use TextMotion.Animated for smooth transitions in a pager. style = LocalTextStyle.current.copy(textMotion = TextMotion.Animated), ) } }<br />
Surface
Modifier.surface (androidx.xr.glimmer.surface) builds custom interactive
components (such as status badges without Card or Button). When creating
custom surfaces, follow the API definitions in surface-source.md.
Shared
MutableInteractionSourceand order : Create a singleremember { MutableInteractionSource() }and pass it to bothModifier.surface(interactionSource = interactionSource, ...)and.clickable(interactionSource = interactionSource, ...)(or.focusable), chaining.surface(...)before.clickable(...):@Composable fun FocusableSurfaceSample() { val interactionSource = remember { MutableInteractionSource() } Box( modifier = Modifier .size(100.dp) .surface(interactionSource = interactionSource) .focusable(interactionSource = interactionSource), contentAlignment = Alignment.Center ) { Text("Focusable") } } @Composable fun ClickableSurfaceSample() { val interactionSource = remember { MutableInteractionSource() } Box( modifier = Modifier .size(100.dp) .surface(interactionSource = interactionSource) .focusable(interactionSource = interactionSource) .clickable( interactionSource = interactionSource, indication = null ) { /* Handle click action */ }, contentAlignment = Alignment.Center ) { Text("Clickable") } }<br />
Voice input indicator and indirect pointer gestures
VoiceInputIndicatorandContainedVoiceInputIndicator: Pass alevellambda returning aFloatnormalized to0.0f..1.0f. When driven bySpeechRecognizer(RecognitionListener.onRmsChanged(rmsdB: Float)), normalize and clamprmsdBwith.coerceIn(0f, 1f):// Example state variable to hold the normalized level (0.0 to 1.0) // that the VoiceInputIndicator component expects. val audioLevel = MutableStateFlow(0f) // Initialize the Android SpeechRecognizer val speechRecognizer = SpeechRecognizer.createSpeechRecognizer(context) // Listener to capture speech events and audio level changes val listener = object : RecognitionListener { override fun onRmsChanged(rmsdB: Float) { // Normalize raw dB level to a 0.0-1.0 range. // Android SpeechRecognizer's rmsdB typically ranges from 0 to ~10. audioLevel.value = ((rmsdB - 1f) / 9f).coerceIn(0f, 1f) } // ... Implement other required RecognitionListener methods ... override fun onReadyForSpeech(params: Bundle?) {} override fun onBeginningOfSpeech() {} override fun onEndOfSpeech() {} override fun onError(error: Int) {} override fun onResults(results: Bundle?) {} override fun onPartialResults(partialResults: Bundle?) {} override fun onEvent(eventType: Int, params: Bundle?) {} override fun onBufferReceived(buffer: ByteArray?) {} } // Attach the listener to the recognizer speechRecognizer.setRecognitionListener(listener) // Create an intent to specify the recognition model and behavior val intent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH).apply { putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM) } // Begin listening for audio input speechRecognizer.startListening(intent)<br />
@Composable fun VoiceInputExample(modifier: Modifier) { // Collect the flow as Compose State so the UI reacts to changes in real-time. val currentLevel by audioLevel.collectAsState() VoiceInputIndicator( // The VoiceInputIndicator component provides a visual "pulse" or indicator // that changes based on the 'level' lambda, which returns a Float between 0.0 and 1.0. level = { currentLevel }, modifier = modifier ) }<br />
Modifier.onIndirectPointerGesture: Handle low-level touchpad tap (onClick), forward swipe (onSwipeForward), and backward swipe (onSwipeBackward) gestures usingandroidx.xr.glimmer.onIndirectPointerGestureon a container made focusable (usingfocusTarget,focusable, orsurface(focusable = true)):@Composable fun OnIndirectPointerGestureSample() { var lastGesture by remember { mutableStateOf("None") } Box( modifier = Modifier.fillMaxSize() .onIndirectPointerGesture( enabled = true, onSwipeForward = { lastGesture = "Forward" }, onSwipeBackward = { lastGesture = "Backward" }, onClick = { lastGesture = "Click" }, ) .focusTarget(), contentAlignment = Alignment.Center, ) { Text("Last Gesture: $lastGesture") } }<br />
5. Integrate with system UI
When handling notifications or starting a projected activity from a notification, follow the instructions in Understand notification behavior for intelligent eyewear and Start a glasses activity on display glasses from a notification.
Files
- ATTRIBUTION.mdAdded by AutoGineer1.0 KiB
- LICENSELicence file added by AutoGineer11.1 KiB
- SKILL.md25.7 KiB
- references/android/design/ui/ai-glasses/guides/interaction/inputs.md4.6 KiB
- references/android/develop/xr/jetpack-xr-sdk/access-hardware-projected-context.md25.9 KiB
- references/android/develop/xr/jetpack-xr-sdk/request-hardware-permissions.md9.1 KiB
- references/button-samples-source.md4.9 KiB
- references/button-source.md9.7 KiB
- references/card-samples-source.md10.3 KiB
- references/card-source.md51.7 KiB
- references/deptheffect-source.md7.8 KiB
- references/deptheffectlevels-source.md4.1 KiB
- references/glimmersansflextypography-source.md11.3 KiB
- references/glimmertheme-source.md7.6 KiB
- references/icon-source.md16.8 KiB
- references/iconbutton-source.md4.8 KiB
- references/iconsizes-source.md2.3 KiB
- references/list-source.md713 B
- references/listitem-source.md11.2 KiB
- references/liststate-source.md17.8 KiB
- references/material-hct-source.md6.3 KiB
- references/projectedcontext-source.md12.9 KiB
- references/stack-source.md15.3 KiB
- references/stackitemscope-source.md16.2 KiB
- references/stackstate-source.md12.9 KiB
- references/surface-source.md32.3 KiB
- references/titlechip-samples-source.md2.9 KiB
- references/titlechip-source.md5.1 KiB
references/android/develop/xr/jetpack-xr-sdk/access-hardware-projected-context.md
Applicable XR devices This guidance helps you build experiences for these types of XR devices. Learn about XR device types → Image: untitled Audio &
Display Glasses Learn about XR device types →
<br />
After you've requested and been granted the necessary permissions, your app can access the hardware on the audio glasses or display glasses. The key to accessing the glasses' hardware (instead of the phone's hardware), is to use a projected context.
There are two primary ways to get a projected context, depending on where your code is executing:
Get a projected context if your code is running in an projected activity
If your app's code is running from within your projected activity, its own activity context is already a projected context. In this scenario, calls made within that activity can already access the glasses' hardware.
Get a projected context for code running in a phone app component
If a part of your app outside of your projected activity (such as a phone
activity or a service) needs to access the glasses' hardware, it must explicitly
obtain a projected context. To do this, use the
createProjectedDeviceContext method:
@RequiresApi(Build.VERSION_CODES.BAKLAVA)
@OptIn(ExperimentalProjectedApi::class, ExperimentalCoroutinesApi::class)
private fun monitorProjectedConnectivity(activity: ComponentActivity) {
activity.lifecycleScope.launch {
// Before creating a projected context, check to see if the projected device is connected.
// While this method returns true, the projected context remains valid.
ProjectedContext.isProjectedDeviceConnected(activity, coroutineContext)
.collectLatest { isConnected ->
if (isConnected) {
// From a phone Activity or Service, get a context for the audio and display glasses.
// Re-initialize on reconnect: Obtain another context instance.
val projectedContext = try {
ProjectedContext.createProjectedDeviceContext(activity)
} catch (e: IllegalStateException) {
Log.e(TAG, "Failed to create projected context", e)
return@collectLatest
}
// Use the projectedContext to initialize system services (e.g., CameraManager).
Log.i(TAG, "Projected device connected. Initializing hardware...")
} else {
// The projected context is destroyed when the device disconnects.
// Clean up on disconnect: Listen for 'false' and release resources.
Log.i(TAG, "Projected device disconnected. Cleaning up hardware resources...")
}
}
}
}<br />
Check for validity
Wrap the createProjectedDeviceContext call within the
ProjectedContext.isProjectedDeviceConnected. While this method returns
true, the projected context remains valid to the connected device, and your
phone app activity or service (such as a CameraManager) can access the glasses
hardware.
Clean up on disconnect
The projected context is tied to the lifecycle of the connected device, so it is
destroyed when the device disconnects. When the device disconnects,
ProjectedContext.isProjectedDeviceConnected returns false. Your app
should listen for this change and clean up any system services (such as a
CameraManager) or resources that your app created using that projected
context.
Re-initialize on reconnect
When the glasses reconnect, your app can obtain another projected context
instance using createProjectedDeviceContext, and then re-initialize any
system services or resources using the new projected context.
Record audio with the glasses' microphone
You can record audio from the glasses using Bluetooth Hands-Free Profile (HFP). In this mode, the glasses connect to the phone using standard Headset and Advanced Audio Distribution Profile (A2DP) profiles, functioning like a typical Bluetooth peripheral. If your app is already designed for standard Bluetooth recording, you can use this method to record audio from the glasses without any additional development.
Record audio using Bluetooth HFP
To record audio using Bluetooth HFP, first request the required runtime
permissions, and then record the audio using the AudioManager API, as
described in the following sections.
Request permissions
As with any standard Bluetooth audio device, the RECORD_AUDIO,
BLUETOOTH_CONNECT, and other related permissions are controlled by the
phone and not the connected device (such as audio glasses or display glasses).
Follow these steps to request the permissions:
Declare following permissions in your app's manifest file:
Request both the
RECORD_AUDIOandBLUETOOTH_CONNECTpermissions at runtime using the standard Android permission flow.
Use AudioManager to route audio
After the user has granted your app the necessary runtime permissions, use the
AudioManager API to set the communication device to
TYPE_BLUETOOTH_SCO to route the audio through Bluetooth HFP. This
directs the system to retrieve audio from the Bluetooth peripheral.
val audioManager = context.getSystemService(AudioManager::class.java) ?: return
val hfpDevice = audioManager.availableCommunicationDevices
.firstOrNull { it.type == AudioDeviceInfo.TYPE_BLUETOOTH_SCO } ?: return
// HFP input is limited to 16 kHz, with mono or stereo channel configurations.
val sampleRate = 16_000
val channelMask = AudioFormat.CHANNEL_IN_MONO
val encoding = AudioFormat.ENCODING_PCM_16BIT
// Use a small multiple of the minimum buffer size to keep chunks short for low latency.
val minBufferSize = AudioRecord.getMinBufferSize(sampleRate, channelMask, encoding)
if (minBufferSize <= 0) return
val audioRecord = AudioRecord.Builder()
.setAudioSource(MediaRecorder.AudioSource.VOICE_COMMUNICATION)
.setAudioFormat(
AudioFormat.Builder()
.setSampleRate(sampleRate)
.setChannelMask(channelMask)
.setEncoding(encoding)
.build()
)
.setBufferSizeInBytes(minBufferSize * 2)
.build()
try {
// Route voice communication audio through the Bluetooth device.
if (!audioManager.setCommunicationDevice(hfpDevice)) return
audioRecord.startRecording()
// Read audio with audioRecord.read() on a background thread.
} finally {
// Stop and release when done.
if (audioRecord.recordingState == AudioRecord.RECORDSTATE_RECORDING) audioRecord.stop()
audioRecord.release()
audioManager.clearCommunicationDevice()
}<br />
[!NOTE] Note: To learn more about Bluetooth connectivity, refer to our documentation.
Capture an image with the glasses' camera
To capture an image with the glasses' camera, set up and bind the CameraX's
ImageCapture use case to the glasses' camera using the correct
context for your app:
private fun setUpGlassesCamera(activity: ComponentActivity) {
activity.lifecycleScope.launch {
// Before creating a projected context, check to see if the projected device is connected.
ProjectedContext.isProjectedDeviceConnected(activity, coroutineContext)
.collectLatest { isConnected ->
if (isConnected) {
// 1. Get the CameraProvider using the projected context.
// When using the projected context, DEFAULT_BACK_CAMERA maps to the audio and display glasses' camera.
val projectedContext = try {
ProjectedContext.createProjectedDeviceContext(activity)
} catch (e: IllegalStateException) {
Log.e(TAG, "Projected context could not be created", e)
return@collectLatest
}
val cameraProviderFuture = ProcessCameraProvider.getInstance(projectedContext)
cameraProviderFuture.addListener({
val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()
val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
// 2. Check for the presence of a camera.
if (!cameraProvider.hasCamera(cameraSelector)) {
Log.w(TAG, "The selected camera is not available.")
return@addListener
}
// 3. Query supported streaming resolutions dynamically using Camera2 Interop.
val cameraInfo = cameraProvider.getCameraInfo(cameraSelector)
val map = Camera2CameraInfo.from(cameraInfo).getCameraCharacteristic(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP
)
val supportedResolutions = map?.getOutputSizes(ImageFormat.JPEG)?.toList() ?: emptyList()
Log.i(TAG, "Supported resolutions for JPEG: ${supportedResolutions.joinToString()}")
// 4. Dynamically select the best target resolution from the queried list.
val preferredResolution = Size(1920, 1080)
val dynamicTargetResolution = when {
// Case A: The preferred resolution is explicitly supported by the glasses.
supportedResolutions.contains(preferredResolution) -> preferredResolution
// Case B: Find the highest supported 16:9 resolution as a fallback.
else -> {
val targetAspectRatio = 16.0 / 9.0
val tolerance = 0.01
supportedResolutions
.filter { abs((it.width.toDouble() / it.height) - targetAspectRatio) < tolerance }
.maxByOrNull { it.width * it.height }
// Case C: Absolute fallback if no 16:9 resolutions are found.
?: supportedResolutions.firstOrNull()
?: preferredResolution
}
}
Log.i(TAG, "Selected target resolution: $dynamicTargetResolution")
val resolutionStrategy = ResolutionStrategy(
dynamicTargetResolution,
ResolutionStrategy.FALLBACK_RULE_CLOSEST_LOWER
)
val resolutionSelector = ResolutionSelector.Builder()
.setResolutionStrategy(resolutionStrategy)
.build()
// 5. Initialize the ImageCapture use case with options.
val imageCapture = ImageCapture.Builder()
// Optional: Configure resolution, format, etc.
.setResolutionSelector(resolutionSelector)
.build()
try {
// Unbind use cases before rebinding.
cameraProvider.unbindAll()
// Bind use cases to camera using the Activity as the LifecycleOwner.
cameraProvider.bindToLifecycle(
activity,
cameraSelector,
imageCapture
)
} catch (exc: Exception) {
Log.e(TAG, "Use case binding failed", exc)
}
}, ContextCompat.getMainExecutor(activity))
}
}
}
}<br />
Key points about the code
Obtains an instance of the
ProcessCameraProviderusing the projected device context.Within the projected context's scope, the glasses' primary, outward-pointing camera maps to the
DEFAULT_BACK_CAMERAwhen selecting a camera.A pre-binding check uses
cameraProvider.hasCamera(cameraSelector)to verify that the selected camera is available on the device before proceeding.Uses Camera2 Interop with
Camera2CameraInfoto read the underlyingCameraCharacteristics#SCALER_STREAM_CONFIGURATION_MAP, which can be useful for advanced checks on supported resolutions.A custom
ResolutionSelectoris built to precisely control the output image resolution forImageCapture.Creates an
ImageCaptureuse case that is configured with a customResolutionSelector.Binds the
ImageCaptureuse case to the activity's lifecycle. This automatically manages the opening and closing of the camera based on the activity's state (for example, stopping the camera when the activity is paused).Configuring for single-stream hardware: The glasses camera pipeline is constrained to a single active stream at a time. When
ImageCaptureis bound alone, CameraX attaches an internal repeating stream (MeteringRepeating) for focus and metering by default, which causes the single-stream pipeline to stall. To bindImageCaptureby itself without a preview stream, disable the forced repeating stream by settingsetRepeatingStreamForced(false)onCameraXConfig.Builder(for example, by implementingCameraXConfig.Providerin yourApplicationclass or configuringProcessCameraProviderbefore initialization).
After the glasses' camera is set up, you can capture an image with the CameraX's
ImageCapture class. Refer to the CameraX's documentation to learn about using
takePicture to capture an image.
Capture a stream with the glasses' camera
To capture streams instead of an image, replace the ImageCapture component
from the capture image example with VideoCapture or
ImageAnalysis components and change the capture logic.
For more information about these APIs how to use them, see the CameraX video capture and CameraX iamge analysis documentation.
Choose a resolution and frame rate
You should query the device to find its supported resolutions and their corresponding maximum frame rates. The following table lists examples of values you might see, depending on your app's use case. To conserve battery power and reduce heat on the glasses, choose the lowest resolution and frame rate that work for your use case.
| Use case | Resolution | Frame rate |
|---|---|---|
| Text recognition (OCR) or world sensing | 1920 x 1080 | 1 FPS |
| AI video streaming (multimodal assistants) | 1024 x 768 | 1 FPS |
| Localization and visual positioning (VPS) | 640 x 360 or 640 x 480 | 10 FPS |
| Video calling | 640 x 360 or 640 x 480 | 24 FPS |
[!IMPORTANT] Important: Supported resolutions and their maximum frame rates vary by device and can change. You should query them at runtime instead of hardcoding them.
Query the maximum frame rate for each resolution
CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES reports the device's overall
frame rate bounds, which can be higher than the streaming rate that a given
resolution supports. To get the actual maximum for each resolution on audio and
display glasses, read its minimum frame duration and convert it to FPS with
StreamConfigurationMap.getOutputMinFrameDuration(format, size):
@androidx.annotation.OptIn(ExperimentalCamera2Interop::class)
fun getMaxFpsBySize(cameraInfo: CameraInfo, format: Int): Map<Size, Int> {
val map = Camera2CameraInfo.from(cameraInfo).getCameraCharacteristic(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP
) ?: return emptyMap()
return map.getOutputSizes(format).associateWith { size ->
val minFrameDurationNs = map.getOutputMinFrameDuration(format, size)
if (minFrameDurationNs > 0) (1_000_000_000L / minFrameDurationNs).toInt() else 0
}
}<br />
Next, use that value to calculate the minFps and maxFps bounds for your
target frame rate range. Cap the lower bound (minFps) at the resolution's
maximum streaming frame rate calculated from getOutputMinFrameDuration, and
set a higher upper bound (maxFps) as a catch-up window:
val maxFpsBySize = getMaxFpsBySize(cameraInfo, ImageFormat.YUV_420_888)
val maxSupportedFps = maxFpsBySize[Size(640, 480)] ?: return ImageAnalysis.Builder().build()
// minFps is the target streaming frame rate and should be at most maxSupportedFps.
val minFps = minOf(desiredFps, maxSupportedFps)
// Set the upper bound to 2x minFps so the stream can catch up if frames are delayed.
val maxFps = minFps * 2<br />
Key points about the code
Use the format that matches your use case. For
ImageAnalysis, that'sYUV_420_888.For
VideoCapture, use theClassoverloads instead:getOutputSizes(MediaRecorder::class.java)andgetOutputMinFrameDuration(MediaRecorder::class.java, size).Specify a range where
maxFpsis greater thanminFps(such asminFps*2) rather than setting both bounds to the same value likeRange(minFps, minFps). The lower bound (minFps) sets the target streaming frame rate and should be at most the value calculated fromgetOutputMinFrameDuration, while the upper bound (maxFps) acts as a catch-up window for delayed frames to prevent persistent latency. For example, [1,2] or [1,3] for a 1 FPS stream, and [24,30] or [24,48] for a 24 FPS stream.
[!NOTE] Note: Using the lower bound as the streaming frame rate reflects current device behavior and might change on future devices.
Configure the frame rate
To configure the frame rate range based on your CameraX use case:
// Configure Resolution Strategy for the stream
val resolutionStrategy = ResolutionStrategy(
size,
ResolutionStrategy.FALLBACK_RULE_CLOSEST_LOWER
)
val resolutionSelector = ResolutionSelector.Builder()
.setResolutionStrategy(resolutionStrategy)
.build()
// Create and return the ImageAnalysis builder with proper frame rate ranges
// ImageAnalysis: OCR, AI streaming, and VPS
val analysisBuilder = ImageAnalysis.Builder()
.setResolutionSelector(resolutionSelector)
Camera2Interop.Extender(analysisBuilder)
.setCaptureRequestOption(
CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE,
Range(minFps, maxFps)
)
return analysisBuilder.build()<br />
This example shows configuration for both VideoCapture and ImageAnalysis,
but you can implement whichever use cases your app supports.
Understand how the stream behaves
Keep the following system behaviors in mind while working with capture streams:
Unsupported resolutions are rejected: If you request a size that isn't in the list, Camera2 rejects the configuration before the stream opens.
Requesting frame rates above the maximum: If you request a minimum frame rate above the maximum supported for your resolution, the session still opens, but the target frame rate from the glasses is clamped to that supported maximum while the upper bound continues to act as the catch-up rate limit. For example, requesting [30,60] at 640 x 480 clamps the target frame rate to 24 FPS (behaving like [24,60]).
Thermal throttling: When the glasses get warm, the frame rate drops but the resolution stays the same. Build your app to handle lower or uneven frame rates, for example by checking frame timestamps rather than assuming a fixed rate.
Access a phone's hardware from a projected activity
A projected activity can also access the phone's hardware (such as the
camera or microphone) by using createHostDeviceContext(context) to get
the host device's (phone) context:
@OptIn(ExperimentalProjectedApi::class)
private fun getPhoneContext(activity: ComponentActivity): Context? {
return try {
// From a projected Activity, get a context for the phone.
ProjectedContext.createHostDeviceContext(activity)
} catch (e: IllegalStateException) {
Log.e(TAG, "Failed to create host device context", e)
null
}
}<br />
When accessing hardware or resources that are specific to the host device (phone) in a hybrid app (an app containing both mobile and glasses experiences), you must explicitly select the correct context to make sure your app can access the correct hardware:
Use the
Activitycontext from the phoneActivityor theProjectedContext.createHostDeviceContextto get the phone's context.Don't use
getApplicationContextbecause the application context can incorrectly return the glasses' context if a projected activity was the most-recently-launched component.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Security
No issues found by our static scan· No issues found
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- Google LLC
- Publisher
- Android (Google) (Official)
- Distributed by
- Android (Google)
- Source
- android/skills · xr/display-glasses-with-jetpack-compose-glimmer · commit bf68e2d
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE.txt)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by Google LLC or Android (Google).
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download display-glasses-with-jetpack-compose-glimmer.zip