Zoom Rtms

OpenAIOfficialCreated by ZoomDistributed by OpenAI

Skill
Tool integration
Automation and integrations
MIT
No issues found

Download ZIP

From the author

Use when using RTMS.

— the description by Zoom

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

The zoom-rtms skill provides the necessary protocols and architectural guidance for integrating Zoom Realtime Media Streams. It covers the WebSocket-based connection lifecycle, including signaling and media handshakes, heartbeat management, and reconnection strategies. The skill supports processing various media types such as audio, video, screen sharing, transcripts, and chat, while providing reference data for message types and event enums.

What it does

Establishes WebSocket connections for signaling and media streams.
Handles media handshake requests for audio, video, screen share, transcript, and chat data.
Provides reference tables for RTMS message types and event enums.
Defines mandatory heartbeat and reconnection logic for stream stability.
Supports configuration for transcript language identification.
Outlines the process for extracting stream IDs and generating HMAC-SHA256 signatures.

When to use it

When you need to ingest raw meeting or contact center media into a backend pipeline.
When building services for transcription, AI analysis, or recording of Zoom sessions.
When you need to subscribe to specific meeting events or participant media streams.
When you require programmatic access to media data without a visible participant bot.

Advantages

Provides a structured approach to managing the complex WebSocket lifecycle.
Includes comprehensive references for message types and event handling.
Offers clear guidance on signature generation and security requirements.
Enables granular control over media subscription and transcript settings.

Usage examples

“How do I initiate a connection to an RTMS stream?”: The agent will guide you through receiving the rtms_started webhook, generating an HMAC-SHA256 signature, and performing the signaling and media handshakes.
“What is the correct way to handle heartbeats in RTMS?”: The agent will explain that you must respond to msg_type 12 (Keep Alive Request) with msg_type 13 (Keep Alive Response) to maintain the connection.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/openai/zoom-rtms/download
Unzip it into the cross-client skills folder of your project:
```
mkdir -p .agents/skills && unzip zoom-rtms.zip -d .agents/skills
```

Documentation by Zoom

Zoom Realtime Media Streams

Use this skill when the integration needs real-time meeting or contact-center media as a backend stream. If the workflow needs a visible participant bot, compare against build-zoom-bot and zoom-meeting-sdk-linux before choosing RTMS.

Workflow

Confirm the source surface: Meetings RTMS, Contact Center voice media, or another documented RTMS stream.
Decide whether RTMS is sufficient or whether a Meeting SDK bot is required for participant identity, meeting controls, or UI-visible behavior.
Implement the WebSocket lifecycle first: event subscription, connection validation, media start, heartbeat, reconnect, and shutdown.
Process media types intentionally: audio, video, screen share, chat, live transcript, and metadata have different payload and timing constraints.
Design downstream pipelines for buffering, transcription, AI analysis, recording, or tool invocation.
Debug by isolating app setup, webhook/event delivery, stream authorization, network connectivity, and media payload decoding.

References

Full preserved guide: references/full-guide.md
Connection architecture: concepts/connection-architecture.md
Lifecycle flow: concepts/lifecycle-flow.md
Media types: references/media-types.md
Data types: references/data-types.md
RTMS bot example: examples/rtms-bot.md
Common issues: troubleshooting/common-issues.md

Files

ATTRIBUTION.mdAdded by AutoGineer860 B
LICENSELicence file added by AutoGineer1.1 KiB
RUNBOOK.md2.6 KiB
SKILL.md1.6 KiB
agents/openai.yaml140 B
concepts/connection-architecture.md9.1 KiB
concepts/lifecycle-flow.md13.4 KiB
examples/ai-integration.md11.3 KiB
examples/manual-websocket.md16.8 KiB
examples/rtms-bot.md25.9 KiB
examples/sdk-quickstart.md9.3 KiB
references/connection.md8.0 KiB
references/data-types.md13.7 KiB
references/environment-variables.md1.3 KiB
references/full-guide.md22.9 KiB
references/media-types.md6.0 KiB
references/quickstart.md6.6 KiB
references/webhooks.md7.4 KiB
troubleshooting/common-issues.md10.4 KiB

examples/ai-integration.md

AI Integration Patterns

Patterns for integrating RTMS with AI services for transcription, analysis, and meeting assistants. These examples work with meetings, webinars, and Video SDK sessions.

Audio Transcription with External Services

Deepgram Integration

import rtms from "@zoom/rtms";
import { createClient } from "@deepgram/sdk";

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];
const deepgram = createClient(process.env.DEEPGRAM_API_KEY);

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();

  // Configure for Deepgram-compatible audio
  client.setAudioParams({
    codec: 1,          // L16 (PCM)
    sampleRate: 1,     // 16kHz
    channel: 1,        // Mono
    dataOpt: 1         // Mixed stream
  });

  // Create live transcription connection
  const connection = deepgram.listen.live({
    model: "nova-2",
    language: "en",
    smart_format: true,
    punctuate: true,
  });

  connection.on("Results", (data) => {
    const transcript = data.channel.alternatives[0].transcript;
    if (transcript) {
      console.log(`[Deepgram]: ${transcript}`);
    }
  });

  client.onAudioData((buffer, timestamp, metadata) => {
    // Send audio to Deepgram
    connection.send(buffer);
  });

  client.onLeave(() => {
    connection.finish();
  });

  client.join(payload);
});

AssemblyAI Integration

import rtms from "@zoom/rtms";
import { AssemblyAI } from "assemblyai";

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];
const aai = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();

  client.setAudioParams({
    codec: 1,          // L16 (PCM)
    sampleRate: 1,     // 16kHz
    channel: 1         // Mono
  });

  const transcriber = aai.realtime.createService({
    sampleRate: 16000,
  });

  transcriber.connect();

  transcriber.on("transcript", (transcript) => {
    if (transcript.text) {
      console.log(`[AssemblyAI]: ${transcript.text}`);
    }
  });

  client.onAudioData((buffer, timestamp, metadata) => {
    transcriber.sendAudio(buffer);
  });

  client.onLeave(() => {
    transcriber.close();
  });

  client.join(payload);
});

Whisper (Local) Integration

import rtms from "@zoom/rtms";
import { Whisper } from "whisper-node";

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];
const whisper = new Whisper("base.en");

let audioBuffer = Buffer.alloc(0);
const BUFFER_SIZE = 16000 * 10; // 10 seconds at 16kHz

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();

  client.setAudioParams({
    codec: 1,          // L16 (PCM)
    sampleRate: 1,     // 16kHz
    channel: 1         // Mono
  });

  client.onAudioData(async (buffer, timestamp, metadata) => {
    // Accumulate audio
    audioBuffer = Buffer.concat([audioBuffer, buffer]);

    // Transcribe when buffer is full
    if (audioBuffer.length >= BUFFER_SIZE) {
      const transcript = await whisper.transcribe(audioBuffer);
      console.log(`[Whisper]: ${transcript}`);
      audioBuffer = Buffer.alloc(0);
    }
  });

  client.join(payload);
});

Meeting Summarization

OpenAI/GPT Integration

import rtms from "@zoom/rtms";
import OpenAI from "openai";

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const transcripts = [];
let summaryInterval;

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();

  client.onTranscriptData((buffer, timestamp, metadata) => {
    const text = buffer.toString('utf8');
    transcripts.push({
      speaker: metadata.userName,
      text: text,
      time: new Date(timestamp)
    });
  });

  // Generate summary every 5 minutes
  summaryInterval = setInterval(async () => {
    if (transcripts.length === 0) return;

    const fullTranscript = transcripts
      .map(t => `${t.speaker}: ${t.text}`)
      .join('\n');

    const summary = await openai.chat.completions.create({
      model: "gpt-4",
      messages: [
        {
          role: "system",
          content: "Summarize this meeting transcript. Include key points, decisions, and action items."
        },
        {
          role: "user",
          content: fullTranscript
        }
      ]
    });

    console.log("Meeting Summary:", summary.choices[0].message.content);
  }, 5 * 60 * 1000);

  client.onLeave(async () => {
    clearInterval(summaryInterval);

    // Generate final summary
    const fullTranscript = transcripts
      .map(t => `${t.speaker}: ${t.text}`)
      .join('\n');

    const summary = await openai.chat.completions.create({
      model: "gpt-4",
      messages: [
        {
          role: "system",
          content: `Create a comprehensive meeting summary with:
- Key topics discussed
- Decisions made
- Action items with owners
- Follow-up items`
        },
        {
          role: "user",
          content: fullTranscript
        }
      ]
    });

    console.log("Final Summary:", summary.choices[0].message.content);
  });

  client.join(payload);
});

Real-Time Sentiment Analysis

import rtms from "@zoom/rtms";

async function analyzeSentiment(text) {
  // Use any sentiment API (OpenAI, HuggingFace, etc.)
  const response = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      model: 'gpt-3.5-turbo',
      messages: [{
        role: 'user',
        content: `Analyze sentiment (positive/neutral/negative): "${text}"`
      }]
    })
  });

  const data = await response.json();
  return data.choices[0].message.content;
}

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();
  let recentTranscripts = [];

  client.onTranscriptData(async (buffer, timestamp, metadata) => {
    const text = buffer.toString('utf8');
    recentTranscripts.push(text);

    // Analyze every 10 segments
    if (recentTranscripts.length >= 10) {
      const combinedText = recentTranscripts.join(' ');
      const sentiment = await analyzeSentiment(combinedText);
      console.log(`Sentiment: ${sentiment}`);
      recentTranscripts = [];
    }
  });

  client.join(payload);
});

Audio Recording with Gap Filling

For continuous playback, fill audio gaps with silence:

import rtms from "@zoom/rtms";
import fs from 'fs';

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];
const SAMPLE_RATE = 16000;
const BYTES_PER_SAMPLE = 2; // 16-bit
const MS_PER_FRAME = 20;
const BYTES_PER_FRAME = SAMPLE_RATE * BYTES_PER_SAMPLE * MS_PER_FRAME / 1000;

function generateSilentFrame(durationMs) {
  const samples = SAMPLE_RATE * durationMs / 1000;
  return Buffer.alloc(samples * BYTES_PER_SAMPLE);
}

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();
  const streamId = payload.rtms_stream_id;

  const audioStream = fs.createWriteStream(`recordings/${streamId}.pcm`);
  let lastTimestamp = null;

  client.setAudioParams({
    codec: 1,          // L16 (PCM)
    sampleRate: 1,     // 16kHz
    channel: 1,        // Mono
    dataOpt: 1,        // Mixed stream
    duration: 20       // 20ms chunks
  });

  client.onAudioData((buffer, timestamp, metadata) => {
    if (lastTimestamp !== null) {
      const gap = timestamp - lastTimestamp;

      // Fill gaps >= 500ms with silence
      if (gap >= 500) {
        const silentFrames = Math.floor(gap / MS_PER_FRAME);
        console.log(`Gap detected: ${gap}ms, filling ${silentFrames} frames`);

        for (let i = 0; i < silentFrames; i++) {
          audioStream.write(generateSilentFrame(MS_PER_FRAME));
        }
      }
    }

    lastTimestamp = timestamp;
    audioStream.write(buffer);
  });

  client.onLeave(() => {
    audioStream.end();
    console.log(`Recording saved: recordings/${streamId}.pcm`);
  });

  client.join(payload);
});

Multi-Format Transcript Output

Generate VTT, SRT, and TXT simultaneously:

import rtms from "@zoom/rtms";
import fs from 'fs';

const RTMS_EVENTS = ["meeting.rtms_started", "webinar.rtms_started", "session.rtms_started"];

function formatVttTimestamp(ms) {
  const s = Math.floor(ms / 1000);
  const m = Math.floor(s / 60);
  const h = Math.floor(m / 60);
  const msec = ms % 1000;
  return `${String(h).padStart(2, '0')}:${String(m % 60).padStart(2, '0')}:${String(s % 60).padStart(2, '0')}.${String(msec).padStart(3, '0')}`;
}

function formatSrtTimestamp(ms) {
  return formatVttTimestamp(ms).replace('.', ',');
}

rtms.onWebhookEvent(({ event, payload }) => {
  if (!RTMS_EVENTS.includes(event)) return;

  const client = new rtms.Client();
  const streamId = payload.rtms_stream_id;

  const baseDir = `recordings/${streamId}`;
  fs.mkdirSync(baseDir, { recursive: true });

  fs.writeFileSync(`${baseDir}/transcript.vtt`, 'WEBVTT\n\n');
  let srtIndex = 1;
  let startTimestamp = null;

  client.onTranscriptData((buffer, timestamp, metadata) => {
    const text = buffer.toString('utf8');
    const userName = metadata.userName;

    if (startTimestamp === null) {
      startTimestamp = timestamp;
    }

    const relative = timestamp - startTimestamp;
    const endTime = relative + 2000; // 2 second duration

    // VTT format
    const vttLine = `${formatVttTimestamp(relative)} --> ${formatVttTimestamp(endTime)}\n${userName}: ${text}\n\n`;
    fs.appendFileSync(`${baseDir}/transcript.vtt`, vttLine);

    // SRT format
    const srtLine = `${srtIndex++}\n${formatSrtTimestamp(relative)} --> ${formatSrtTimestamp(endTime)}\n${userName}: ${text}\n\n`;
    fs.appendFileSync(`${baseDir}/transcript.srt`, srtLine);

    // Plain text
    const txtLine = `[${new Date(timestamp).toISOString()}] ${userName}: ${text}\n`;
    fs.appendFileSync(`${baseDir}/transcript.txt`, txtLine);
  });

  client.join(payload);
});

Environment Variables

# Zoom RTMS
ZM_RTMS_CLIENT=your_client_id
ZM_RTMS_SECRET=your_client_secret

# AI Services
OPENAI_API_KEY=sk-...
DEEPGRAM_API_KEY=...
ASSEMBLYAI_API_KEY=...

# OpenRouter (free models)
OPENROUTER_API_KEY=sk-or-...

Free AI Model Considerations

When using free models (Gemma, Qwen, DeepSeek via OpenRouter):

Limitation	Impact	Solution
No image support	Can't analyze screen shares	Use paid model or skip image analysis
Context limits	Long transcripts may fail	Chunk transcripts, summarize incrementally
Rate limiting	May get 429 errors	Implement retry with backoff, stagger requests

Recommended for production: OpenRouter with google/gemini-2.5-pro - supports vision + XML tagging.

Next Steps

SDK Quickstart - Basic RTMS setup
Manual WebSocket - Protocol details
Media Types - Audio/video configuration

Requirements and compatibility

Written for: OpenAI Codex

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Zoom

Publisher

OpenAI (Official)

Distributed by

OpenAI

Source

openai/plugins · plugins/zoom/skills/rtms · commit d420d13

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from plugins/zoom/LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Zoom or OpenAI.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download zoom-rtms.zip

From the author

AutoGineer summary

What it does

When to use it

Advantages

Usage examples

Install

Documentation by Zoom

Zoom Realtime Media Streams

Workflow

References

Files

examples/ai-integration.md

AI Integration Patterns

Audio Transcription with External Services

Deepgram Integration

AssemblyAI Integration

Whisper (Local) Integration

Meeting Summarization

OpenAI/GPT Integration

Real-Time Sentiment Analysis

Audio Recording with Gap Filling

Multi-Format Transcript Output

Environment Variables

Free AI Model Considerations

Next Steps

Requirements and compatibility

Security

Attribution and licence

Related skills

Build MCP App

MCP Integration

Social Publishing

Hermes Tweet

Paypal Integration

Social Publishing Publisher

Download ZIP