AutoGineer
Generated banner for Bigquery Basics
Generated logo for Bigquery Basics

Bigquery Basics

GoogleOfficialPublished by Google

  • Skill
  • Tool integration
  • Cloud and DevOps
  • Apache-2.0
  • Low risk
Download ZIP

From the author

Manages datasets, tables, and jobs in BigQuery. Use when you need to interact with BigQuery, run SQL queries, manage BigQuery resources (datasets, tables, views), or perform basic data ingestion and analysis.
— the description by Google

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

BigQuery Basics provides a structured approach to interacting with Google BigQuery, a serverless data platform. This skill covers essential resource management, including creating datasets and tables, executing SQL queries, and tracking data changes. It emphasizes standardized usage attribution for all CLI and API interactions to ensure proper telemetry within agent environments.

What it does

  • Enables the BigQuery API for cloud projects.
  • Creates and manages BigQuery datasets and tables.
  • Executes SQL queries against public and private datasets.
  • Tracks incremental table changes using APPENDS and CHANGES functions.
  • Provides guidance on IAM, security, and infrastructure-as-code practices.
  • Implements mandatory usage attribution for CLI and API requests.

When to use it

  • When you need to perform data ingestion or analysis within BigQuery.
  • When managing BigQuery resources like datasets, tables, or views via CLI.
  • When you need to audit or process incremental data changes in a table.
  • When integrating BigQuery operations into automated workflows or infrastructure scripts.

Advantages

  • Standardized command attribution for better observability.
  • Comprehensive coverage of core BigQuery resource management.
  • Includes advanced features like change history tracking.
  • Provides clear examples for both CLI and programmatic access.

Usage examples

“Create a new BigQuery dataset named sales_data in the US location.”
The agent executes the bq mk command with the appropriate flags to create the dataset in the specified region.
“Run a SQL query to select all rows from my_dataset.orders where the status is 'complete'.”
The agent uses the bq query command with --use_legacy_sql=false to execute the SQL statement and return the results.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/google/bigquery-basics/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip bigquery-basics.zip -d .agents/skills

Documentation by Google

BigQuery Basics

BigQuery is a serverless, AI-ready data platform that enables high-speed analysis of large datasets using SQL and Python. Its disaggregated architecture separates compute and storage, allowing them to scale independently while providing built-in machine learning, geospatial analysis, and business intelligence capabilities.

Attribution

MUST READ: Before executing any gcloud, REST API, or MCP command, you MUST read Usage Attribution. For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix gcloud with CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:bigquery-basics)" and append agent-skills/1.0 (skill:bigquery-basics) to User-Agent headers.

Setup and Basic Usage

  1. Enable the BigQuery API:

    CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:bigquery-basics)" \
gcloud services enable bigquery.googleapis.com --quiet

  2. Create a Dataset:

    bq mk --dataset --location=US my_dataset

  3. Create a Table:

    Create a file named schema.json with your table schema:

    [
  {
    "name": "name",
    "type": "STRING",
    "mode": "REQUIRED"
  },
  {
    "name": "post_abbr",
    "type": "STRING",
    "mode": "NULLABLE"
  }
]

    Then create the table with the bq tool:

    bq mk --table my_dataset.mytable schema.json

  4. Run a Query:

    bq query --use_legacy_sql=false \
'SELECT name FROM `bigquery-public-data.usa_names.usa_1910_2013` \
WHERE state = "TX" LIMIT 10'

Reference Directory

  • Core Concepts: Storage types, analytics workflows, and BigQuery Studio features.

  • Change History: Tracking and querying incremental table changes using APPENDS and CHANGES.

  • Continuous Queries: Running continuous SQL statements to analyze incoming data in real time.

  • CLI Usage: Essential bq command-line tool operations for managing data and jobs.

  • Client Libraries: Using Google Cloud client libraries for Python, Java, Node.js, and Go.

  • MCP Usage: Using the BigQuery remote MCP server and Gemini CLI extension.

  • Infrastructure as Code: Terraform examples for datasets, tables, and reservations.

  • IAM & Security: Roles, permissions, and data governance best practices.

  • Usage Attribution: Standardized usage attribution tags for gcloud CLI, REST API, and MCP requests.

If you need product information not found in these references, use the Developer Knowledge MCP server search_documents tool.

Related Skills

  • BigQuery AI & ML Skill: SKILL.md file for BigQuery AI and ML capabilities (forecast, anomaly detection, text generation).

Files

references/continuous-queries.md

BigQuery Continuous Queries

BigQuery continuous queries are SQL statements that run continuously in an unbounded fashion. They let you analyze incoming data in BigQuery in real time.

You can output the results of a continuous query in several ways:

  • Write to a BigQuery table by using an INSERT statement.

  • Export to Pub/Sub, Bigtable, or Spanner by using an EXPORT DATA statement.

Use Cases

Continuous queries turn BigQuery into an event-driven data processing engine, unlocking real-time capabilities:

  • Event-Driven Workflows & Agentic Systems: You can trigger downstream applications or autonomous agents based on complex events detected in incoming data streams. For example, integrate with Pub/Sub to send real-time events to downstream agentic systems for further processing.

  • Real-Time AI Inference: Apply generative AI models directly on live data streams to generate text or embeddings on the fly, enabling personalized customer interactions or real-time anomaly detection.

  • Reverse ETL: Seamlessly push enhanced event data from BigQuery directly to operational databases like Spanner or Bigtable for low-latency application serving.

Syntax and Usage

To run a continuous query, you must specify the earliest data to process using the APPENDS function (or CHANGES for certain Pub/Sub exports) in the FROM clause.

The start timestamp defines the point in time at which the continuous query begins processing data.

Example: Writing to a BigQuery Table
INSERT INTO `myproject.real_time_taxi_streaming.transformed_taxirides`
SELECT
  timestamp,
  meter_reading,
  ride_status
FROM
  APPENDS(TABLE `myproject.real_time_taxi_streaming.taxirides`,
    CURRENT_TIMESTAMP() - INTERVAL 10 MINUTE)
WHERE
  ride_status = 'dropoff';
Example: Writing to a Pub/Sub Topic
EXPORT DATA
  OPTIONS (
    format = 'CLOUD_PUBSUB',
    uri = 'https://pubsub.googleapis.com/projects/myproject/topics/taxi-real-time-rides')
AS (
  SELECT
    TO_JSON_STRING(
      STRUCT(
        ride_id,
        timestamp,
        latitude,
        longitude)) AS message,
    TO_JSON(
      STRUCT(
        CAST(passenger_comment AS STRING) AS passenger_comment))
  FROM
    CHANGES(TABLE `myproject.real_time_taxi_streaming.taxi_rides`,
      CURRENT_TIMESTAMP() - INTERVAL 10 MINUTE)
  WHERE _CHANGE_TYPE = 'DELETE'
);

Important Considerations & Limitations

  • Authorization: A continuous query run by a user account runs for a maximum of two days and then automatically stops. To run a continuous query for up to 150 days, you must use a service account.

  • Reservations: Running continuous queries requires an Enterprise edition or Enterprise Plus edition reservation with a CONTINUOUS job type assignment.

  • Supported Operations: Continuous queries support a limited set of stateful operations, such as specific types of JOINs, aggregations, and windowing functions. Many standard SQL capabilities like SELECT DISTINCT, PIVOT, and subqueries like EXISTS are not supported unless part of a supported stateful operation.

For more detail on how to use or structure continuous queries, please refer to the public documentation for BigQuery continuous queries:

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs an MCP server, Needs a command-line tool

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by
Google
Publisher
Google (Official)
Source
google/skills · skills/cloud/bigquery-basics · commit 2194d3b
Licence
Apache License 2.0 (redistribution allowed)
  • LICENSE (added by AutoGineer from LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced

AutoGineer is not affiliated with or endorsed by Google or Google.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download bigquery-basics.zip