# Chat Completions Source: https://docs.cloudglue.dev/api-reference/endpoint/chat/completions POST /chat/completions Generate a model response to a conversation that can include references to video content # List Chat Completions Source: https://docs.cloudglue.dev/api-reference/endpoint/chat/completions-get GET /chat/completions List all chat completions with optional filtering # Delete Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/delete DELETE /collections/{collection_id} Delete a collection # Get Collection Video Info Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/file-get GET /collections/{collection_id}/videos/{file_id} Retrieve information about a specific video file in a collection # Remove Video from Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/files-delete DELETE /collections/{collection_id}/videos/{file_id} Remove a video file from a collection # List Videos in Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/files-get GET /collections/{collection_id}/videos List all files in a collection # Add Video to Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/files-post POST /collections/{collection_id}/videos **DEPRECATED: Use POST /collections/\{collection_id\}/media instead.** Add a video to a collection. This endpoint is deprecated - use the /media endpoint which supports both video and audio files. # List Collections Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/get GET /collections List all collections # Get Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/get-id GET /collections/{collection_id} Retrieve details about a specific collection # Add Media to Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/media-post POST /collections/{collection_id}/media Add a video or audio file to a collection. This is the recommended endpoint for adding media files to collections. **Media Type Handling:** - **Video files**: Processed with full visual analysis (scene description, text extraction, etc.) - **Audio files**: Visual features automatically disabled; only speech and audio analysis available **Audio File Restrictions:** - Audio files cannot be added to face-analysis collections - Shot-detector segmentation is not available for audio files # Create Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/post POST /collections Create a new collection to organize and process video files. Collections are used to group files together and process them in a consistent way. # Update Collection Source: https://docs.cloudglue.dev/api-reference/endpoint/collections/put-id PUT /collections/{collection_id} Update a collection # Cancel Deep Search Source: https://docs.cloudglue.dev/api-reference/endpoint/deep-search/cancel POST /deepSearch/{id}/cancel Cancel a background deep search that is in progress. If the deep search is already completed, failed, or cancelled, this returns the deep search as-is. # Delete Deep Search Source: https://docs.cloudglue.dev/api-reference/endpoint/deep-search/delete DELETE /deepSearch/{id} Delete a deep search by ID. This operation is idempotent - deleting a non-existent deep search returns success. # List Deep Searches Source: https://docs.cloudglue.dev/api-reference/endpoint/deep-search/get GET /deepSearch List all deep searches with pagination and filtering options. # Get Deep Search Source: https://docs.cloudglue.dev/api-reference/endpoint/deep-search/get-id GET /deepSearch/{id} Retrieve a specific deep search by its ID. # Create Deep Search Source: https://docs.cloudglue.dev/api-reference/endpoint/deep-search/post POST /deepSearch Create a new deep search over one or more collections. Deep search uses agentic retrieval and LLM summarization to find specific moments across your video data. The search can be processed synchronously (default), streamed via SSE, or run in the background. # Delete Media Description Source: https://docs.cloudglue.dev/api-reference/endpoint/describe/delete DELETE /describe/{job_id} Delete a media description job # Get Media Description Source: https://docs.cloudglue.dev/api-reference/endpoint/describe/get GET /describe/{job_id} Retrieve the current state of a media description job # List Media Descriptions Source: https://docs.cloudglue.dev/api-reference/endpoint/describe/list GET /describe List all media description jobs with optional filtering # Update Describe Job Source: https://docs.cloudglue.dev/api-reference/endpoint/describe/patch PATCH /describe/{job_id} Toggle the `use_in_default_index` flag on an existing describe job. Enabling this makes the file searchable by default in the deep search and response APIs. # Describe Video Source: https://docs.cloudglue.dev/api-reference/endpoint/describe/post POST /describe Get a comprehensive multimodal description of a video # Delete Extraction Source: https://docs.cloudglue.dev/api-reference/endpoint/extract/delete DELETE /extract/{job_id} Delete an extraction job # Get Extraction Source: https://docs.cloudglue.dev/api-reference/endpoint/extract/get GET /extract/{job_id} Retrieve the current state of an extraction job. Results are paginated with a default limit of 50 segment entities per request (maximum 100). Use limit and offset parameters to paginate through all segment entities. # List Extraction Jobs Source: https://docs.cloudglue.dev/api-reference/endpoint/extract/list GET /extract List all extract jobs with optional filtering # Extract from Video Source: https://docs.cloudglue.dev/api-reference/endpoint/extract/post POST /extract Extract structured data from a video # Delete File Source: https://docs.cloudglue.dev/api-reference/endpoint/files/delete DELETE /files/{file_id} Delete a file # List Files Source: https://docs.cloudglue.dev/api-reference/endpoint/files/get GET /files List files that have been uploaded to Cloudglue # Get File Source: https://docs.cloudglue.dev/api-reference/endpoint/files/get-id GET /files/{file_id} Retrieve details about a specific file # Upload File Source: https://docs.cloudglue.dev/api-reference/endpoint/files/post POST /files Upload a video, audio, or image file that can be used with Cloudglue services # Update File Source: https://docs.cloudglue.dev/api-reference/endpoint/files/put-id PUT /files/{file_id} Update a file # Sync File from URL Source: https://docs.cloudglue.dev/api-reference/endpoint/files/sync POST /files/sync Materialize a publicly accessible URL into a Cloudglue file without a data connector or a collection. Accepts direct http(s) video/audio file URLs (e.g. `.mp4`), public Dropbox share links, TikTok video URLs, and Loom share URLs. Idempotent: syncing the same URL returns the existing file. YouTube URLs are not supported here — add them to a collection via the add-media endpoint instead. Connector-native URIs (e.g. `s3://`, `gs://`, `gdrive://`, Zoom/Gong/Recall/Grain links) require the matching data connector — use POST /data-connectors/{id}/sync. # Refresh Source Metadata Source: https://docs.cloudglue.dev/api-reference/endpoint/files/sync-source-metadata POST /files/{file_id}/sync Re-fetches the file's source_metadata live from its data connector (google-drive, dropbox, zoom, gong, recall, grain, or iconik) and updates the stored value. If the file belongs to any metadata collections, their search documents are re-indexed with the refreshed metadata. For iconik files without a thumbnail, the asset's poster keyframe is also copied and set as the file's thumbnail_url. Works for both metadata-only files and fully ingested connector files. Free — no media is downloaded or processed. # Delete Find Moments Run Source: https://docs.cloudglue.dev/api-reference/endpoint/find-moments/delete DELETE /find-moments/{job_id} Deletes the run and its moments and findings. An in-flight run is cancelled and refunded; a completed run is not refunded. # Get Find Moments Run Source: https://docs.cloudglue.dev/api-reference/endpoint/find-moments/get GET /find-moments/{job_id} Returns the run; when completed, includes moments and findings shaped by the read parameters. Selection never destroys accepted results: total_moments is always the full accepted count. # List Find Moments Runs Source: https://docs.cloudglue.dev/api-reference/endpoint/find-moments/list GET /find-moments Lists runs, newest first, with cursor pagination. Pagination is over runs, never over joined moment rows. # Find Moments Source: https://docs.cloudglue.dev/api-reference/endpoint/find-moments/post POST /find-moments Runs exhaustive moment discovery for one criterion over one video. Every accepted moment is persisted; ranking and truncation are read-time parameters on GET. Reuses a compatible describe or creates one internally - a missing describe is never an error. # Cancel Query Export Source: https://docs.cloudglue.dev/api-reference/endpoint/query/cancel POST /query/{id}/cancel Cancel an in-progress background export. The run is marked cancelled synchronously, the export stream is aborted mid-flight (the partial upload is discarded), and the reserved credits are refunded. A run that has already completed or failed is returned unchanged. # Get Query Result Source: https://docs.cloudglue.dev/api-reference/endpoint/query/get GET /query/{id} Retrieve a stored query run by its ID, including its result rows. Results larger than the inline storage cap are replayed truncated (truncated: true). # List Query Results Source: https://docs.cloudglue.dev/api-reference/endpoint/query/list GET /query List all query runs with pagination and filtering options. List items omit the columns and rows payloads — fetch an individual run via GET /query/{id} for the full result. # Run Structured Query Source: https://docs.cloudglue.dev/api-reference/endpoint/query/post POST /query Run a synchronous read-only SQL query over the structured data extracted from one or more collections. Queries execute against three virtual tables — files, entities, and segment_entities — built from each file's most recent completed extraction. Each query costs 2 credits. Results are returned inline and stored, so completed runs can be re-fetched via GET /query/{id}. Use GET /query/schema to discover the queryable tables and per-collection fields. # Get Query Schema Source: https://docs.cloudglue.dev/api-reference/endpoint/query/schema GET /query/schema Introspect the virtual tables and per-collection extracted fields available to SQL queries over the given collections. Use this to discover column names, entity field names, types, and levels, plus each collection's verbatim extract schema and prompt, before writing a query. Fields with level 'file' appear as rows in the entities table; fields with level 'segment' live inside the segment_entities.entities JSON column. # Cancel Response Source: https://docs.cloudglue.dev/api-reference/endpoint/responses/cancel POST /responses/{id}/cancel Cancel a background response that is in progress. If the response is already completed, failed, or cancelled, this returns the response as-is. # Delete Response Source: https://docs.cloudglue.dev/api-reference/endpoint/responses/delete DELETE /responses/{id} Delete a response by ID. This operation is idempotent - deleting a non-existent response returns success. # List Responses Source: https://docs.cloudglue.dev/api-reference/endpoint/responses/get GET /responses List all responses with pagination and filtering options. # Get Response Source: https://docs.cloudglue.dev/api-reference/endpoint/responses/get-id GET /responses/{id} Retrieve a specific response by its ID. # Create Response Source: https://docs.cloudglue.dev/api-reference/endpoint/responses/post POST /responses Create a new response using the Response API. This endpoint provides an OpenAI Responses-compatible interface for chat completions with video collections. The response can be processed synchronously (default) or asynchronously using the `background` parameter. # List Search Responses Source: https://docs.cloudglue.dev/api-reference/endpoint/search/get GET /search Get a list of search responses. Order by `created_at` in descending order by default. # Get Search Response Source: https://docs.cloudglue.dev/api-reference/endpoint/search/get-id GET /search/{search_id} Get a search response by search_id. # Search Content Source: https://docs.cloudglue.dev/api-reference/endpoint/search/post POST /search Search for videos or video segments in collections to find relevant videos or moments/clips in a video # Delete Segments Source: https://docs.cloudglue.dev/api-reference/endpoint/segments/delete DELETE /segments/{job_id} Delete a specific segments job # Get Segments Source: https://docs.cloudglue.dev/api-reference/endpoint/segments/get GET /segments/{job_id} Retrieve the current state of a segmentation job # List Segments Jobs Source: https://docs.cloudglue.dev/api-reference/endpoint/segments/list GET /segments List all segmentation jobs with optional filtering # Segment Video Source: https://docs.cloudglue.dev/api-reference/endpoint/segments/post POST /segments Create intelligent segments for video or audio files based on shot detection or narrative analysis. **Audio File Support:** - Audio files support **narrative** criteria only (shot detection is not available for audio). - Audio files default to the 'balanced' strategy and may opt into 'transcript'. **Note: YouTube URLs and audio files are supported for narrative-based segmentation only.** Shot-based segmentation requires direct video file access. Use Cloudglue Files, HTTP URLs, or files from data connectors for shot-based segmentation. **Narrative Segmentation Strategies:** - **comprehensive** (default for non-YouTube/non-audio files): Uses a VLM to deeply analyze logical segments of video. Only available for video files (not YouTube or audio). - **balanced** (default for YouTube videos and audio files): Balanced analysis approach using multiple modalities. Supports YouTube URLs and audio files. - **transcript**: Cheap and fast speech-transcript-based segmentation. Requires a transcript and returns an error when none is available — use `balanced` for silent or visual-only content (or `comprehensive` for non-YouTube/non-audio video files). **YouTube URLs and Audio Files**: Only the 'balanced' and 'transcript' strategies are accepted. 'comprehensive' will be rejected with an error. **Chapter Count Parameters:** - **number_of_chapters**: Target number of chapters. If only this is provided, min_chapters and max_chapters are calculated automatically. - **min_chapters**: Minimum number of chapters. If provided with number_of_chapters and max, validates min is less than or equal to number_of_chapters which is less than or equal to max. - **max_chapters**: Maximum number of chapters. If provided with number_of_chapters and min, validates min is less than or equal to number_of_chapters which is less than or equal to max. - If none are provided, chapter counts are calculated automatically based on file duration. # Introduction Source: https://docs.cloudglue.dev/api-reference/introduction Cloudglue API Reference ## Features ### Video Document Parsing Foundational APIs that transform unstructured video and audio into structured, queryable context. Get a comprehensive moment-by-moment description on a video, including transcript, diarization, visual descriptions, audio desecriptions, sound, on-screen text, and more. Perfect for getting every detail on a video. Extract structured data from videos at scale, across modalities, using a prompt or custom schema. Making videos easy to program against, query against, and categorize in your application. Split videos into meaningful parts with segmentation options like intelligent shot detection, and narrative (chapters). Turning videos into logical sequences. ### Video Reasoning Higher-level APIs that enable multimodal search, chat, and reasoning directly over video content. Add semantic search over videos and segments with natural-language queries. Enable this in your application with just a few lines of code. Add conversational AI that can query, compare, and reason across hundreds of videos, complete with full citations, with just a few lines of code. Next-gen conversational API with streaming, entity-backed knowledge, multi-turn support, and background processing. ## Base URL The base URL for all API endpoints is: ``` https://api.cloudglue.dev ``` ## Authentication For authentication, it’s required to include an Authorization header. The header should contain Bearer cg-123456789, where cg-123456789 represents your API Key. ```json theme={null} Authorization: Bearer cg-123456789 ``` ## Response codes Cloudglue employs traditional HTTP response codes. | Status | Description | | ------ | ------------------------------------------------ | | 200 | Request was successful | | 400 | Bad Request. Check the parameters and try again. | | 401 | API key is invalid or was not provided | | 402 | Insufficient credits on the account. | | 404 | Specific resource was not found | | 429 | Usage limit has been exceeded. | | 500 | Signifies a problem with Cloudglue. | Refer to the Error Codes section for a detailed explanation of all potential API errors. ## Usage limits The Cloudglue API has a rate limit to ensure the stability and reliability of the service. You exceed the rate limit, you will receive a 429 error code. For more information on the usage limits, please refer to the [Usage Limits](/getting-started/usage-limits) page. # Changelog Source: https://docs.cloudglue.dev/changelog Product updates and announcements Subscribe to receive the latest developer updates, features, and other changes directly in your inbox. # Describe Prompts Tell describe what matters in your media. [`POST /v1/describe`](/api-reference/endpoint/describe/post) and a [media description collection's](/core-concepts/media-description-collection) `describe_config` now take an optional **`prompt`** — free-form guidance naming the domain terms, product names, acronyms, or people to expect and how they're spelled, or what the description should call out. The guidance is applied to the visual, scene-text, speech, and summary passes, and to the file-level title and summary. It steers emphasis and vocabulary only: it can't make a description report content that isn't in the media, and it doesn't change the response shape. To *constrain* transcript speaker labels to a known cast, keep using `participants`. Describing the same file under different guidance produces a new job rather than reusing the cached one. [Learn more →](/core-concepts/describe#guiding-the-description-with-a-prompt) # Entity/Moments-Only Knowledge Bases `knowledge_base.collections` is now **optional** on the [Responses API](/deep-dives/responses-api) when `type` is `entity_backed_knowledge` with at least one entity collection — a [moments collection](/core-concepts/moments-collection) or [entity collection](/core-concepts/entity-collection) can be the whole knowledge base by itself. Without a transcript corpus the model answers from read-only SQL over the collection's structured data and, for moments collections, semantic moment search — citing moments' own timestamps. See [Entity/Moments-Only Knowledge Bases](/deep-dives/responses-api#entity-moments-only-knowledge-bases). # Moments Collections Run Find Moments criteria across a whole video library automatically — then enumerate, query, and semantically search the results. ## New Features * [**Moments Collections**](/core-concepts/moments-collection) — A new `collection_type: "moments"`: attach criteria in `moments_config` and every file added is processed against all of them (one run per file-criterion pair, billed upfront, with automatic reuse of identical runs and existing media descriptions). Criteria are mutable after creation via [attach](/api-reference/endpoint/collections/moment-criteria-post) (backfills existing members with live progress counters) and [detach](/api-reference/endpoint/collections/moment-criteria-delete) (completed runs persist as history). * [**Collection Moments & Findings**](/api-reference/endpoint/collections/list-collection-moments) — Enumerate a collection's moments and findings with provenance (`file_id`, `job_id`, `criterion_name`), cursor pagination, and score sorts within a single criterion. * [**Moments in Structured Queries**](/deep-dives/structured-queries#moments-tables) — Three new virtual tables (`moments`, `moment_findings`, `moment_collection_links`) bring moments to SQL and natural-language queries — and to the Responses agent's query tool. * [**Moment Search**](/api-reference/endpoint/search/post) — `POST /v1/search` with `scope: "moment"` runs semantic search over a moments collection, optionally narrowed by `criterion_name`. Hits carry the full moment record plus a `search_score` (query relevance, distinct from `criterion_score` and `rank_score`). # Find Moments Define what a "moment" means to you — and get back every span in a video that matches, with structured properties, scores, and timestamps. ## New Features * [**Find Moments API**](/api-reference/endpoint/find-moments/post) — Submit a **criterion** (plain-language instructions plus a JSON schema for the properties each match carries, with optional scoring, findings, anchors, and speaker filters) and get back every qualifying moment in the file. Runs are exhaustive; `limit` / `min_score` / `sort` shape what you read back after the fact. Media descriptions are resolved or created automatically as part of the run, identical repeat runs are free cache hits, and `find_moments.job.*` [webhook events](/getting-started/webhooks) report progress. [Learn more →](/core-concepts/find-moments) * **Findings** — Criteria with a `finding_schema` can report run-level observations that no single timestamped span could express, like a seeded talk-track topic that was never covered. # Entity Collection Search Entity collections are now first-class search targets — find files and moments by their extracted structured data across Search, Deep Search, and Responses. ## New Features * [**Search over Entity Collections**](/api-reference/endpoint/search/post) — Entity collections are now searchable: file-level extractions (`enable_video_level_entities` or `enable_metadata_mode`) at `scope: 'file'` with the flattened entity data as the result summary, and segment-level extractions (the default) at `scope: 'segment'` with real timestamps. Indexing happens automatically when files are added to the collection and is **free** — only the extraction itself bills as usual. [Learn more →](/core-concepts/entity-collection#searching-entity-collections) * [**Deep Search Auto Scope**](/api-reference/endpoint/deep-search/post) — `scope` is now optional on deep search. Omit it and the search planner picks the right scope per search plan, so a single knowledge base can mix collection types (rich transcripts, metadata, and entity collections at either level); the response echoes `scope: null`. Explicit scopes behave exactly as before. * [**Entity Collections on Responses**](/deep-dives/responses-api#entity-collections-in-the-knowledge-base) — `nimbus-002-preview` now accepts entity collections directly in `knowledge_base.collections`, retrieving entity content via search — including per-segment entity values with timestamps for grounded citations. ## Improvements * [**`searchable_status` on Collection Files**](/api-reference/endpoint/collections/files-get) — Collection file responses for entity collections now report indexing progress (`pending` → `processing` → `completed`, or `failed`). Re-adding an already-processed file rebuilds a missing or failed index at no cost. * **`doc_lexical` Covers Entity Documents** — The file-scope `doc_lexical` search modality now performs keyword and exact matching over extracted entity documents, alongside generated summaries and metadata documents. # Structured Queries Run read-only SQL — or a natural-language question compiled to SQL — over the structured data Cloudglue extracts from your videos. ## New Features * [**Structured Query API**](/api-reference/endpoint/query/post) — Query your files, extracted entities, and connector metadata with read-only SQL over three virtual tables (`files`, `entities`, `segment_entities`). Run it synchronously for inline records, ask in **natural language** (compiled to SQL), use **dry-run** to preview a query's output schema without executing it, or stream large result sets as **background CSV/JSONL exports** to a 24-hour signed URL (cancellable mid-run). [Deep dive →](/deep-dives/structured-queries) * **Structured queries on the agent** — `nimbus-002-preview` on [Responses](/deep-dives/responses-api) can run structured queries over entity collections itself, surfacing each as a `cloudglue_query_call` output item. # Cloudglue v0.7 is here This release introduces the Deep Search API, a new transcript-only narrative segmentation strategy, the `nimbus-002-preview` model on Responses, programmatic Data Connectors access, TikTok URL support, word-level transcript timestamps, and a wave of describe and segmentation improvements. ## New Features * [**Deep Search API**](/api-reference/endpoint/deep-search/post) — Run multi-step, citation-grounded research queries across a collection, a specific list of files, or your account's default index, fanning out across speech, visual, OCR, and tag modalities to surface answers with traceable sources. [Try it in the playground →](https://app.cloudglue.dev/home/playground) * [**`nimbus-002-preview` Model on Responses**](/api-reference/endpoint/responses/post) — A new preview model for [Responses](/getting-started/responses-api) with stronger long-form reasoning over your videos, available alongside the existing default model. * **Transcript-Only Narrative Segmentation** — New `transcript` option for `narrative_config.strategy` provides fast, low-cost chapter segmentation derived purely from speech transcripts. Ideal for podcasts, lectures, and dialogue-heavy content. Joins the existing `comprehensive` (VLM-based) and `balanced` (multi-modal, default) strategies. [Learn more →](/core-concepts/segmentation) * [**Data Connectors API**](/api-reference/endpoint/data-connectors/list) — Programmatically list connected data sources and [browse their files](/api-reference/endpoint/data-connectors/list-files) directly from the API. Manage connections from the [dashboard →](https://app.cloudglue.dev/home/data-connectors) * [**TikTok URL Support**](/data-connectors/tiktok) — Process TikTok videos directly by passing the URL to any Cloudglue endpoint, alongside YouTube and other public URL sources. * **Word-Level Transcript Timestamps** — New `include_word_timestamps` query parameter on transcribe and describe endpoints returns a `words` array with per-word `start_time` and `end_time` for high-precision alignment. ## Improvements * **Subtitle & Transcript Response Formats** — New `response_format` options for speech describe: `speech_srt` and `speech_vtt` for subtitle files, `speech_markdown` for diarized transcripts, and `speech_text` for plain timestamped text. * **Chapters & Shots in Describe Responses** — New `include_chapters` and `include_shots` query parameters return narrative chapter boundaries (when segmentation is `narrative`) or detected shot boundaries (when `shot-detector`) directly on [describe](/api-reference/endpoint/describe/post) and related responses. * **Thumbnails on Describe, Extract & Segmentations** — New `include_thumbnails` parameter adds file-level and per-segment `thumbnail_url` fields directly on [describe](/api-reference/endpoint/describe/post), extract, and segmentation responses. * **Fill Gaps for Shot-Based Segmentation** — New `fill_gaps` option (default `true`) ensures complete timeline coverage when using the shot-detector strategy. Set to `false` to preserve only the raw detected shot boundaries. [Learn more →](/core-concepts/segments) * **Expanded Speech Format Support** — Broader audio/speech format coverage across upload and processing. * [**HLS Stream URLs on Shareable Assets**](/api-reference/endpoint/share/post) — Shareable asset responses now include an HLS stream URL for adaptive playback in custom players. * [**Build with AI Hub**](/getting-started/build-with-ai) — A new starting point for agent-driven development, bundling [Cloudglue Skills](https://github.com/cloudglue/skills) (version-locked SDK knowledge for coding agents) and a hosted [Docs MCP Server](https://docs.cloudglue.dev/mcp) that exposes the documentation as live tools. ## Breaking Changes * **JavaScript SDK package renamed** — The npm package moved from `@aviaryhq/cloudglue-js` to [`@cloudglue/cloudglue-js`](https://www.npmjs.com/package/@cloudglue/cloudglue-js). Update your `package.json` to install from the new package name; the API surface is unchanged. ## SDK Updates All v0.7 features are available in the latest SDKs: * [JavaScript SDK →](https://www.npmjs.com/package/@cloudglue/cloudglue-js/v/0.7.9) * [Python SDK →](https://pypi.org/project/cloudglue/0.7.7/) # Cloudglue v0.6 is here This release introduces the Responses API, shareable assets, audio file support, transcript-only extraction, and a wave of new endpoints and improvements across the platform. ## New Features * [**Responses API**](/api-reference/endpoint/responses/post) — OpenAI Responses-compatible interface for multi-turn conversations with video collections. Supports system instructions, temperature control, and rich citations with timestamps. [Try it in the playground →](https://app.cloudglue.dev/home/playground) [Quickstart guide →](/getting-started/responses-api) * [**Shareable Assets**](/api-reference/endpoint/share/post) — Create shareable and embeddable links for videos and video segments. Share media programmatically via the API or from the [file manager](https://app.cloudglue.dev/home/files) in the web app. [Learn more →](/core-concepts/shareable-assets) * **Audio File Support** — Upload and process audio files (mp3, m4a, etc.) alongside video. Full support across describe, extract, collections, search, and shareable assets. * **Transcript Mode for Extract** — New `enable_transcript_mode` option for extract jobs to operate purely on transcript text, skipping full audio/video processing for faster, cheaper entity extraction. ## Improvements * [**Hybrid Search**](/api-reference/endpoint/search/post) — Combine multiple search modalities (`general_content`, `speech_lexical`, `ocr_lexical`, `tag_semantic`, `tag_lexical`) in a single query for more comprehensive results. Results are fused across modalities automatically. * **Narrative Segmentation for On-Demand & Collections** — The narrative segmentation strategy is now available for on-demand operations ([describe](/api-reference/endpoint/describe/post#body-segmentation_config), [extract](/api-reference/endpoint/extract/post#body-segmentation_config)) and [collections](/api-reference/endpoint/collections/post#body-default_segmentation_config), allowing scenes to be organized by narrative chapters for search, chat, and other applications. * **Modality Filtering for Describe** — Filter describe output by specific modalities (visual, speech, OCR, audio) for targeted results. * [**List & Retrieve Chat Completions**](/api-reference/endpoint/chat/completions-get) — New endpoints to list and fetch previous chat completions by ID. [Get by ID →](/api-reference/endpoint/chat/completions-get-id) * [**Listing Operations for Jobs**](/api-reference/endpoint/face-detect/list) — New list endpoints for face detection and face match jobs with pagination and filtering support. [Face match list →](/api-reference/endpoint/face-match/list) * [**Segment Describe Endpoints**](/api-reference/endpoint/files/segment-describes-list) — Retrieve describe results at the individual segment level. [Get segment describe →](/api-reference/endpoint/files/segment-describes-get) * [**Collection Media Upload**](/api-reference/endpoint/collections/media-post) — New endpoint to add media directly to collections via URL. ## Breaking Changes * **Video-level and segment-level entity extraction are now mutually exclusive** — `enable_video_level_entities` and `enable_segment_level_entities` can no longer both be true in the same extract job. Segment-level extraction remains the default. ## SDK Updates All v0.6 features are available in the latest SDKs: * [JavaScript SDK →](https://www.npmjs.com/package/@aviaryhq/cloudglue-js/v/0.6.0) * [Python SDK →](https://pypi.org/project/cloudglue/0.6.1/) # Cloudglue v0.5 is here This release introduces user-defined tags, enhanced segment metadata, keyframe thumbnails, and improved job management APIs. ## New Features * [**User-Defined Tags**](/api-reference/endpoint/tags/post) - Create and manage custom tags to organize your video content. Tags can be applied to both files and segments, making it easy to categorize and retrieve content based on your own taxonomy. [View all tag endpoints →](/api-reference/endpoint/tags/get) * [**Tag-Based Search**](/api-reference/endpoint/search/post) - Search your files and segments by tags in addition to semantic search. Filter results by user-defined tags at both the file and segment level. Available as a search modality in the playground. [Search files by tags →](/api-reference/endpoint/files/tags-get), [Search segments by tags →](/api-reference/endpoint/files/segment-tags-get) * [**Segment Metadata**](/api-reference/endpoint/files/segments-put) - Add and update custom structured metadata on individual video segments. Store additional context, annotations, or custom attributes directly on segments for richer data organization. Segment metadata is also available to filter by in search, providing rich structured querying capabilities to your searches. [Get segment details →](/api-reference/endpoint/files/segments-get) ## Improvements * [**Keyframe Thumbnails for Segments**](/api-reference/endpoint/segmentations/thumbnails-get) - Generate optional keyframe thumbnails for your video segments. Keyframe thumbnails are now visible during search results, providing visual context for segment-level matches. * [**Job Management APIs**](/api-reference/endpoint/describe/delete) - Delete individual describe and extract jobs when no longer needed. List jobs without retrieving the full data payload using the new `include_data` query parameter for more efficient job management. [Delete extract jobs →](/api-reference/endpoint/extract/delete) [List describe jobs →](/api-reference/endpoint/describe/list) [List extract jobs →](/api-reference/endpoint/extract/list) ## SDK Updates All v0.5 features are available in the latest SDKs: * [JavaScript SDK →](https://www.npmjs.com/package/@aviaryhq/cloudglue-js/v/0.5.0) * [Python SDK →](https://pypi.org/project/cloudglue/0.5.0/) # Cloudglue v0.4 is here We're excited to share the latest major release with powerful new capabilities for face analysis, enhanced segmentation, and improved search functionality. ## New Features * [**Face Analysis Collection & Face Search**](/api-reference/endpoint/collections/files-face-detections) - Detect and search for faces in your videos with our new face-analysis collection type. Create collections specifically for face detection, then search across your video library using face matching. Perfect for identifying speakers, tracking individuals, or finding specific people across multiple videos. [Try it in our playground →](https://app.cloudglue.dev/home/playground) * [**Audio Descriptions**](/api-reference/endpoint/describe/post) - Generate detailed audio descriptions for your videos with the new `enable_audio_description` option in describe jobs. Get comprehensive descriptions of sounds, music, and audio events alongside visual and speech analysis. * [**New Segment Options**](/core-concepts/segments) - Segment with **narrative** (comprehensive and balanced) or **shot-based** strategies, with full advanced options for fine-grained control like min/max parameters. "Narrative" is perfect for generating video chapters. "Shot-based" is perfect for capturing transitional shots. * When paired with a search collection for search use cases, you can additionally retrieve moments aligned to your segment strategy. * [**Multi-Modal Search**](/api-reference/endpoint/search/post) - Search your video content across more modalities: file-level summaries, segment-level content, and now face-based image matching. Enhanced with powerful programmability features including score thresholding to filter results, group by file for organized results, and flexible sort options (by relevance score or item count) for better control over search results. * [**Data Connectors: Google Drive, Zoom, Recall.ai & Gong**](/data-connectors/overview) - Connect your Google Drive, Zoom, Recall.ai, and Gong accounts directly to Cloudglue to process your files and meeting recordings. Skip manual file uploads and seamlessly integrate your cloud storage and recorded calls with all Cloudglue features including transcription, extraction, and search. [Learn about Google Drive →](/data-connectors/google-drive) [Learn about Zoom →](/data-connectors/zoom) [Learn about Recall.ai →](/data-connectors/recall) [Learn about Gong →](/data-connectors/gong) ## Improvements * [**Time-Based Filtering for Describe Output**](/api-reference/endpoint/describe/get) - Filter describe job results by specific time ranges using `start_time_seconds` and `end_time_seconds` parameters. Perfect for analyzing specific portions of longer videos without processing the entire file. * [**Pagination for Extract Jobs**](/api-reference/endpoint/extract/get) - Extract jobs now support pagination with `limit` and `offset` parameters, making it easier to retrieve large numbers of extracted entities in manageable chunks. * [**Filter Parameters for Collection Files**](/api-reference/endpoint/collections/files-get) - List collection files with powerful filtering capabilities. Filter by metadata, video properties (duration, audio presence), and file attributes (filename, size, creation date) using flexible query operators. # Cloudglue v0.3.0 is here Welcome to the first official update of the Cloudglue changelog. We're excited to share the latest features we've been cooking up for you all. ## New Features * [**Scene Segmentation**](/core-concepts/segments) - Break your videos into meaningful segments automatically using AI-powered shot detection or prompt driven narrative chapters. Perfect for finding natural boundaries within your videos [Try it in our playground →](https://app.cloudglue.dev/home/playground) * [**Video Search**](/api-reference/endpoint/search/post) - Retrieve clips and videos directly from your collection, without asking chat completion! You can now search your video content directly using natural language queries at both video and segment levels. Find specific moments, topics, or conversations across your entire video library with semantic search. [Try it in our playground →](https://app.cloudglue.dev/home/playground) * [**Data Connectors (AWS S3, Dropbox)**](/data-connectors/overview) - Connect your AWS S3 buckets, or Dropbox account directly to Cloudglue and skip manual file uploads entirely. Use your existing S3 URIs and Dropbox files with all our endpoints. [Try it in your dashboard →](https://app.cloudglue.dev/home/data-connectors) * [**Thumbnails**](/api-reference/endpoint/files/thumbnails-get) - Generate visual previews for your video segments automatically. Get thumbnail images for key moments in your videos to enhance your applications and user interfaces. # Chat Completions Source: https://docs.cloudglue.dev/core-concepts/chat-completions Learn how to use Cloudglue's chat completion capabilities Cloudglue provides a way to chat over your videos using a variety of rich information from the video: from speech, to visual scene descriptions, to scene text. Chatting can feel natural, and allows you to ask questions about your videos that are based on the content of the video, quickly and easily. ## Citations Alongside the answer from chat completion, Cloudglue also provides supporting citations for the answer. These citations are a way to see where the answer is coming from contextually, and a way to verify that the answer is correct. ## Models Cloudglue's chat completions are powered by the `nimbus-001` model, which uses the rich information from the video to answer your questions. Nimbus-001 understands across all modalities including: * Speech * Visual scenes * Scene text For multi-step reasoning, cross-video synthesis, and entity-backed knowledge, see the [Responses API](/api-reference/endpoint/responses/post) with `nimbus-002-preview` — a light reasoning model capable of inspecting your video assets from different dimensions. ## Collections We power chat completions through collections. In order to chat over a video, that video needs to be part of a collection. When chatting over a collection, you can chat with all the videos in the collection at once, providing much more insightful and rich answers. # On-Demand Describe Source: https://docs.cloudglue.dev/core-concepts/describe Learn about on-demand describe operations ## What is On-Demand Media Description? On-Demand Media Description allows you to describe a video, exactly as you need it, when you need it. It is a great to use when you have one-off videos that you want to describe, or, when you want to get deeper descriptions for just one or a set of particular videos. To learn more about the different types of media descriptions that Cloudglue offers [here](/core-concepts/media-description). You can use this to quickly get the speech transcripts of a single YouTube video, or get more in-depth descriptions that include scenes and visual descriptions for a different video. You can control the level of description you want to perform, to get the most out of your videos. If you're finding that you need to describe a lot of videos, in the same format, then you should consider using a [Media Description Collection](/core-concepts/media-description-collection). ## How to use it You can describe a video on-demand by using the [Describe API](/api-reference/endpoint/describe/post). ## Describing images On-Demand Describe also works on still [image files](/core-concepts/files). Images are described at the **file level only** (no segmentation) and billed at a flat **1 credit** per request. Because an image has no audio track, `enable_speech` and `enable_audio_description` are not available for images — describe defaults to a visual description (visual scene description, scene text, and summary). ## Speaker identification When `enable_speech` is on, the transcript labels each turn with a speaker. For files added through a [data connector](/core-concepts/data-connectors) that carries an attendee list (e.g. Grain), those real names are used automatically. For uploaded files — which have no attendee metadata — you can pass the cast yourself via the optional `participants` field. When provided, speaker naming is constrained to that list: each speaker is matched to one of the supplied names or left as a generic `Speaker N`, and names that aren't on the list are never invented. ```json theme={null} { "url": "cloudglue://files/your-file-id", "enable_speech": true, "participants": [ { "name": "Alice Smith", "scope": "internal" }, { "name": "Bob Jones", "scope": "external" } ] } ``` ## Guiding the description with a prompt Every describe request accepts an optional `prompt`: free-form guidance that tells the description passes what matters in this media. It is the right place for the things a model cannot know from the pixels alone — the spelling of your product names and acronyms, the jargon of your domain, who tends to be on these calls, or what you want called out. ```json theme={null} { "url": "cloudglue://files/your-file-id", "enable_speech": true, "enable_scene_text": true, "prompt": "Enterprise sales call. Spell product names as Cloudglue, Aviary, and Foreman. The host is Alice Smith. Call out any pricing, contract terms, or renewal dates discussed." } ``` The prompt is applied to the visual, scene-text, speech, and summary passes, and to the file-level title and summary. Two things it deliberately does **not** do: * **It cannot add content that isn't there.** Naming a term or a person is context, not an instruction to report them — if they don't appear in the media, the description stays silent about them. * **It does not constrain speaker labels.** For that, use [`participants`](#speaker-identification), which is a closed set: a speaker is matched to one of those names or left as `Speaker N`. A prompt is a hint the speaker-identification pass weighs as background, never as evidence. The prompt is part of the describe config, so it is also part of the cache key: describing the same file under different guidance produces a new job rather than reusing the earlier result. It is limited to 2000 characters. Describing many files under the same guidance? Set `describe_config.prompt` once on a [Media Description Collection](/core-concepts/media-description-collection) instead of repeating it per request. ## Chat Completion If you are looking to chat with your videos, you should consider using a [Media Description Collection](/core-concepts/media-description-collection), as chat completion is not supported for on-demand descriptions. # Entities Source: https://docs.cloudglue.dev/core-concepts/entities Transform video content into structured, programmable data by extracting meaningful objects, events, and attributes ## What are Entities? Entities are structured pieces of information extracted from video content. Think of them as the meaningful objects, events, or concepts that appear in a video. For example, in a cooking video, entities might include ingredients, cooking techniques, or kitchen equipment. Entities can be tied to specific moments in the video (like when a chef uses a particular technique) or apply to the entire video (like the cuisine type or difficulty level). While media description provides dense descriptions of everything in a video (speech, text, and visuals), and chat interfaces enable ad-hoc queries, entities serve a distinct purpose: they extract specific, structured information in a format ready for your application. Instead of parsing through paragraphs of description or formulating questions, you get exactly the data points you need in a consistent, programmable format. ## Video-Level vs. Segment-Level Entities Unlike documents or images which are static, videos unfold over time. This temporal nature creates two distinct types of entities: * **Video-Level Entities**: Information that applies to the entire video (e.g., the overall topic, the presenter's name, or the production quality) * **Segment-Level Entities**: Information that appears at specific timestamps (e.g., when a product is shown, when a specific person speaks, or when a technique is demonstrated) This distinction is crucial for applications that need to understand both the overall context and the specific moments where information appears. For instance, in a product review video, you might want to know both the overall rating (video-level) and exactly when specific features are discussed (segment-level). ## Prompts and Schemas Cloudglue uses two complementary approaches to extract entities: 1. **Natural Language Prompts**: These guide the extraction process by describing what information you're interested in. For instance, "Extract all kitchen equipment used and ingredients shown in this cooking video" tells Cloudglue what to look for in culinary content. 2. **Entity Schemas**: These define the exact structure of the information you want. A schema acts like a template, ensuring the extracted data follows a consistent format. For example: ```json theme={null} { "recipe": { "name": "string", "cuisine": "string", "servings": "string" }, "equipment": [ { "name": "string", "type": "string" } ], "ingredients": [ { "name": "string", "amount": "string" } ] } ``` You can use these approaches independently or together, depending on your needs: * **Schema Only**: Best for straightforward, unambiguous concepts where the structure matters most. For example, extracting speaker names and timestamps, or identifying jersey numbers and license plates. The concepts are clear enough that no additional guidance is needed. * **Prompt Only**: Ideal for exploration or when you're more interested in discovering what's in the video than enforcing a specific structure. For instance, "What teaching methods does this instructor use?" or "Extract the key arguments from this debate." This approach provides flexibility in understanding new content. * **Both Together**: The most powerful approach when you need both specific guidance and structured output. The prompt helps focus on exactly what you want to extract (e.g., "Identify weather conditions in each scene as either 'sunny', 'cloudy', 'rainy', or 'snowy'"), while the schema ensures the data comes back in a format your application can immediately use. This combination is particularly valuable for building reliable, production-ready applications. ## Value and Applications Entity extraction transforms unstructured video content into structured, programmable data. This opens up numerous possibilities: * **Content Management**: Automatically catalog and organize video libraries based on their content * **Search and Discovery**: Enable precise querying of video content using entity attributes (e.g., "WHERE scene.weather = 'sunny' AND person.attire = 'business\_suit'" maps directly to structured data fields). Entities extracted in [Entity Collections](/core-concepts/entity-collection#searching-entity-collections) are also automatically indexed for semantic and keyword search * **Analytics**: Track trends and patterns across video content (e.g., "COUNT scenes WHERE weather = 'rainy' GROUP BY month") * **Application Integration**: Feed structured video data directly into applications, databases, or recommendation systems The power of entities lies in their ability to bridge the gap between rich video content and the structured data that applications need. Whether you're building a video-heavy application, analyzing content at scale, or automating video workflows, entities provide the foundation for working with video content programmatically. ## Next Steps To learn more about implementing entity extraction, including detailed examples and step-by-step guides, check out our [Extraction Guide](/deep-dives/extraction-guide). The guide provides comprehensive examples of schema definition, prompt crafting, and practical applications of entity extraction. # Entity Collections Source: https://docs.cloudglue.dev/core-concepts/entity-collection Organize and analyze extracted structured data across multiple videos ## What are Entity Collections? Entity Collections in Cloudglue enable you to organize multiple videos and extract consistent, structured data across your entire video library. Unlike the on-demand Extract API which processes videos individually, Entity Collections apply a unified extraction schema and prompt to every video in the collection, ensuring standardized data structures for comparative analysis, search, and insights. This allows you to create a queryable database of video content organized around the specific [entities](/core-concepts/entities) that matter to your use case. While the on-demand Extract API is perfect for one-off structured data extraction, Entity Collections shine when you need to work with multiple videos over time. Collections automatically process new videos as they're added, maintain consistent extraction parameters across your content library, and enable powerful cross-video analysis. For example, you can track cooking techniques across recipe videos, compare product features across demos, or identify patterns in educational content—all without manually configuring extraction for each video. Entity Collections integrate seamlessly with the rest of Cloudglue's ecosystem, working alongside Files for storage and Media Descriptions for detailed content understanding. You can create collections with custom extraction schemas tailored to your specific domain, whether that's cooking videos with ingredient tracking, product demonstrations with feature extraction, or educational content with teaching method identification. To get started with Entity Collections, see our [structured data extraction guide](/getting-started/structured-data-extraction) or explore a complete implementation in our [cooking videos example](/use-cases/extract-structured-data-from-your-videos). ## Searching Entity Collections Entity Collections are first-class [Search](/core-concepts/search) targets. When a file is added to an entity collection, its extracted entities are automatically indexed into search documents — indexing is free, and only the extraction itself is billed as usual. You can then find files or moments by their extracted data using the [Search API](/api-reference/endpoint/search/post) or [Deep Search](/api-reference/endpoint/deep-search/post). The search scope follows the collection's extraction level, since each entity collection extracts at exactly one level: | Extraction level | Search scope | What a result looks like | | -------------------------------------------------------------------- | -------------- | ------------------------------------------------------------- | | File-level (`enable_video_level_entities` or `enable_metadata_mode`) | `file` only | A file result with the flattened entity data in its `summary` | | Segment-level (`enable_segment_level_entities`, the default) | `segment` only | A segment result with real start and end timestamps | Searching an entity collection at the wrong scope returns a clear 400 error rather than empty results. Entity collections don't require `enable_summary` — their entity documents stand in for generated summaries. For exact-term matching over file-level entity data (product names, IDs, specific field values), include the `doc_lexical` modality in a `scope: 'file'` search — it performs keyword and exact matching over entity documents alongside generated summaries and metadata documents. Each collection file exposes a `searchable_status` (`pending` → `processing` → `completed`, or `failed` if no completed extraction exists to index from) so you can tell when it becomes searchable. Re-adding an already-processed file to the collection rebuilds a missing or failed index at no cost. Beyond direct search, entity collections plug into higher-level retrieval: [Deep Search](/api-reference/endpoint/deep-search/post) can include them in a knowledge base alongside other collection types (omit `scope` to let the planner pick the right scope per collection), and the [Responses API](/deep-dives/responses-api) accepts them as knowledge base collections with the `nimbus-002-preview` model. And search is a complement, not a replacement: your extracted entities remain fully available in structured form via the [entities endpoints](/api-reference/endpoint/collections/files-entities) and [Structured Queries](/deep-dives/structured-queries). # On-Demand Extract Source: https://docs.cloudglue.dev/core-concepts/extract Transform individual videos into structured, programmable data using customized extraction parameters ## What is On-Demand Extract? On-Demand Extract is Cloudglue's powerful capability that transforms individual videos into structured, programmable data. Unlike [Entity Collections](/core-concepts/entity-collection) which process multiple videos with consistent schemas, On-Demand Extract allows you to customize extraction parameters for each video, making it perfect for one-off analyses, exploratory data extraction, or videos with unique content structures. The Extract API uses a combination of natural language prompts and structured schemas to identify and extract the exact [entities](/core-concepts/entities) you need from your video content. With On-Demand Extract, you can define exactly what information you want to extract on a per-video basis. This flexibility is ideal when you're refining your extraction approach, working with diverse video content, or when you need to quickly extract specific information without setting up a collection. For example, you might use On-Demand Extract to analyze a product demonstration video before deciding on a schema for your entire product catalog, or to extract unique information from a specific marketing video that doesn't fit your standard extraction patterns. On-Demand Extract works directly with your [files](/core-concepts/files) (video, audio, and images), and YouTube videos (though YouTube extraction only uses speech and metadata as input signals). You can choose between two extraction modes: * **Video-level entities**: Information that applies to the entire video as a whole * **Segment-level entities**: Information tied to specific timestamps within the video These modes are mutually exclusive - you must choose one or the other per extraction job. Segment-level extraction is enabled by default. Image files are extracted at the **file level only** (no segmentation) and are billed at a flat **1 credit** per request. Segment-level entities, transcript mode, chapters, and shots are not available for images. The extracted data can be immediately used in your application or analyzed for insights. If you also want the extracted entities to be searchable, use an [Entity Collection](/core-concepts/entity-collection#searching-entity-collections) — collection extractions are automatically indexed for search, while on-demand extractions are not. To get started with On-Demand Extract, see our [structured data extraction guide](/getting-started/structured-data-extraction) which provides code examples and best practices for defining prompts and schemas that yield high-quality structured data. ### Transcript-Based Extraction You can enable transcript-based extraction using the `enable_transcript_mode` flag. When enabled, entities are extracted from the spoken content only, skipping visual analysis entirely. Speaker information is preserved when available, so prompts and schemas can ask about who said what. Works for both uploaded files and YouTube videos. Useful for: * Speech-heavy content like podcasts and interviews * Faster and more cost-effective extraction when visual analysis isn't needed * Audio files where only speech content matters * Speaker-aware extraction (e.g. "list each speaker and the topics they covered") ### Metadata-Based Extraction You can enable metadata-based extraction using the `enable_metadata_mode` flag. When enabled, entities are extracted from the file's **metadata document** instead of the media content: the filename, file details (duration, resolution), any user-provided metadata, and the source metadata synced from data connectors (meeting titles, participants, recording dates, custom fields). The media itself is never analyzed, and for data-connector URLs it is never downloaded either — an existing file (including a metadata-only file) is used as-is, and a brand-new connector URL becomes a metadata-only file. Only a brand-new non-connector URL (e.g. a direct http upload) is still ingested normally before its metadata is read. Metadata-based extraction is always **file level** (segment-level entities, segmentation, chapters, and shots are not available), is billed at a flat **1 credit** per file, and is mutually exclusive with `enable_transcript_mode`. It also works on metadata-only files from [metadata collections](/core-concepts/collections) without triggering media ingestion, and can be set on an entities collection's `extract_config` so every added file is extracted from its metadata. Useful for: * Extracting structured facts a connector already knows (participants, meeting dates, hosts, custom fields) without paying for media processing * Building structured indexes over large metadata-only libraries where downloading media is unnecessary or impractical * Normalizing messy user metadata and filenames into a consistent schema # Files Source: https://docs.cloudglue.dev/core-concepts/files Understanding how files work in Cloudglue Files make up the foundation of operating with Cloudglue. Files represent a video or audio file that you want to operate on. Once you upload a file, you can use it anywhere in the Cloudglue platform, whether it may be for [on-demand operations like describing](/core-concepts/describe) or for use in [collections like entities collections](/core-concepts/entity-collection). We do not use your files for any purpose other than to provide you with the service you signed up for. ## Working with Files ### Uploading Files #### Web App The Cloudglue web app allows you to upload files by dragging and dropping them into the web app or by selecting them from your file system. Upload File Uploader Dialog **Note**: Local uploads are best for files under 250 MB. For larger files, use a [data connector](/data-connectors/overview). #### API You can also upload files via our [Files API](/api-reference/endpoint/files/post). ##### Example ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/files \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: multipart/form-data' \ --form 'metadata={}' \ --form file=@file.mp4 ``` #### SDKs You can also use our [SDKs](/sdks) to upload files to Cloudglue. ##### Example ```typescript theme={null} import { Cloudglue } from '@cloudglue/cloudglue-js'; const cloudGlue = new Cloudglue({ apiKey: 'cg-YOUR_API_KEY', }); const file = await cloudGlue.files.uploadFile({ file: new File([...videoData], 'video.mp4'), metadata: {}, }); console.log(file); ``` ```python theme={null} import cloudglue file = cloudglue.files.upload(file_data, metadata={}) ``` ### File Metadata When uploading a file, you can pass in metadata. This metadata is for your own reference. You can use this to store your own custom data, such as identifiers or names from your own system. When you retrieve a file, the metadata is included in the response. ##### Example ```typescript theme={null} const file = await cloudGlue.files.uploadFile({ file: new File([...videoData], 'video.mp4'), metadata: { my_own_id: '12345687890', type: 'Sales Footage', }, }); const file = await cloudGlue.files.getFile(file.id); console.log(file); ``` ```python theme={null} file = cloudglue.files.upload(file_data, metadata={ "my_own_id": "12345687890", "type": "Sales Footage" }) file = cloudglue.files.get(file.id) print(file) ``` ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/files \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: multipart/form-data' \ --form 'metadata={"my_own_id": "12345687890", "type": "Sales Footage"}' curl --request GET \ --url https://api.cloudglue.dev/v1/files/file_1234567890 \ --header 'Authorization: Bearer cg-YOUR_API_KEY' ``` ##### Response ```json theme={null} { ..., "metadata": { "my_own_id": "12345687890", "type": "Sales Footage" } } ``` ### File Status Cloudglue is inherently asynchronous. We use the `file status` to indicate the current status of the file. The possible values are: * `pending` - The file is pending and has not been processed yet. * `processing` - The file is being processed. * `completed` - The file has been processed successfully. * `failed` - The file failed to process. You can check the status of a file by calling the `getFile` method. ##### Example ```typescript theme={null} const file = await cloudGlue.files.getFile(file.id); console.log(file.status); ``` ```python theme={null} file = cloudglue.files.get(file_id) print(file.status) ``` ```bash theme={null} curl --request GET \ --url https://api.cloudglue.dev/v1/files/file_1234567890 \ --header 'Authorization: Bearer cg-YOUR_API_KEY' ``` ##### Response ```typescript theme={null} 'pending' | 'processing' | 'completed' | 'failed'; ``` ##### Webhooks To listen for file status updates asynchronously, you can use our [webhooks](/getting-started/webhooks). You can provide an endpoint to listen to the following file events: * `file.job.processing` - When a file is being processed. * `file.job.completed` - When a file has been processed successfully. * `file.job.failed` - When a file has failed to process. * `file.job.deleted` - When a file is deleted. ## File Limits & Requirements ### Size Limits * **Maximum file size**: 2 GB per file * **Local upload limit**: 250 MB for uploads via web app or files API * Larger local uploads may fail or lead to slow processing times * For files larger than 250 MB, use a data connector such as AWS S3 ### Duration Limits * **Minimum duration**: 2 seconds * **Maximum duration**: 2 hours * Files longer than 2 hours must be split into multiple parts before uploading * Each part must still respect the 2 GB size limit ### Upload Limits We do not charge you for the storage of your files. However, we do have some limits in place to prevent abuse. To see the limits, please see the [Rate Limits](/getting-started/rate-limits#file-uploads) page. ### Higher Limits If you need larger file sizes or expect to upload many large files, [contact us](mailto:support@cloudglue.dev) to discuss custom limits. ## Supported File Types Cloudglue supports the following file types: ### Video Files * `video/mp4` * `video/quicktime` * `video/avi` * `video/webm` * `application/octet-stream` ### Audio Files * `audio/mpeg` (MP3) * `audio/mp4` (M4A, AAC) * `audio/wav` * `audio/x-wav` * `audio/flac` * `audio/ogg` * `audio/webm` ### Image Files * `image/jpeg` * `image/png` * `image/webp` * `image/heic` * `image/heif` Images are processed at the **file level only** (no segmentation). On-Demand [Describe](/core-concepts/describe) and [Extract](/core-concepts/extract) run on the whole image and are billed at a flat **1 credit** per operation. Speech and audio-description options do not apply to images. Don't see your file type? [Contact us](mailto:support@cloudglue.dev) about it. ## Audio File Processing Audio files are processed using the transcript by default for speech-based analysis. For comprehensive audio analysis including music, sound effects, and ambient sounds, you can enable `enable_audio_description` in your describe configuration. * **Transcript-only mode** (default): Uses transcript for title/summary generation. Best for podcasts, interviews, and speech-heavy content. Faster and more cost-effective. * **Audio description mode** (`enable_audio_description: true`): Uses Gemini for comprehensive audio analysis including speech, music, sound effects, and ambient sounds. Best for rich audio content with significant non-speech audio. Audio files cannot be added to face-analysis collections since facial analysis requires visual content. ## Optimal File Formatting To ensure fast processing and reliable playback, uploaded videos should follow these industry-standard encoding parameters. We will start rejecting non-standard inputs in the future: | Parameter | Recommended Value | Notes | | ------------------ | ---------------------- | ---------------------------------------------------------------------------------------------------------------- | | **Container** | MP4 (.mp4) | Use MP4 container with moov atom at the start (use `-movflags +faststart` option when encoding) | | **Video Codec** | H.264 (AVC) | H.264 is widely supported. Other codecs (HEVC/ProRes/etc.) may be transcoded and could be rejected in the future | | **Resolution** | ≤1080p/2K (≤2048×2048) | Higher resolutions increase processing time and may exceed file-size limits | | **Audio Codec** | AAC | Non-AAC audio will be transcoded and may be rejected | | **Audio Channels** | Mono or stereo | Multi-channel audio (5.1, 7.1, etc.) will be down-mixed | ### Transcoding with ffmpeg to ideal video format Use ffmpeg to convert your source video into a supported format: ```bash theme={null} ffmpeg -i input_video.ext \ -c:v libx264 \ -vf "scale=w=min(iw\,1920):h=-2" \ -c:a aac -b:a 192k -ac 2 \ -movflags +faststart \ output.mp4 ``` This command: * Uses H.264 video codec * Limits width to ≤1920px while maintaining aspect ratio * Encodes stereo AAC audio at 192k bitrate * Puts MP4 metadata at start for faster streaming ### Checking Video Specifications Use this script to check if your video meets the recommended specifications: ```bash theme={null} #!/bin/bash # Usage: ./check-video.sh FILE="$1" # Extract video properties: codec, width, height read -r vcodec width height <<<$(ffprobe -v error -select_streams v:0 \ -show_entries stream=codec_name,width,height \ -of default=noprint_wrappers=1:nokey=1 "$FILE") # Extract audio properties: codec, channels read -r acodec channels <<<$(ffprobe -v error -select_streams a:0 \ -show_entries stream=codec_name,channels \ -of default=noprint_wrappers=1:nokey=1 "$FILE") # Flag if any property falls outside the recommended specs needs_transcode=0 if [[ "$vcodec" != "h264" ]] || [[ "$width" -gt 2048 ]] || [[ "$height" -gt 2048 ]]; then needs_transcode=1 fi if [[ "$acodec" != "aac" ]] || [[ "$channels" -gt 2 ]]; then needs_transcode=1 fi if [[ $needs_transcode -eq 1 ]]; then echo "This file does not meet the recommended specs and should be transcoded." else echo "This file meets the recommended specs." fi ``` ### Splitting Large Videos If your video is longer than 2 hours or larger than 2 GB, split it into compliant chunks: ```bash theme={null} # Split into 90-minute segments without re-encoding ffmpeg -i long_video.mp4 -map 0:v:0 -map 0:a:0? -dn -c copy -f segment \ -segment_time 01:30:00 -reset_timestamps 1 \ part_%03d.mp4 ``` To check duration and size before splitting: ```bash theme={null} ffprobe -i input.mp4 -hide_banner -v error \ -show_entries format=duration,size \ -of default=noprint_wrappers=1:nokey=1 ``` ## Best Practices Summary * **For quick experiments**: Use local uploads for files under 250 MB * **For larger files**: Set up a data connector such as AWS S3 * **Before uploading**: Ensure files are 2 seconds to 2 hours duration and ≤2 GB * **For optimal performance**: Use MP4 with H.264 video, AAC audio, ≤1080p resolution * **For long videos**: Split into chunks using ffmpeg and track parts with metadata * **Always check**: Use ffprobe to verify file specifications before upload For questions about custom limits or larger file requirements, [contact our support team](mailto:support@cloudglue.dev). # Find Moments Source: https://docs.cloudglue.dev/core-concepts/find-moments Find every moment in a video that matches a criterion you define — with structured properties, scores, and findings ## What is Find Moments? Find Moments turns a plain-language definition of "the thing you care about" into timestamped, structured results. You describe a **criterion** — instructions plus a JSON schema for the properties each match should carry — and the [Find Moments API](/api-reference/endpoint/find-moments/post) scans the whole video and returns every span that qualifies. Each moment comes back with start and end times, a title, the reason it qualified, your schema's properties filled in, and scores for ranking. Unlike [Search](/core-concepts/search), which retrieves content similar to a query, Find Moments is exhaustive discovery: a run evaluates the entire video against your criterion and keeps everything that qualifies. Ask for "moments where the customer voices a complaint," "every pricing discussion," or "each time the presenter makes a claim about performance," and you get all of them — not the top few. Find Moments works directly with your [files](/core-concepts/files). Under the hood it reads the file's media description; if a suitable one doesn't exist yet, one is created automatically as part of the run — you never have to orchestrate describe jobs yourself. ## Criteria A criterion is the unit of reuse in Find Moments. It has: * **`name`** — an identifier for the criterion (e.g. `customer_complaint`) * **`instructions`** — plain language describing what qualifies as a moment * **`moment_schema`** — a JSON schema for the structured properties each moment carries (e.g. `{ "heat": "high" }`) * **`scoring`** (optional) — exactly one scored dimension (`key`, `min`, `max`); the model grades each moment on it * **`finding_schema`** (optional) — a schema for run-level findings that aren't tied to a single span, such as "topic X was never covered" * **`anchors`** (optional) — named timestamps within a moment (e.g. `objection_start`); required anchors that can't be located drop the moment, optional ones are simply omitted Run options control execution: `signals_required` (which description signals the criterion needs, e.g. speech only), `boundary_policy` (`sentence`, `tight`, `loose`), `speaker_filter`, and min/max moment durations. ## Scores Each moment can carry up to two scores, and search adds a third — they measure different things: | Score | Where | Meaning | | ----------------- | ---------------- | ---------------------------------------------------------------- | | `criterion_score` | Moment | The grade on your declared scoring dimension, on your scale | | `rank_score` | Moment | Normalized 0–1 ordering within a run; what the default sort uses | | `search_score` | Search hits only | Query relevance for one search; unrelated to the two above | Raw criterion scores are only comparable within one criterion — the API never sorts across criteria by score. ## Runs are exhaustive; reads are shaped Every run is a complete pass over the video. "Top-k" is a read-time view, not an execution mode: use `limit`, `min_score`, and `sort` on the [GET endpoint](/api-reference/endpoint/find-moments/get) to shape what you read back, and change them freely without re-running anything. Repeat requests are cache hits. Submitting the same criterion and options for the same file returns the completed run — and costs nothing. Pass `cache_policy: "refresh"` to force a fresh run. ## Findings When your criterion declares a `finding_schema`, a run can also return **findings** — run-level observations validated against that schema. The canonical use is auditing: a talk-track criterion can report `{ "topic": "speed_to_value", "status": "not_observed" }` for a seeded topic the video never covered, something no timestamped moment could express. ## Working at scale To run criteria across many videos automatically, use a [Moments Collection](/core-concepts/moments-collection): attach criteria once, and every file added is processed against all of them, with results queryable across the whole collection via enumeration, [structured queries](/deep-dives/structured-queries), and moment-scope [search](/core-concepts/search). ## Pricing A run costs a flat **4 credits**. If the file needs a new media description to cover the requested signals, that description is billed at the standard describe rate as part of the run; existing compatible descriptions are reused at no extra cost, and cache-hit runs cost **0**. # Media Description Source: https://docs.cloudglue.dev/core-concepts/media-description Understanding media descriptions and their capabilities ## What are Media Descriptions? Media Descriptions are structured pieces of information that describe a video and its content. The most familiar kind of media description is a speech transcript, which composes of the verbal speech from a video. We see this commonly in places like YouTube, and nowadays many meeting software tools also offer this as a feature. While speech transcripts are a great source of rich and deep information, they are not the only kind of media description that Cloudglue offers. Cloudglue also exposes other kinds of media descriptions, including: * **Visual scene descriptions**: A description of the visual scene in a video. * **Scene text**: The text that is visible on the screen in a video. Combining speech, with these other media description types allows Cloudglue to turn your videos into a rich and structured source of information. ## What can you use it for? ### Speech Speech transcripts are the most intuitive kind of media description, and are the most commonly used. They are a great source of information for a variety of use cases, including: * Searching for specific words or phrases * Finding key takeaways or action items from a meeting * Generating a summary of a conversation ### Visual scene descriptions Visual scene descriptions are a great way to know what is physically happening in a video. They are a great way to understand the context and setting of a video, and are a great way to identify key visual elements or objects in a scene. Examples of visual scene descriptions include: * Detecting important visual cues or actions * Monitoring product placement or branding * Identifying people or objects in the scene * Understanding visual transitions between scenes * Analyzing visual composition and framing ### Scene text Scene text is the text that is visible on the screen in a video. Commonly, this could be closed captioning, but it could also be other text that is visible on the screen. Examples of scene text include: * Closed captioning * Text from a presentation * Text from a document or on the background * Menu text or product labels ### Putting it all together Separately, each of these media description types are great on their own, but when combined, they can provide a very methodical and deep way to understand different scenes throughout a video. Speech allows you to know what is being said for context at a time, while visual scene descriptions and scene text allow you to really understand who, what, when, and where a scene is taking place. # Media Description Collections Source: https://docs.cloudglue.dev/core-concepts/media-description-collection Managing and utilizing collections of media descriptions For more information about media descriptions, see [Media Descriptions](/core-concepts/media-description). ## What are Media Description Collections? Media Description Collections in Cloudglue enable you to describe multiple videos in a consistent format. Unlike [On-Demand Description](/core-concepts/describe) which descibes each video individually and potentially with different settings, all videos inside a Media Description collection are transcribed the exact same way. Collections automatically transcribe each video as they are added in the collection, managing the flow for you, making it ideal for working with videos that are smiilar in concept. This allows you to have a consistent way to understand the content of multiple videos. Media Description collections also enable you to chat with the content of your videos. You can ask questions about the videos, search across all the videos in the collection and more. ## Guiding the descriptions with a prompt `describe_config` accepts an optional `prompt`: free-form guidance applied when every file in the collection is described. Because a collection exists to describe similar videos the same way, this is usually where guidance belongs — set it once at creation instead of repeating it on each request. ```json theme={null} { "name": "Enterprise sales calls", "collection_type": "media-descriptions", "describe_config": { "enable_speech": true, "enable_scene_text": true, "enable_summary": true, "prompt": "Enterprise sales calls. Spell product names as Cloudglue, Aviary, and Foreman. Call out any pricing, contract terms, or renewal dates discussed." } } ``` The prompt steers what the descriptions emphasize and how domain terms are spelled. It cannot make a description report content that isn't in the video, and it doesn't change the response shape. Like the rest of `describe_config` it is fixed at creation: files already in the collection were described under the original guidance, so changing it later would leave the collection inconsistent. Create a new collection to describe the same files under different guidance. See [On-Demand Describe](/core-concepts/describe#guiding-the-description-with-a-prompt) for the per-request equivalent. # Moments Collections Source: https://docs.cloudglue.dev/core-concepts/moments-collection Run moment criteria across a whole video library automatically, then enumerate, query, and search the results ## What are Moments Collections? A Moments Collection applies [Find Moments](/core-concepts/find-moments) criteria across every video in the collection. Create it with `collection_type: "moments"` and one or more criteria in `moments_config`; from then on, each file you add is processed against all attached criteria automatically — one run per (file, criterion) pair. The result is a standing, queryable dataset of moments across your library: every complaint in a quarter of sales calls, every safety violation across inspection footage, every claim in a season of episodes. ```json theme={null} POST /v1/collections { "collection_type": "moments", "name": "sales-call-moments", "moments_config": { "criteria": [ { "criterion": { "name": "customer_complaint", "instructions": "Find moments where the customer voices a complaint or pain point.", "moment_schema": { "type": "object", "properties": { "heat": { "type": "string", "enum": ["low", "medium", "high"] } }, "required": ["heat"] }, "scoring": { "key": "complaint_strength", "min": 1, "max": 10 } }, "signals_required": ["speech"] } ] } } ``` ## Managing criteria Criteria are mutable after creation: * [Attach a criterion](/api-reference/endpoint/collections/moment-criteria-post) — starts a backfill over the collection's existing members; the attachment's `backfill_status` and `files_total` / `files_completed` / `files_failed` counters track progress (echoed live on the single-collection GET). * [Detach a criterion](/api-reference/endpoint/collections/moment-criteria-delete) — stops future processing. Completed runs and their moments persist as account history: they remain visible on their own run endpoints, in account-scoped search, and in the query tables. Runs are shared with the on-demand API: if a file already has a completed run for an identical criterion and options, the collection reuses it instead of paying for a new one. ## Reading results * **Enumeration**: [List Collection Moments](/api-reference/endpoint/collections/list-collection-moments) and [List Collection Moment Findings](/api-reference/endpoint/collections/list-collection-moment-findings) return records with provenance (`file_id`, `job_id`, `criterion_name`) and cursor pagination. The default sort is positional (by file, then start time); score sorts require narrowing to a single criterion, because raw criterion scores are never comparable across criteria. * **Structured queries**: the [Query API](/deep-dives/structured-queries) exposes `moments`, `moment_findings`, and `moment_collection_links` virtual tables for SQL and natural-language questions ("which rep drew the most high-heat complaints?"). * **Search**: `POST /v1/search` with `scope: "moment"` runs semantic search over the collection's moments, optionally narrowed by `criterion_name`. Hits carry the full moment record plus a `search_score` (query relevance — distinct from `criterion_score` and `rank_score`). * **Agent**: the [Responses API](/deep-dives/responses-api) can answer moments questions over these collections through its SQL tool and moment search with no extra setup — since 0.7.30 a moments collection can even be the [whole knowledge base](/deep-dives/responses-api#entity-moments-only-knowledge-bases) by itself. ## Pricing Adding a file is billed upfront per attached criterion: **4 credits per (file, criterion) run**, plus one standard describe charge if the file needs a new media description (one description covers all criteria — the collection creates a single description spanning every criterion's required signals). Attaching a criterion later bills each backfill run the same way as it executes. Reused runs and reused descriptions cost nothing. # Search Source: https://docs.cloudglue.dev/core-concepts/search Search across video files and segments to find relevant content ## What is Search? Search in Cloudglue enables you to find relevant video content across your collections using natural language queries. Whether you're looking for specific topics discussed in videos or particular moments within video segments, Search provides powerful semantic search capabilities to help you discover exactly what you need. Search works with your [Media Description Collections](/core-concepts/media-description-collection) to find content based on speech transcripts, visual descriptions, and scene text. It also works with [Entity Collections](/core-concepts/entity-collection), where extracted structured data is indexed for search, and metadata collections, where connector and user metadata is indexed. This makes it easy to locate specific information across large video libraries without having to manually review each video. ## Search Scopes Cloudglue offers three text search scopes (plus [face search](/api-reference/endpoint/search/post)), each optimized for different use cases: ### File-Level Search File-level search helps you find entire videos that are relevant to your query. This scope is perfect when you want to discover which videos in your collection contain information about a particular topic. **Key Features:** * Searches across entire video summaries and titles * Returns whole video files as results * Ideal for content discovery and organization * For media description collections, requires `enable_summary: true` in the describe configuration; metadata collections and file-level entity collections are searchable at file scope without it **Use Cases:** * "Find all videos about machine learning" * "Which videos discuss quarterly earnings?" * "Show me cooking videos featuring Italian cuisine" ### Segment-Level Search Segment-level search pinpoints specific moments within videos where relevant content appears. This scope is ideal when you need to find exact timestamps where particular topics are discussed or events occur. **Key Features:** * Searches within individual video segments * Returns specific time ranges with precise timestamps * Includes relevant speech, visual descriptions, and scene text * Provides detailed context for each result **Use Cases:** * "When did the speaker mention the new product launch?" * "Find moments where charts or graphs are shown" * "Locate instances where specific technical terms are discussed" ### Moment-Level Search Moment-level search (`scope: "moment"`) works over [Moments Collections](/core-concepts/moments-collection): it semantically searches the moments your [Find Moments](/core-concepts/find-moments) criteria discovered, rather than raw video content. **Key Features:** * Searches moment titles, reasons, and criterion-defined properties * Returns full moment records with timestamps, `criterion_score`, and `rank_score`, plus a `search_score` (query relevance for this search — distinct from the moment's own scores) * Optionally narrowed to one criterion with `criterion_name` * Only `collections` and `criterion_name` narrow moment search — `search_modalities`, `group_by_key`, and `filter` are not supported at this scope **Use Cases:** * "Which discovered complaints mention pricing?" * "Find the objection-handling moments about integrations" ## Search Capabilities ### Semantic Understanding Search uses advanced language models to understand the meaning behind your queries, not just exact keyword matches. This means you can search using natural language and find semantically related content even when different words are used. **Examples:** * Searching for "revenue growth" will also find content about "income increase" or "profit expansion" * A query about "customer satisfaction" can discover discussions about "user happiness" or "client feedback" ### Multi-Modal Content Search works across all types of multimodal media description data: * **Speech Transcripts**: Find spoken words and conversations * **Visual Scene Descriptions**: Locate visual elements and actions * **Scene Text**: Discover text that appears on screen For exact-term matching, lexical modalities are available alongside semantic search. At file scope, the `doc_lexical` modality performs keyword and exact matching over file-level documents — generated summaries, metadata documents, and extracted entity documents. ### Advanced Filtering Enhance your searches with powerful filtering options: * **Metadata Filters**: Filter by custom metadata fields you've added to your files * **Video Properties**: Filter by duration, audio presence, file size, and more * **File Attributes**: Filter by filename, creation date, or file ID * **Multiple Criteria**: Combine multiple filters for precise results ## How Search Works ### Requirements To use Search, you need: 1. **A searchable collection**: Search works with collections of type `media-descriptions`, `rich-transcripts`, `entities`, and `metadata` 2. **Processed Content**: Files must be fully processed (described, extracted, or metadata-indexed) before they become searchable 3. **Summary Configuration**: For file-level search over media description collections, the collection must have `enable_summary: true` ### Collection Types and Scopes Each collection type supports specific search scopes: | Collection type | `file` scope | `segment` scope | | --------------------------------------------------------------------------------- | --------------------------------- | --------------- | | `media-descriptions` / `rich-transcripts` | Yes (with `enable_summary: true`) | Yes | | `entities` — file-level (`enable_video_level_entities` or `enable_metadata_mode`) | Yes | No | | `entities` — segment-level (`enable_segment_level_entities`, the default) | No | Yes | | `metadata` | Yes | No | Moments collections are searched at their own scope — `scope: "moment"` — and are rejected at `file`/`segment`/`face` scopes (and vice versa). Entity collections are searched by their extracted structured data: file-level results carry the flattened entity data in the `summary` field, and segment-level results include real start and end timestamps. See [Entity Collections](/core-concepts/entity-collection#searching-entity-collections) for details, including the `searchable_status` indexing lifecycle. ### Search Process 1. **Query Processing**: Your natural language query is analyzed to understand intent and context 2. **Content Matching**: The system searches across relevant media description data using semantic similarity 3. **Ranking**: Results are ranked by relevance score (0-1, with higher scores indicating better matches) 4. **Filtering**: Any specified filters are applied to narrow down results 5. **Response**: Results are returned with detailed context and metadata ## Search Results ### File Search Results When performing file-level searches, you receive: * **File Information**: ID, filename, and collection details * **Generated Content**: AI-generated title and summary (for entity collections, the `summary` carries the file's flattened entity data) * **Relevance Score**: Confidence score for the match quality * **Metadata**: Any custom metadata associated with the file ### Segment Search Results For segment-level searches, results include: * **Timing Information**: Precise start and end timestamps * **Content Context**: Relevant speech, visual descriptions, and scene text * **File Details**: Information about the source video * **Relevance Score**: Match quality assessment * **Segment ID**: Unique identifier for the specific segment ## Practical Applications ### Content Management * **Video Library Organization**: Quickly categorize and tag large video collections * **Duplicate Detection**: Find similar content across different videos * **Content Auditing**: Locate videos containing specific topics or compliance-related content ### Research and Analysis * **Interview Analysis**: Find specific topics or quotes across multiple interview videos * **Market Research**: Locate customer feedback or product mentions * **Educational Content**: Find explanations of particular concepts across training videos ### Media Production * **B-Roll Discovery**: Find specific visual content for editing projects * **Fact Checking**: Locate original sources for claims or statements * **Content Repurposing**: Identify segments suitable for creating clips or highlights ## Getting Started To start using Search: 1. **Create a Media Description Collection**: Set up a collection wit media description settings appropriate to your use case 2. **Add Videos**: Upload or add videos to your collection 3. **Wait for Processing**: Ensure all videos are fully described 4. **Perform Searches**: Use the [Search API](/api-reference/endpoint/search/post) to find content For file-level search capabilities, make sure to set `enable_summary: true` when creating your Media Description Collection. ## Best Practices ### Query Optimization * **Be Specific**: More detailed queries generally yield better results * **Use Natural Language**: Write queries as you would ask a person * **Include Context**: Add relevant context to help disambiguate your intent ### Collection Setup * **Enable Summaries**: Include summary generation for file-level search capabilities * **Media Description**: Enable visual descriptions and scene text for comprehensive search coverage * **Consistent Metadata**: Use standardized metadata fields for effective filtering ### Result Management * **Use Relevance Scores**: Higher scores indicate better matches * **Apply Filters**: Narrow down results using metadata and file property filters * **Combine Scopes**: Use both file and segment search for comprehensive discovery ## Limitations * **Collection Type**: Works with media description, rich transcript, entity, and metadata collections — each at the scopes shown above; face search requires face-analysis collections * **Processing Dependency**: Videos must be fully processed before they become searchable * **Summary Requirement**: File-level search over media description collections requires summary generation to be enabled * **Language Support**: Search quality depends on the language and clarity of video content ## Next Steps To learn more about implementing search in your applications: 1. **Explore the API**: Check out the [Search API documentation](/api-reference/endpoint/search/post) # Segments Source: https://docs.cloudglue.dev/core-concepts/segments Intelligent video segments that break down videos into meaningful parts using shot detection or narrative analysis ## What are Segments? Segments are intelligently divided portions of a video that break down longer content into meaningful, manageable parts. Unlike simple time-based cuts, Cloudglue uses advanced computer vision and AI analysis to identify natural boundaries in your video content. **Note:** YouTube URLs are supported for narrative-based segmentation only. Shot-based segmentation requires direct video file access. Segments serve as the foundation for many Cloudglue features, enabling you to work with specific parts of videos rather than entire files. Whether you're looking to create highlights, analyze specific scenes, or build interactive video experiences, segments provide the precise time boundaries you need. ## Types of Segment Creation Cloudglue offers two distinct approaches to creating video segments, each optimized for different types of content and use cases: ### Shot-Based Segments Shot-based segments use computer vision to detect visual changes and scene transitions in your video. This approach identifies when the camera angle changes, when scenes cut to different locations, or when significant visual elements enter or leave the frame. **Best for:** * **Dynamic Content**: Videos with camera movement, action sequences, or multiple scenes * **Visual Analysis**: Content where visual transitions are the primary organizing principle * **Production Footage**: Videos with clear shot boundaries and scene changes **Configuration Options:** * **Detector Types**: * `adaptive`: Designed for dynamic footage with camera movement and action (default) * `content`: Optimized for controlled footage with clear visual transitions * **Duration Limits**: Set minimum (0.6 seconds) and maximum (10 minutes) segment durations * **Default Settings**: 0.6 seconds minimum, 5 minutes maximum duration * **Gap Filling**: `fill_gaps` (default: `true`) ensures complete timeline coverage. Gaps longer than `min_duration_seconds` become their own segments (split by `max_duration_seconds` if needed), while shorter gaps are merged into the nearest adjacent segment. Set to `false` to preserve only the raw detected shot boundaries. ### Narrative-Based Segments Narrative-based segments use AI analysis to identify logical story segments and chapters within your video content. This approach understands the flow of information, topic changes, and narrative structure to create segments that make sense from a content perspective. **Best for:** * **Educational Content**: Lectures, tutorials, and training videos * **Documentaries**: Long-form content with distinct topics or chapters * **Presentations**: Videos with clear topic transitions and logical flow * **Storytelling**: Content where narrative structure is more important than visual changes **Configuration Options:** * **Custom Prompts**: Provide additional guidance to influence how the AI identifies logical boundaries * **Content-Aware**: Automatically adapts to different types of narrative content **YouTube Support:** Narrative-based segmentation supports YouTube URLs directly. The system will: * Fetch the video transcript automatically * Analyze the content and topic flow * Generate logical chapter boundaries based on narrative structure This makes it ideal for segmenting educational videos, lectures, and presentations hosted on YouTube. ## Practical Applications ### Content Management * **Video Libraries**: Automatically organize long-form content into manageable chapters * **Content Discovery**: Enable users to jump to specific topics or scenes * **Archive Management**: Break down lengthy recordings into searchable segments ### Media Production * **Editing Workflows**: Identify natural cut points for video editing * **Highlight Reels**: Extract key moments from longer content * **Content Repurposing**: Create shorter clips from longer videos ### Educational Platforms * **Course Navigation**: Break lectures into logical learning modules * **Topic Indexing**: Enable students to find specific subjects quickly * **Progress Tracking**: Allow learners to resume from specific segments ### Analytics and Insights * **Engagement Analysis**: Track which segments receive the most attention * **Content Performance**: Understand which parts of videos are most valuable * **User Behavior**: Analyze how viewers navigate through segmented content ## Getting Started ### Basic Shot-Based Segments ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/segments \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://example.com/video.mp4", "criteria": "shot" }' ``` ### Advanced Shot Configuration ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/segments \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://example.com/video.mp4", "criteria": "shot", "shot_config": { "detector": "content", "min_duration_seconds": 2, "max_duration_seconds": 180 } }' ``` ### Shot Configuration with Gap Filling ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/segments \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://example.com/video.mp4", "criteria": "shot", "shot_config": { "detector": "adaptive", "min_duration_seconds": 5, "max_duration_seconds": 60, "fill_gaps": true } }' ``` With `fill_gaps` enabled (the default), any time gaps between detected shots are automatically filled. Gaps longer than `min_duration_seconds` become their own segments (split by `max_duration_seconds` if needed), while shorter gaps are merged into the nearest adjacent segment. This ensures the segments cover the entire video timeline with no missing time ranges. ### Narrative-Based Segments with YouTube URL ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/segments \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://www.youtube.com/watch?v=VIDEO_ID", "criteria": "narrative", "narrative_config": { "prompt": "Identify main topics and key concept transitions" } }' ``` ### Narrative-Based Segments with Custom Prompt ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/segments \ --header 'Authorization: Bearer cg-YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://example.com/lecture.mp4", "criteria": "narrative", "narrative_config": { "prompt": "Focus on topic changes and key concept transitions in this educational content" } }' ``` ## Best Practices ### Choosing the Right Criteria * **Use Shot-Based** for: Action videos, multi-scene content, visual-heavy material * **Use Narrative-Based** for: Educational content, presentations, story-driven videos * **Consider Content Type**: Match your segment creation approach to your content's primary organizing principle ## Next Steps To learn more about implementing segments in your applications: 1. **Explore the API**: Check out the [Segments API documentation](/api-reference/endpoint/segments/post) 2. **Try Both Approaches**: Experiment with both shot-based and narrative-based segment creation 3. **Integrate with Other Features**: Combine segments with transcription, extraction, or search capabilities 4. **Build Applications**: Use segment data to create interactive video experiences Segments provide the foundation for working with video content at a granular level, enabling you to build sophisticated applications that understand and interact with video content intelligently. # Shareable Assets Source: https://docs.cloudglue.dev/core-concepts/shareable-assets Create shareable and embeddable media links ## Overview Shareable assets let you create links for your video and audio files or specific segments. These links can be shared externally or embedded in your applications. Shareable asset page ## Visibility Each shareable asset has a `visibility` that controls who can view it: * **`public`** (default) — Anyone with the link can view the share page and stream the media without signing in. This is ideal for external sharing, embedding, and public previews. * **`private`** — The share page requires sign-in and is restricted to members of the account that owns the asset. The `stream_url` is a signed, token-gated playback URL and the `media_download_url` is short-lived (a few minutes), so private media can't be streamed or downloaded without going through the API or the gated share page. Set `visibility` when creating the asset. It cannot be changed after creation. ## Use Cases * **Sharing clips** — Generate a link to a specific video segment to share with collaborators or stakeholders. * **Embedding in apps** — Embed video players directly into your web applications using iframes. * **Public previews** — Create publicly accessible previews of processed media for review or distribution. ## Creating Shareable Assets You can create shareable assets in two ways: 1. **Via the API** — Use the [create share endpoint](/api-reference/endpoint/share/post) to programmatically generate shareable links for files or segments. 2. **Via the web app** — Use the file manager in the Cloudglue dashboard to create and manage shareable links. ## Embedding Shareable asset URLs support a minimal layout mode for clean iframe embedding: ``` https://app.cloudglue.dev/share/{asset_id}?layout=minimal ``` Use the `layout=minimal` query parameter to hide navigation chrome and show only the media player. Minimal layout view ## HLS Streaming Once a shareable asset has finished processing, the `stream_url` field provides an HLS stream URL that can be used directly with any HLS-compatible video player. This URL is only available when the asset status is `completed`. For private assets the `stream_url` includes a signed playback token that is valid for 12 hours, so it should not be cached long-term — fetch a fresh one from the [get share endpoint](/api-reference/endpoint/share/get) when you need to play the media. This is distinct from the `media_download_url`, which expires much sooner (a few minutes) and needs to be refreshed more frequently. Popular HLS-compatible players include: * [HLS.js](https://github.com/video-dev/hls.js) — JavaScript library for browsers that don't natively support HLS * [Video.js](https://videojs.com/) — Full-featured web video player with HLS support * [Mux Player](https://docs.mux.com/guides/mux-player-web) — Drop-in web component optimized for Mux streams ## Start Time You can link directly to a specific timestamp using the `s` query parameter (in seconds): ``` https://app.cloudglue.dev/share/{asset_id}?s=120 ``` This starts playback at 2 minutes into the media. ## API Reference * [Create shareable asset](/api-reference/endpoint/share/post) * [Get shareable asset](/api-reference/endpoint/share/get) * [List shareable assets](/api-reference/endpoint/share/list) * [Update shareable asset](/api-reference/endpoint/share/put) * [Delete shareable asset](/api-reference/endpoint/share/delete) # AWS S3 Source: https://docs.cloudglue.dev/data-connectors/aws-s3 Connect your S3 buckets to Cloudglue using cross-account IAM roles The AWS S3 data connector allows you to connect any S3 bucket to Cloudglue, enabling you to use your S3 URIs directly with our endpoints without manually uploading files. ## How It Works This connector creates a secure bridge between your S3 buckets and Cloudglue using IAM Roles: * **Direct S3 access** - We only access your files when you explicitly use them. * **No manual uploads** - Skip the process of manually uploading files to Cloudglue. * **S3 URI support** - Use your existing S3 URIs directly with Cloudglue endpoints. ## Overview 1. **Configure S3** using either the CloudFormation template or the manual setup process. 2. **Provide the role ARN** and your `externalId` to Cloudglue. 3. **Configure the connector** on our web platform. ## AWS Setup ### Option 1: CloudFormation Template (Recommended) We provide a CloudFormation template that automates the AWS setup process, handling IAM role creation, policies, and trust relationships. #### CloudFormation Template You can download the template [here](https://media.cloudglue.dev/templates/cloudglue-assumed-role.cft.yaml). ```yaml theme={null} AWSTemplateFormatVersion: '2010-09-09' Description: 'CloudFormation template for Cloudglue external S3 access role' Parameters: BucketName: Type: String Description: 'S3 bucket name that you want to grant access to for Cloudglue' Default: 'your-bucket-name-here' ExternalId: Type: String Description: 'External ID to prevent confused deputy problem (required for third-party access)' MinLength: 2 MaxLength: 1224 AllowedPattern: '^[a-zA-Z0-9+=,.@:/-]+$' Resources: CloudglueExternalS3AccessRole: Type: AWS::IAM::Role Properties: RoleName: cloudglue-external-s3-access-role Description: 'IAM role for Cloudglue external S3 access with cross-account permissions' AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: AWS: arn:aws:iam::992382634011:root Action: sts:AssumeRole Condition: StringEquals: sts:ExternalId: !Ref ExternalId ManagedPolicyArns: [] Policies: - PolicyName: CloudglueExternalS3AccessPolicy PolicyDocument: Version: '2012-10-17' Statement: - Sid: AllowCrossAccountRoleReadAccess Effect: Allow Action: - s3:GetObject - s3:ListBucket - s3:GetBucketLocation Resource: - !Sub 'arn:aws:s3:::${BucketName}' - !Sub 'arn:aws:s3:::${BucketName}/_' - Sid: AllowCrossAccountRoleWriteAccess Effect: Allow Action: - s3:PutObject Resource: - arn:aws:s3:::cloudglue-external-transfer - arn:aws:s3:::cloudglue-external-transfer/_ Outputs: RoleArn: Description: 'ARN of the created IAM role' Value: !GetAtt CloudglueExternalS3AccessRole.Arn Export: Name: !Sub '${AWS::StackName}-RoleArn' RoleName: Description: 'Name of the created IAM role' Value: !Ref CloudglueExternalS3AccessRole Export: Name: !Sub '${AWS::StackName}-RoleName' ``` **To use this template:** 1. Download the template file. 2. Navigate to the **CloudFormation** console in your AWS account. 3. Click **Create stack** (with new resources). 4. Select **Upload a template file** and upload the `.yaml` file. 5. Name the stack, set your `BucketName`, and the `externalId`. 6. Deploy the stack. ### Option 2: Manual Setup If you prefer manual configuration, you can create the required IAM role and policies yourself. **Important**: The IAM role MUST be named exactly `cloudglue-external-s3-access-role` for the connector to work. **To create the role** 1. Navigate to the **IAM console** -> **Roles**. 2. Click **Create role**. 3. Select **AWS account** as the trusted entity. 4. Select **Another AWS account** and enter our account ID: `992382634011`. AWS Role Step 1 5. Click **Next**. Skip the permissions page (we will add an inline policy later). 6. Name the role `cloudglue-external-s3-access-role`. 7. Click **Create role**. **Add policies** 1. Open the role you just created. 2. Click **Add permissions** -> **Create inline policy**. 3. Use the **JSON policy editor** and paste the following: Remember to replace `YOUR_BUCKET_NAME` with your actual S3 bucket name in the manual policy setup. #### Role Policies ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowCrossAccountRoleReadAccess", "Effect": "Allow", "Action": ["s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation"], "Resource": [ "arn:aws:s3:::YOUR_BUCKET_NAME", "arn:aws:s3:::YOUR_BUCKET_NAME/*" ] }, { "Sid": "AllowCrossAccountRoleWriteAccess", "Effect": "Allow", "Action": ["s3:PutObject"], "Resource": [ "arn:aws:s3:::cloudglue-external-transfer", "arn:aws:s3:::cloudglue-external-transfer/*" ] } ] } ``` 4. Click **Next**. 5. Name the policy and click **Create policy**. #### Trust Relationship The trust relationship is already attached to the role, so you can skip this step. ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::992382634011:root" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "ID_YOU_GENERATE" } } } ] } ``` Want to learn more about trust relationships? See the [AWS documentation on trust policies with IAM roles](https://aws.amazon.com/blogs/security/how-to-use-trust-policies-with-iam-roles/). Need help with CloudFormation templates? Check out the [AWS CloudFormation Templates resources](https://aws.amazon.com/cloudformation/resources/templates/). ## Connecting to Cloudglue 1. **Get the role ARN** and `externalId` from your AWS account. Role ARN with CloudFormation Role ARN with manual setup 2. Navigate to [Cloudglue data connectors](https://app.cloudglue.dev/home/data-connectors), selecting the **AWS S3** card, and complete the form. S3 Connector Architecture Ready to set up your data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to configure your S3 connector. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Managing Access You maintain full control over your data access. Remove this data connector by doing any of the following: * **Revoke access** by deleting the IAM role. * **Modify permissions** by updating the role policies. * **Remove connection** by revoking the connector on our website. Once removed: * You cannot import new recordings into Cloudglue until you reconnect. * Any recordings you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Adding more buckets If you need to add more buckets to the connector, you can do so by modifying the policy attached to the role. All you need to do is add the new bucket to the `Resource` array. ```json theme={null} { "Action": [ "s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::YOUR_BUCKET_NAME", "arn:aws:s3:::YOUR_BUCKET_NAME/*", // Add more buckets here "arn:aws:s3:::YOUR_BUCKET_NAME2", "arn:aws:s3:::YOUR_BUCKET_NAME2/*" ], "Effect": "Allow", "Sid": "AllowCrossAccountRoleReadAccess" }, ``` ## How to use your S3 files with Cloudglue To use S3 files, you can use the S3 URI for a file in your Cloudglue API requests. This would be prefixed with `s3://`. Example: ``` s3:/// ``` ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "s3:///", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ## Security We use AWS best practices with cross-account IAM roles. This approach: * **Maintains your control** - You own and manage the role. * **Follows AWS security patterns** - Uses standard cross-account role assumption. * **Provides granular permissions** - Only grants the minimum access needed. * **Easy to revoke** - Simply delete the role or modify permissions. # Bulk Imports Source: https://docs.cloudglue.dev/data-connectors/bulk-imports Import an entire connected source into a collection with saved, re-runnable import definitions A bulk import lists a connected data source and brings everything matching your filters into a collection — instead of calling the API once per file. It is a **saved definition** you can re-run, so "keep this collection in sync with that Zoom account" is a durable thing rather than a script you have to babysit. ## Two kinds of import What an import ingests follows the **collection you create it on**. You never choose this directly — it is reported back as `import_type`. | Collection type | `import_type` | What a run does | Cost | | --------------------------------------------------------------------- | ------------- | ----------------------------------------------------------------------------- | --------------- | | `metadata` | `metadata` | Indexes whatever metadata the source exposes. No media is downloaded. | Free | | `rich-transcripts`, `media-descriptions`, `entities`, `face-analysis` | `media` | Ingests and processes the actual files, exactly like adding each one by hand. | Billed per file | Which fields you get depends entirely on the connector — a metadata import stores what that source happens to publish about each file, and sources differ a lot. A Zoom recording carries things like a topic, host, start time and duration; a Gong call adds parties and its own call analysis; Grain contributes AI summaries and highlights; a Dropbox or Drive file may be little more than a name, size and timestamps; iconik brings whatever custom metadata your workspace has configured. Anything present is indexed and becomes searchable and filterable. The two compose into the workflow most teams want: bulk-index a whole source into a **metadata** collection for free, search it to find the files actually worth processing, then run a **media** import with narrower filters against a processing collection to ingest only those. ## Create an import ```bash theme={null} curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports" \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "name": "All sales calls", "connector_id": "'"$CONNECTOR_ID"'", "max_files": 500 }' ``` The first run starts immediately unless you pass `start: false`. The response echoes the definition plus `latest_run`. ### Filtering what gets imported `filters` is an array of listing passes, mirroring the parameters you would pass when browsing a connector. Each set runs as its own pass and results are deduplicated, so overlapping sets are safe. ```json theme={null} { "filters": [ { "from": "2026-07-01", "to": "2026-09-30", "title_search": "customer" } ] } ``` Which keys a connector honors depends on the connector — `folder_id` for Google Drive, `path` and `recursive` for Dropbox, `team` and `meeting_type` for Grain, `title_search` and date ranges for most. Unsupported keys are **rejected when you create the import** rather than silently ignored, because a saved definition re-runs forever and a quietly dropped filter would import the whole source. Zoom and Gong default to a six-month lookback when no `from` is given. That date is frozen into the definition at creation so re-runs keep listing the same window — otherwise a `refresh` + `delete_missing` run would remove files that had merely aged out of the window. ## Watch a run ```bash theme={null} curl "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID" \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" ``` `files_queued` is the denominator; `files_imported` (media) or `files_indexed` (metadata) plus `files_failed` settle against it. ```json theme={null} { "import_type": "media", "runs": [ { "status": "processing", "progress": { "files_listed": 180, "files_queued": 154, "files_imported": 96, "files_skipped": 26, "files_failed": 2 } } ] } ``` Large runs take as long as the work takes — hundreds of long videos is normal for hours. A run keeps going in the background; you can close the connection and poll later. ## Re-running: append, refresh, and mirroring Every run picks a mode, defaulting to the definition's `default_mode`. Imports files the collection does not have yet and retries previously failed ones. Files already imported are skipped: not re-downloaded, not re-billed. This is the mode to re-run on a schedule. Additionally re-fetches source metadata for files this import already brought in. For media imports the bytes are never re-downloaded — only the metadata is refreshed. After a **complete** listing, removes files this import previously added that the source no longer returns. Files added by a *different* import, files you added yourself, and files this import has never listed are all left alone. Only the file's place in this collection is removed — the file itself, and its use in any other collection, is untouched. A run that stopped at `max_files` skips this step entirely, because a truncated listing cannot prove anything is missing. To keep a file regardless of what the source does, add it to a collection this import does not target. ```bash theme={null} curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID/runs" \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "mode": "refresh", "delete_missing": true }' ``` ## When a run stops early Media imports are billed per file, so a run can stop on something account-wide — running out of credits, or hitting a usage limit. When that happens the run ends with a `failed` status and a plain-language `error`, and **everything already imported is kept**. Recovery is always the same move: fix the cause, then re-run in `append` mode. Already-imported files are skipped and only the remainder is attempted. ```bash theme={null} curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID/runs" \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "mode": "append" }' ``` Problems with a single file — too long, unsupported type, no longer in the source — fail only that file. The run continues and reports it in `files_failed`. ## Cancelling ```bash theme={null} curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID/runs/$RUN_ID/cancel" \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" ``` Files already imported stay in the collection. Deleting the import definition likewise removes only the definition and its run history — imported content is never removed. ## Things to know * **One active run per collection.** Starting a second while one is in flight returns a `409`; creating a definition still succeeds, it just comes back with `latest_run: null` so you can run it once the current one finishes. * **Definitions are immutable.** To change filters or connector, delete and recreate. * **`max_files`** caps the files a run takes from the listing, and a capped run never removes anything. * **Runs use the workspace's default API key** to read the connector, so the workspace needs one. ## Reference # Dropbox Source: https://docs.cloudglue.dev/data-connectors/dropbox Connect your Dropbox files to Cloudglue Connect your Dropbox account to use video files from Dropbox with Cloudglue APIs. ## Adding the Dropbox Data Connector 1. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 2. Click the Dropbox data connector card. Dropbox data connector focused 3. Follow the popup instructions to authorize Cloudglue access to your Dropbox account. 4. Verify the connection by checking for the **Connected** indicator on the card. Ready to set up your Dropbox data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to get started. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using Dropbox with Cloudglue We offer a couple of ways to use your Dropbox files with Cloudglue. ### Option 1: Share Links via API We support Dropbox file share links — the links you get from **Share → Copy link** on a file in Dropbox. Example: ``` https://www.dropbox.com/scl/fi/SOME_ID/video.mp4?rlkey=SOME_KEY&st=SOME_TIME&dl=0 ``` Share links work in two ways: * **General endpoints** (describe, transcribe, extract, add to collection): the link is downloaded anonymously, so it must be publicly accessible. Login-gated or expired links return a `403` explaining the access problem. * **Connector sync** (`POST /v1/data-connectors/{id}/sync` on a Dropbox connector): the link is resolved through your connected account's access, so it also works for files that require login, as long as the connected account can access them. The same file pasted as different copies of the link (Dropbox regenerates the `st=` parameter each time a link is copied) deduplicates to a single Cloudglue file. Share links must point to a single **file**. Folder share links (`/scl/fo/...`) are not supported and return a `400`. ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://www.dropbox.com/scl/fi/SOME_ID/video.mp4?rlkey=SOME_KEY&st=SOME_TIME&dl=0", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ### Option 2: Custom URI via API To use Dropbox files via our API that Cloudglue has access to, you can use the `dropbox://` prefix in your Cloudglue API requests. This is the URI format returned by [`GET /v1/data-connectors/{id}/files`](/api-reference/endpoint/data-connectors/list-files) — you can pass those URIs back verbatim. Example: ``` dropbox://// ``` The path is matched tolerantly: leading-slash count and percent-encoding variations of the same path (e.g. `dropbox:///folder name/file.mp4` vs `dropbox://folder%20name/file.mp4`) resolve and deduplicate to the same file. ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "dropbox:////", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ## Browsing and filtering files When [listing connector files](/api-reference/endpoint/data-connectors/list-files), `title_search` matches file names and `from`/`to` filter by the file's `client_modified` date (matched while paging — Dropbox has no native filters). `path` (from a folder entry's `metadata`) drills into folders. Because these filters are matched while paging, a filtered page can contain fewer than `limit` items — even zero — while more matches remain. Keep paginating until `next_page_token` is null rather than stopping at the first short or empty page. ## Source metadata When a Dropbox file is imported, Cloudglue captures the file's source metadata onto the resulting file under `source_metadata`: name, paths, size, client/server modification times, revision, content hash, and — for video — `media_info` with duration and dimensions. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id). `media_info` is only captured at sync/lookup time: Dropbox's list endpoint stopped returning media info, so items seen while browsing carry `null` there. Public `dl.dropboxusercontent.com` URLs ingested without the connector carry no source metadata at all (there's no API metadata for anonymous links). [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) previews a file's source metadata without importing it (works with `dropbox://` paths and share links). URL-encode the `url` parameter — real paths can contain spaces, `&`, `#`, or percent-encoded characters: ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=dropbox%3A%2F%2Fvideos%2Fdemo.mp4' \ --header 'Authorization: Bearer YOUR_API_KEY' # url = encodeURIComponent('dropbox://videos/demo.mp4') ``` ## UI Importing for Testing Use the Cloudglue integration UI to quickly import Dropbox files for testing your endpoints. File Chooser To access this page, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click on **Select Videos** on the Dropbox connector. Quick Import Button ## Managing Access You maintain full control over your data access. Remove this data connector by doing any of the following: * **Revoke access** by deleting the Dropbox app from your Dropbox account. * **Remove connection** by revoking the connector on our website. Once removed: * You cannot import new videos from Dropbox until you reconnect. * Any videos you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security Cloudglue uses the [Dropbox app](https://www.dropbox.com/app-integrations) to access your files. This approach: * **Maintains your control** - You own and manage the app. * **Follows Dropbox security patterns** - Uses standard Dropbox app access. * **Provides granular permissions** - Only grants the minimum access needed. * **Easy to revoke** - Simply delete the app or modify permissions. # Gong Source: https://docs.cloudglue.dev/data-connectors/gong Connect your Gong recordings to Cloudglue Connect your Gong account to use your cloud meeting recordings with Cloudglue APIs. ## Adding the Gong Data Connector 1. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 2. Click the Gong data connector card. Gong data connector focused 3. Follow the popup instructions to authorize Cloudglue access to your media. 4. Verify the connection by checking for the **Connected** indicator on the card. Ready to set up your Gong data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to get started. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using Gong with Cloudglue To use Gong recordings with Cloudglue, you can use the `gong://call/` prefix in your Cloudglue API requests. Example: ```bash theme={null} gong://call/1234567890 ``` **Example API Request to get [a multimodal description](/api-reference/endpoint/describe/post) of a recording:** ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "gong://call/1234567890", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ## Browsing and filtering calls [`GET /data-connectors/{id}/files`](/api-reference/endpoint/data-connectors/list-files) lists calls from your Gong workspace, newest first. Private calls are never listed (they also can't be synced). `from`/`to` filter by call start date — when `from` is omitted, listing looks back 6 months — and `title_search` matches call titles. ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/files?from=2026-06-01&title_search=kickoff' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` When filters are applied, a page can contain fewer than `limit` items — even zero — while more calls remain. Keep paginating until `next_page_token` is null rather than stopping at the first short or empty page. Listed calls include audio-only and still-processing calls for visibility, but **only calls with video media can currently be synced** — check each item's `mime_type`: `video/mp4` means importable, `audio/mpeg` or `null` means sync will fail with a not-found error. ## Source metadata When a Gong call is imported, Cloudglue captures the call's source metadata onto the resulting file under `source_metadata`: title, start/scheduled times, duration, direction, system, scope, language, workspace, the participants (names, emails, affiliations), and — when your Gong tenant has Call Spotlight — the AI brief, key points, topics, and trackers. Signed media URLs are never stored. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id). The AI content fields are captured at sync/lookup time. Items returned while browsing carry the call facts only, and tenants without Call Spotlight simply get `null` for the AI fields. [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) previews a call's source metadata without importing it: ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=gong://call/1234567890' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## UI Importing for Testing Use the Cloudglue integration UI to quickly import Gong files for testing your endpoints. File Chooser To access this page, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click on **Select Videos** on the Gong connector. Quick Import Button ## Managing Access You maintain full control over your data access. Remove this data connector by doing the following: * **Remove connection** by revoking the connector on our [website](https://app.cloudglue.dev/home/data-connectors). Once removed: * You cannot import new recordings into Cloudglue until you reconnect. * Any recordings you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security The Gong data importer does not have access to videos marked as private on the Gong platform. To access these videos, you can manually add the recording to Cloudglue using the API. Cloudglue uses Gong OAuth to securely access your recordings with minimum required permissions. This approach: * **Maintains your control** - You own and manage the connector * **Follows Gong security standards** - Uses OAuth with minimum permissions # Google Cloud Storage Source: https://docs.cloudglue.dev/data-connectors/google-cloud-storage Connect your Google Cloud Storage buckets to Cloudglue The Google Cloud Storage data connector allows you to connect any Google Cloud Storage bucket to Cloudglue, enabling you to use your Google Cloud Storage URIs directly with our endpoints without manually uploading files. Google Cloud buckets inside of a **personal Google account** are not supported with this data connector. ## How It Works This connector creates a secure bridge between your Google Cloud Storage buckets and Cloudglue: * **Direct Google Cloud Storage access** - We only access your files when you explicitly use them, and only the files you give access to. * **No manual uploads** - Skip the process of manually uploading files to Cloudglue. * **Google Cloud Storage URI support** - Use your existing Google Cloud Storage URIs directly with Cloudglue endpoints. ## Google Cloud Storage Setup 1. Go to the [Google Cloud Storage console](https://console.cloud.google.com/storage/browser). 2. Navigate to the bucket(s) you want to grant access to. 3. Navigate to the **Permissions** tab in the bucket. 4. On the **Permissions** tab, select **View by principals**, then click **Grant Access**. 5. For the field **New principals**, enter the following service account email: ``` cloudglue-partner-access@cloudglue.iam.gserviceaccount.com ``` 6. Assign the following role to the service account: **Storage Object Viewer**. 7. Click **Save**. Example: Grant access to the service account to the bucket ## Connecting to Cloudglue 1. **Enable the connector** on our web platform for Google Cloud Storage. Google Cloud Storage Connector Architecture 2. **Start using Google Cloud Storage URIs** directly with Cloudglue endpoints. Ready to set up your data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to configure your GCS connector. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## How to use your Google Cloud Storage files with Cloudglue To use Google Cloud Storage files, you can use the Google Cloud Storage URI (also known as gsutil URI) for a file in your Cloudglue API requests. This would be prefixed with `gs://`. Example: ``` gs:/// ``` You can copy this from the Google Cloud Storage console for the file you want to use. Example: Google Cloud Storage URI ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "gs:///", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ## Managing Access You maintain full control over your data access. Remove this data connector by doing any of the following: * **Revoke access** by removing bucket access for the service account in the Google Cloud console. * **Remove connection** by revoking the connector on our website. ## Security We use a special Google Cloud Storage service account to access your buckets. This service account is limited to read-only access and it can only view the specific buckets and files you authorize. * **Provides granular permissions** - Only grants the minimum access needed. * **Easy to revoke** - Simply remove bucket access for the service account. # Google Drive Source: https://docs.cloudglue.dev/data-connectors/google-drive Connect your Google Drive files to Cloudglue Connect your Google Drive account to use your media files with Cloudglue APIs. ## Adding the Google Drive Data Connector 1. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 2. Click the Google Drive data connector card. Google Drive data connector focused 3. Follow the popup instructions to authorize Cloudglue access to your media. 4. Verify the connection by checking for the **Connected** indicator on the card. Ready to set up your Google Drive data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to get started. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using Google Drive files with Cloudglue To use Google Drive files, you can use the `gdrive://file/` prefix in your Cloudglue API requests, or paste a Google Drive file link directly — both forms are accepted everywhere and deduplicate to the same Cloudglue file. Accepted formats: ```bash theme={null} gdrive://file/ https://drive.google.com/file/d//view https://drive.google.com/open?id= ``` Links must point to a single **file**. Google Drive folder links (`drive.google.com/drive/folders/...`) are not supported and return a `400`. Drive links resolve through your connected Google Drive account, so a connector is required. ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "https://drive.google.com/file/d//view", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ## Browsing and filtering files When [listing connector files](/api-reference/endpoint/data-connectors/list-files), `title_search` matches file names and `from`/`to` filter by Drive creation date — both natively via Drive's query API. `folder_id` (from a folder entry's `metadata`) drills into folders. ## Source metadata When a Drive file is imported, Cloudglue captures the file's source metadata onto the resulting file under `source_metadata`: name, MIME type, size, created/modified times, owners and last modifying user, parent folders, sharing status, MD5 checksum, web view link, and Drive's video metadata (duration/dimensions) when available. Short-lived signed URLs (like thumbnail links) are never stored. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id), and the same shape appears in each file entry's `metadata` while browsing. [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) previews a file's source metadata without importing it: ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=gdrive://file/YOUR_FILE_ID' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## UI Importing for Testing Use the Cloudglue integration UI to quickly import Google Drive files for testing your endpoints. File Chooser To access this page, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click on **Select Videos** on the Google Drive connector. Quick Import Button ## Managing Access You maintain full control over your data access. Remove this data connector by doing the following: * **Remove connection** by revoking the connector on our website. Once removed: * You cannot import new recordings into Cloudglue until you reconnect. * Any recordings you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security Cloudglue uses Google Drive Authentication to get limited access to your files. This approach: * **Maintains your control** - You own and manage the connector. * **Follows Google Drive security patterns** - Uses standard Google Drive authentication with minimum permissions. * **Easy to revoke** - Simply delete the connector on our website. # Grain Source: https://docs.cloudglue.dev/data-connectors/grain Connect your Grain recordings to Cloudglue Connect your Grain account to use meeting recordings from [Grain](https://grain.com) with Cloudglue APIs. The OAuth Grain data connector currently supports personal Grain accounts only. For workspace-level access, contact the [Cloudglue team](mailto:support@cloudglue.dev). ## Adding the Grain Data Connector 1. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 2. Click the **Grain** data connector card. 3. Follow the popup to sign in to Grain and authorize Cloudglue (OAuth with PKCE). 4. Verify the connection by checking for the **Connected** indicator on the card. Ready to set up your Grain data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to get started. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using Grain with Cloudglue To use Grain recordings with Cloudglue, you can use the `grain://recording/` prefix in your Cloudglue API requests. Example: ```bash theme={null} grain://recording/00000000-0000-4000-8000-000000000000 ``` **Example API Request to get [a multimodal description](/api-reference/endpoint/describe/post) of a recording:** ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "grain://recording/00000000-0000-4000-8000-000000000000", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` When [listing connector files](/api-reference/endpoint/data-connectors/list-files) for a Grain connector, optional query parameters such as date range, title search, team, and meeting type follow the [Grain recording filter](https://developers.grain.com/#recording-filter) model. ## Source metadata When a Grain recording is imported, Cloudglue captures the recording's source metadata onto the resulting file under `source_metadata`. This includes the title, participants, teams, meeting type, tags, start/end times, and (when Grain returns them) the AI summary, action items, calendar event, and HubSpot associations. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id). ### Sync a recording into a file [`POST /data-connectors/{id}/sync`](/api-reference/endpoint/data-connectors/sync) materializes a Grain recording URI into a Cloudglue file — with `source_metadata` populated — without starting a downstream job. It's idempotent: syncing the same URI returns the existing file. ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/sync \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "grain://recording/00000000-0000-4000-8000-000000000000" }' ``` ### Look up source metadata without importing [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) fetches the recording's source metadata directly from Grain without creating a file — useful for previewing a recording before importing it. ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=grain://recording/00000000-0000-4000-8000-000000000000' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## UI Importing for Testing Use the Cloudglue integration UI to browse Grain recordings and add them to a collection for testing. To access this flow, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click **Select Videos** on the Grain connector after it shows as connected. ## Managing Access You maintain full control over your data access. Remove this data connector from the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors). Once removed: * You cannot import new recordings into Cloudglue until you reconnect. * Any recordings you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security Cloudglue connects to Grain using **OAuth** (authorization code with PKCE on the web app, then short-lived access tokens with refresh). You authorize access in Grain; Cloudglue stores tokens scoped to your connector so you can revoke access by disconnecting the connector in Cloudglue and managing or revoking the OAuth client in Grain as needed. For Grain’s own API behavior, rate limits, and data model details, see the [Grain Public API](https://developers.grain.com/#grain-api). # iconik Source: https://docs.cloudglue.dev/data-connectors/iconik Connect your iconik media library to Cloudglue Connect your [iconik](https://www.iconik.io) media asset management account to use your video and audio assets with Cloudglue APIs. ## Adding the iconik Data Connector 1. In iconik, create an Application Token: go to **Admin → Settings → Application Tokens**, enter an application name (e.g. "Cloudglue"), select the user the token should act as, and click **Add**. Copy both the **App-ID** and the **Token** — iconik shows the token only once. 2. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 3. Click the iconik data connector card and enter the App-ID and Auth Token. Keep the default iconik URL (`https://app.iconik.io`) unless your iconik system is served from another domain. 4. Verify the connection by checking for the **Connected** indicator on the card. The user the Application Token acts as determines which assets Cloudglue can see. The token needs read access to assets, formats, files, and proxies (e.g. the `can_read_assets`, `can_read_formats`, `can_read_files`, and `can_read_proxies` roles). **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using iconik with Cloudglue To use iconik assets with Cloudglue, use the `iconik://asset/{asset_id}` prefix in your Cloudglue API requests. Example: ```bash theme={null} iconik://asset/b947e9da-ec52-11ea-b571-0af7c14f78f9 ``` **Example API Request to get [a multimodal description](/api-reference/endpoint/describe/post) of an asset:** ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "iconik://asset/b947e9da-ec52-11ea-b571-0af7c14f78f9", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` When an asset is imported, Cloudglue downloads the asset's web proxy (a compact, web-ready rendition) when one is available, and falls back to the ORIGINAL file otherwise. Proxies are ideal for AI analysis — much smaller than camera originals with no meaningful quality loss for understanding tasks. The rendition used is recorded as `ingested_rendition` in the file's `source_metadata`. ## Browsing and filtering assets [`GET /data-connectors/{id}/files`](/api-reference/endpoint/data-connectors/list-files) lists active video and audio assets from your iconik account, newest first. `from`/`to` filter by asset creation date, and `title_search` runs a full-text search on asset titles. ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/files?title_search=interview&from=2026-01-01' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` A result set can span multiple pages: when `has_more` is true, pass the response's `next_page_token` back as `page_token` (with the same filter parameters) until it returns null. Each listed asset includes a `thumbnail_url` — the asset's keyframe as a signed iconik URL that expires within hours. Use it to render previews while browsing; never store it (imported files get a durable Cloudglue-hosted thumbnail instead — see [Poster thumbnails](#poster-thumbnails)). ## Source metadata When an iconik asset is imported, Cloudglue captures the asset's source metadata onto the resulting file under `source_metadata`: the asset id and title, media type, creation/modification timestamps, duration, the rendition that was ingested (`proxy` or `original`), and a deep link back to the asset in iconik. Signed download URLs are never stored. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id). ### Custom metadata views (`iconik_metadata`) Values from your iconik **metadata views** — the customer-defined custom fields you see on an asset's "Update Metadata" panel (e.g. a screening view's UGC Title, Description, Keywords, Categories, Ratings) — are captured under `source_metadata.iconik_metadata`, keyed by field name. Only populated fields are stored (iconik reports every field defined across your domain's views, most of them empty), and values keep iconik's array wrapping — including for single-value fields: ```json theme={null} { "source_type": "iconik", "iconik_asset_id": "1c2090a4-…", "title": "5cdc99d9-….mp4", "iconik_metadata": { "uGCTitle": ["Do you have it?"], "keywords": ["whoops fail kicking hitting father daughter"], "aFVRating": ["5 Minus"], "emotionMultiPicklist": ["Cute", "Funny", "Happy", "Thrill"] } } ``` This makes iconik custom metadata a first-class search surface: * In [metadata collections](/api-reference/endpoint/collections/post), every custom field is flattened into the file's indexed search document — so a semantic or `doc_lexical` search over descriptions, keywords, and categories finds the right assets without downloading any media. * Any search or list endpoint accepting `source_metadata` filters can target individual fields with `source_metadata.iconik_metadata.` paths (use `ContainsAny` for multi-value fields like keyword lists). Field names are whatever your iconik views define — check a file's `source_metadata` (or the [source-metadata preview](/api-reference/endpoint/data-connectors/source-metadata)) to see the exact keys for your library. [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) previews an asset's source metadata without importing it: ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=iconik://asset/b947e9da-ec52-11ea-b571-0af7c14f78f9' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## Poster thumbnails iconik assets added to [metadata collections](/api-reference/endpoint/collections/post) are indexed without downloading any media — but they still get a visual identity: Cloudglue copies the asset's **poster keyframe** (your custom keyframe if one is set, otherwise the largest still) into Cloudglue-hosted storage and returns it as the file's `thumbnail_url`. This happens automatically at import, and [`POST /files/{file_id}/sync`](/api-reference/endpoint/files/sync-source-metadata) backfills a thumbnail for iconik files that don't have one yet. Fully ingested files generate thumbnails from the media itself, so the poster copy only applies when no thumbnail exists. iconik's own keyframe URLs are signed and short-lived, and — like download URLs — are never stored. ## UI Importing for Testing Use the Cloudglue integration UI to quickly import iconik assets for testing your endpoints. To access this page, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click on **Select Videos** on the iconik connector. You can browse recent assets, search by title, and import up to 20 assets at a time. ## Managing Access You maintain full control over your data access. To remove the iconik Data Connector, visit the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) and click **Disconnect** under iconik. Once removed: * You cannot import new assets into Cloudglue until you reconnect. * Any assets you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security Cloudglue uses an iconik Application Token to securely access your assets. This approach: * **Maintains your control** - You own and manage the connector. Revoke the Application Token in iconik at any time to stop Cloudglue from accessing your assets. * **Scopes access** - The token acts as the iconik user you selected, so Cloudglue can only see the assets that user can see. # Local Files Source: https://docs.cloudglue.dev/data-connectors/local-files Add video files directly from your computer Add video files (MP4, MOV, WEBM) directly from your computer to a Cloudglue [collection](/core-concepts/entity-collection). ## Add on the dashboard From the **Data Connectors** page, click **Local Files** under Other Sources. Select your files, pick a collection, and add them. Add local files modal ## Add with the API/SDK ```javascript theme={null} import Cloudglue from '@cloudglue/cloudglue-js'; const client = new Cloudglue({ apiKey: 'YOUR_API_KEY' }); // Upload a file const { data } = await client.files.uploadFile({ file }); // Add it to a collection await client.collections.addVideo(collectionId, data.id); ``` See [Upload File](/api-reference/endpoint/files/post) and [Add Video to Collection](/api-reference/endpoint/collections/files-post) for the full API reference. ## Supported formats * **MP4**, **MOV**, **WEBM** * Up to 400MB per file * Up to 5 files at a time via the dashboard ## For larger files or bulk imports For bulk imports or files larger than 400MB, use a [data connector](/data-connectors/overview#data-connectors) or the [API/SDK](/api-reference/endpoint/files/post) directly. # Loom Source: https://docs.cloudglue.dev/data-connectors/loom Add Loom URLs to Cloudglue for analysis Add public Loom video URLs to a Cloudglue [collection](/core-concepts/entity-collection). Loom videos are processed with full multimodal analysis (audio, visuals, and text). Only Loom videos with general access set to public can be uploaded to Cloudglue. Videos with general access set to workspace only or restricted will fail on upload. ## Add on the dashboard From the **Data Connectors** page, click **Loom** under Other Sources. Paste one or more public share Loom URLs, pick a collection, and add them. ## Add with the API/SDK ```javascript theme={null} const result = await client.collections.addVideoByUrl({ collectionId: 'my-collection-id', url: 'https://www.loom.com/share/12345678901234567890123456789012', params: {}, }); ``` See [Add Media to Collection](/api-reference/endpoint/collections/media-post) for the full API reference. To sync a Loom URL into a standalone file without a collection, use [Sync File from URL](/api-reference/endpoint/files/sync): ```bash theme={null} curl -X POST https://api.cloudglue.dev/v1/files/sync \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://www.loom.com/share/12345678901234567890123456789012"}' ``` # How It Works Source: https://docs.cloudglue.dev/data-connectors/overview Connect your data sources to add files to Cloudglue Cloudglue connects directly to platforms where your videos already live. Once connected, you can start adding videos to your Cloudglue workspace — building a structured context layer you can search, query, and build on. ## How It Works Choose between adding files directly, or via one of our data connectors. Connect cloud sources like Google Drive, Zoom, S3, and more. Add video files from your computer. Paste Loom, YouTube, TikTok, or any HTTP URL. ### Data Connectors Data connectors give Cloudglue secure access to your cloud video sources, making it easy to add many files at a time and optimized for large files. Connect once, then add videos anytime. #### Supported Data Connectors We're continuously adding more: Need a connector that's not listed? [Contact our team](mailto:support@cloudglue.dev) to discuss adding support for your data source. #### 1. Select the connector you'd like to grant Cloudglue access to Data Connectors page showing available connectors [Go to Data Connectors →](https://app.cloudglue.dev/home/data-connectors) #### 2. Authenticate Each connector has its own authentication steps. See the individual connector pages above for setup instructions. #### 3. You're ready to add videos! Once connected, you can add videos to a [collection](/core-concepts/entity-collection) in two ways: Browse your connected source and pick which videos to add to a collection. Best for quick testing and one-off imports. Connected data connectors with Select Videos button Select videos to analyze from Google Drive Use connector URIs directly in any Cloudglue API call. Best for production pipelines and bulk imports. ```javascript theme={null} const fileInfo = await client.collections.addVideoByUrl({ collectionId: 'my-collection-id', url: 'gdrive://file/1a2b3c4d5e6f', params: {}, }); ``` Supported URI formats: `gdrive://file/` or `drive.google.com` file links · `s3:///` · `zoom://uuid/` or Zoom recording-detail links · `gong://call/` · `grain://recording/` · `iconik://asset/` · `dropbox://` or Dropbox file share links See each connector's page for its full URI format and examples. #### 4. Start working with your media That's it! You can now start working with your videos — chat across them, search for specific moments, extract structured data, and more. The possibilities are endless. :) ```javascript theme={null} // Ask a question across all videos in a collection const response = await client.collections.chat({ collectionId: 'my-collection-id', messages: [ { role: 'user', content: "What topics were discussed in this week's meetings?", }, ], }); console.log(response.message.content); ``` Explore what you can do: Find specific moments across your videos Ask questions across your collections Pull structured data from video content Get rich descriptions of your videos ### Local Files Add video files (MP4, MOV, WEBM) directly from your computer. [Learn more →](/data-connectors/local-files) **Add on the dashboard:** Add local files modal **Add with the API/SDK:** ```javascript theme={null} const { data } = await client.files.uploadFile({ file }); await client.collections.addVideo(collectionId, data.id); ``` For bulk imports or larger files, use a [data connector](/data-connectors/overview#data-connectors) or the [API/SDK](/api-reference/endpoint/files/post) directly. ### Public URLs You can also add videos by pasting a URL — no data connector setup needed. #### HTTP URLs Add any publicly accessible video via its URL. [Learn more →](/data-connectors/public-urls) ```javascript theme={null} const result = await client.collections.addVideoByUrl({ collectionId: 'my-collection-id', url: 'https://example.com/video.mp4', params: {}, }); ``` #### TikTok URLs Paste public TikTok video URLs to add them to a collection. Full multimodal analysis (audio, visuals, and text). [Learn more →](/data-connectors/tiktok) Add TikTok videos modal #### YouTube URLs Paste public YouTube video URLs to add them to a collection. [Learn more →](/data-connectors/youtube) YouTube URLs are processed as **audio only**. For full video analysis including visual descriptions, add the original video file instead. Add YouTube videos modal *** #### Loom URLs Paste public Loom share video URLs to add them to a collection. Full multimodal analysis (audio, visuals, and text). [Learn more →](/data-connectors/loom) ## Next Steps Ready to get started? Dive into the guides for each method: * **[Data Connectors](/data-connectors/overview#supported-data-connectors)** — Set up Google Drive, Zoom, S3, Gong, and more. Each guide walks you through authentication and your first import. * **[Public URLs](/data-connectors/public-urls)** — Add videos from HTTP URLs, YouTube, TikTok, or Loom — just paste and go. * **[Local Files](/data-connectors/local-files)** — Add video files directly from your computer, or use the API/SDK for programmatic uploads. # HTTP URLs Source: https://docs.cloudglue.dev/data-connectors/public-urls Add videos from any public HTTP URL to Cloudglue Add any publicly accessible video directly to a Cloudglue [collection](/core-concepts/entity-collection) by passing its URL. ## Add with the API/SDK ```javascript theme={null} const result = await client.collections.addVideoByUrl({ collectionId: 'my-collection-id', url: 'https://example.com/video.mp4', params: {}, }); ``` The video will be added to Cloudglue and can be processed with full multimodal analysis (audio, visuals, and text). See [Add Media to Collection](/api-reference/endpoint/collections/media-post) for the full API reference. ## Sync without a collection To materialize a URL into a Cloudglue file without adding it to a collection, use [Sync File from URL](/api-reference/endpoint/files/sync). The call is idempotent — syncing the same URL again returns the existing file. ```bash theme={null} curl -X POST https://api.cloudglue.dev/v1/files/sync \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com/video.mp4"}' ``` This accepts direct video URLs, public Dropbox share links, TikTok URLs, and Loom share URLs. YouTube URLs are not supported here — add those to a collection instead. Connector URIs (`s3://`, `gdrive://file/`, …) sync through their [data connector](/data-connectors/overview#data-connectors) via [Sync Data Connector File](/api-reference/endpoint/data-connectors/sync). ## Supported URL types Cloudglue supports several URL types: * **Direct video URLs** — any public URL pointing to a video file (e.g. `https://example.com/video.mp4`) * **[YouTube URLs](/data-connectors/youtube)** — public YouTube video URLs (audio-only processing) * **[TikTok URLs](/data-connectors/tiktok)** — public TikTok video URLs (full multimodal) * **[Loom](/data-connectors/loom)** - public Loom share URLs (multimodal) * **[Dropbox file share links](/data-connectors/dropbox)** — `www.dropbox.com/scl/fi/...` links download anonymously when publicly accessible; login-gated links return a `403` (sync them through a Dropbox connector instead) * **[Data connector URIs](/data-connectors/overview#data-connectors)** — connector-specific URIs like `gdrive://file/`, `s3:///`, etc. ## URL types with special handling * **[Google Drive links](/data-connectors/google-drive)** (`drive.google.com/file/d/...`) resolve through your connected Google Drive connector — they are not anonymous downloads. * **Video page links from Vimeo, OneDrive (`1drv.ms`, `onedrive.live.com`), and Box** are not supported: these pages serve HTML, not video bytes, so they fail fast with a `400` naming the host. Provide a direct video file URL instead (known direct-download forms on those hosts, like `app.box.com/shared/static/...` or `player.vimeo.com/progressive_redirect/...`, still work). # Recall.ai Source: https://docs.cloudglue.dev/data-connectors/recall Connect your Recall.ai meeting recordings to Cloudglue Connect your Recall.ai account to use your media recordings with Cloudglue APIs. ## Adding the Recall Data Connector 1. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 2. Click the Recall data connector card. Recall data connector focused 3. Follow the popup instructions to authorize Cloudglue access to your recordings. 4. Verify the connection by checking for the **Connected** indicator on the card. Ready to set up your Recall data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to get started. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using Recall with Cloudglue To use Recall.ai recordings with Cloudglue, you can use the `recall://recording/{recording_id}` prefix in your Cloudglue API requests. Example: ```bash theme={null} recall://recording/1234567890 ``` **Example API Request to get [a multimodal description](/api-reference/endpoint/describe/post) of a recording:** ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "recall://recording/1234567890", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` ## Browsing and filtering recordings [`GET /data-connectors/{id}/files`](/api-reference/endpoint/data-connectors/list-files) lists recordings from your Recall account, and `from`/`to` filter by recording creation date. ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/files?from=2026-06-01&to=2026-06-30' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` A date range can span multiple pages: when `has_more` is true, pass the response's `next_page_token` back as `page_token` (with the same filter parameters) until it returns null. Recall's list API exposes no meeting title, so items are named `Recall Recording {id}`; the title (when Recall has one) appears as `meeting_title` inside each item's `metadata`. `title_search` is ignored for Recall. Recordings of every status are listed — check `metadata.status_code`; only `done` recordings can be synced. ## Source metadata When a Recall recording is imported, Cloudglue captures the recording's source metadata onto the resulting file under `source_metadata`: timestamps (created/started/completed/expires), status, meeting title and platform when available, artifact availability flags (`has_transcript`, `has_audio`, `has_participant_events`), the capturing bot's id, and any custom metadata you attached to the bot in Recall. Signed artifact download URLs are never stored. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id). [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) previews a recording's source metadata without importing it: ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=recall://recording/1234567890' \ --header 'Authorization: Bearer YOUR_API_KEY' ``` ## UI Importing for Testing Use the Cloudglue integration UI to quickly import Recall files for testing your endpoints. File Chooser To access this page, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click on **Select Videos** on the Recall connector. Quick Import Button ## Managing Access You maintain full control over your data access. To remove the Recall Data Connector, visit the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) and click **Disconnect** under Recall. Once removed: * You cannot import new recordings into Cloudglue until you reconnect. * Any recordings you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security Cloudglue uses your Recall.ai API key to securely access your recordings. This approach: * **Maintains your control** - You own and manage the connector. Revoke the API Key at any time to stop Cloudglue from accessing your recordings. # TikTok Source: https://docs.cloudglue.dev/data-connectors/tiktok Add TikTok URLs to Cloudglue for analysis Add public TikTok video URLs to a Cloudglue [collection](/core-concepts/entity-collection). TikTok videos are processed with full multimodal analysis (audio, visuals, and text). ## Add on the dashboard From the **Data Connectors** page, click **TikTok** under Other Sources. Paste one or more TikTok URLs, pick a collection, and add them. Add TikTok videos modal ## Add with the API/SDK ```javascript theme={null} const result = await client.collections.addVideoByUrl({ collectionId: 'my-collection-id', url: 'https://www.tiktok.com/@user/video/1234567890', params: {}, }); ``` See [Add Media to Collection](/api-reference/endpoint/collections/media-post) for the full API reference. To sync a TikTok URL into a standalone file without a collection, use [Sync File from URL](/api-reference/endpoint/files/sync): ```bash theme={null} curl -X POST https://api.cloudglue.dev/v1/files/sync \ -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://www.tiktok.com/@user/video/1234567890"}' ``` # YouTube Source: https://docs.cloudglue.dev/data-connectors/youtube Add YouTube URLs to Cloudglue for analysis Add public YouTube video URLs to a Cloudglue [collection](/core-concepts/entity-collection). YouTube URLs are processed as **audio only**. For full video analysis including visual descriptions, add the original video file via [Local Files](/data-connectors/local-files) or a [data connector](/data-connectors/overview#data-connectors) instead. ## Add on the dashboard From the **Data Connectors** page, click **YouTube** under Other Sources. Paste one or more YouTube URLs, pick a collection, and add them. Add YouTube videos modal ## Add with the API/SDK ```javascript theme={null} const result = await client.collections.addVideoByUrl({ collectionId: 'my-collection-id', url: 'https://www.youtube.com/watch?v=dQw4w9WgXcQ', params: {}, }); ``` See [Add Media to Collection](/api-reference/endpoint/collections/media-post) for the full API reference. YouTube URLs can only be added to a collection. [Sync File from URL](/api-reference/endpoint/files/sync) does not accept YouTube URLs. # Zoom Source: https://docs.cloudglue.dev/data-connectors/zoom Connect your Zoom recordings to Cloudglue Connect your Zoom account to use your cloud meeting recordings with Cloudglue APIs. ## Adding the Zoom Data Connector 1. Navigate to the [Data Connectors page](https://app.cloudglue.dev/home/data-connectors) in the Cloudglue app. 2. Click the Zoom data connector card. Zoom data connector focused 3. Follow the popup instructions to authorize Cloudglue access to your recordings. 4. Verify the connection by checking for the **Connected** indicator on the card. Ready to set up your Zoom data connector? Visit [our app](https://app.cloudglue.dev/home/data-connectors) to get started. **Need help setting up your data connector?** Contact our team directly for assistance with the setup process. ## Using Zoom with Cloudglue Cloudglue accepts multiple URL formats for referencing Zoom recordings. Zoom has two meeting identification concepts: * **Meeting ID** - A numeric identifier for a meeting (can have multiple instances if recurring). * **Meeting Instance UUID** - A unique identifier for a specific meeting occurrence. Cloudglue uses a [URL encoded](https://developer.mozilla.org/en-US/docs/Glossary/Percent-encoding) UUID. ### Supported URL Formats Retrieves the recording from a specific meeting instance. If multiple recordings exist, Cloudglue uses the longest one. **Cloudglue URL format:** > **Note:** When using a meeting instance UUID, you must URL-encode it (for example, using `encodeURIComponent()` in JavaScript). The example below shows the encoded form on the left, and the code used to generate it on the right. ```bash theme={null} zoom://uuid/QFwZYEreTl2e6MBFSslXjQ%3D%3D # zoom://uuid/${encodeURIComponent("QFwZYEreTl2e6MBFSslXjQ==")} ``` [**Example API Request to get a multimodal description of a recording:**](/api-reference/endpoint/describe/post) ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "zoom://uuid/QFwZYEreTl2e6MBFSslXjQ%3D%3D", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` Retrieves the recording from the most recent instance. If multiple recordings exist in that instance, retrieves the longest one. **Cloudglue URL format:** ```bash theme={null} zoom://id/81586198865 ``` [**Example API Request to get a multimodal description of a recording:**](/api-reference/endpoint/describe/post) ```bash theme={null} curl --request POST \ --url https://api.cloudglue.dev/v1/describe \ --header 'Authorization: Bearer YOUR_API_KEY' \ --header 'Content-Type: application/json' \ --data '{ "url": "zoom://id/81586198865", "enable_summary": true, "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true }' ``` You can also paste Zoom web links directly: * **Recording detail links** (recommended): `https://zoom.us/recording/detail?meeting_id=` — the URL in your browser when viewing a recording in the Zoom web portal. Works reliably for recordings of any age. * **Meeting join links**: `https://zoom.us/j/` (or `/s/`) — resolves to the latest recording of that meeting. * **Recording share links** (best effort): `https://*.zoom.us/rec/share/` — resolved by matching against your account's cloud recordings. Zoom often generates a *new* share token each time a link is copied, so a freshly copied link may not be locatable; if that happens, use the recording detail link instead. Only recordings owned by the connected Zoom user can be found this way. `rec/play` links are not supported and return a `400` with guidance. Regardless of which URL format you provide, Cloudglue will automatically normalize it to a meeting instance UUID for processing, and different spellings of the same recording deduplicate to one Cloudglue file. Find your meeting instance UUID by going to Zoom Web -> Recordings & Transcripts, then click on a recording to see the detail page, your video's UUID is in the browser URL query parameters `meeting_id=your-video-uuid`. You can also paste that detail-page URL directly — it's the most reliable link form. ## Browsing and filtering recordings When [listing connector files](/api-reference/endpoint/data-connectors/list-files), `from`/`to` filter by meeting start date (defaulting to a 6-month lookback when omitted) and `title_search` matches the meeting topic. Each listed item's `metadata` carries the recording's source metadata (below), so you can inspect host, timezone, and recording-file structure before importing. ## Source metadata When a Zoom recording is imported, Cloudglue captures the meeting's source metadata onto the resulting file under `source_metadata`: meeting UUID and id, topic, start time, host id/email, timezone, duration, and the structure of the meeting's recording files (types, sizes, timestamps). Signed download/play URLs, share links, and passcodes are never stored. The `source_metadata` field is returned on the [file object](/api-reference/endpoint/files/get-id). `host_email` is nullable whenever Zoom omits it — both the list endpoint and the per-meeting endpoint may return `host_email: null` depending on account scopes and whether Zoom includes the field for that recording. [`GET /data-connectors/{id}/source-metadata`](/api-reference/endpoint/data-connectors/source-metadata) previews a recording's source metadata without importing it: As everywhere on this page, the meeting instance UUID must be URL-encoded — and since it rides inside the `url` query parameter here, encode the whole parameter value: ```bash theme={null} curl --request GET \ --url 'https://api.cloudglue.dev/v1/data-connectors/YOUR_CONNECTOR_ID/source-metadata?url=zoom%3A%2F%2Fuuid%2FQFwZYEreTl2e6MBFSslXjQ%253D%253D' \ --header 'Authorization: Bearer YOUR_API_KEY' # url = encodeURIComponent(`zoom://uuid/${encodeURIComponent("QFwZYEreTl2e6MBFSslXjQ==")}`) ``` ## UI Importing for Testing Use the Cloudglue integration UI to quickly import Zoom files for testing your endpoints. This integration UI only supports importing files from the Zoom account that you connected to Cloudglue. This will not show recordings for the organization that you are a member of. If you need to use files from the organization, you can use the API to import the files. File Chooser To access this page, visit [our app](https://app.cloudglue.dev/home/data-connectors) and click on **Select Videos** on the Zoom connector. Quick Import Button ## Managing Access You maintain full control over your data access. Remove this data connector by doing any of the following: * **Revoke access** by deleting the Cloudglue app from your [Zoom account](https://marketplace.zoom.us) -> **Manage** -> **Added Apps**. * **Remove connection** by revoking the connector on our website. Once removed: * You cannot import new recordings into Cloudglue until you reconnect. * Any recordings you previously imported will remain in Cloudglue. These can be viewed and deleted from [File Management](https://app.cloudglue.dev/home/files). ## Security Cloudglue uses Zoom OAuth to access recordings with minimal permissions. This approach: * **Maintains your control** - You own and manage the connector. * **Follows Zoom security standards** - Uses OAuth with minimum permissions. # Chat Completion with Media Description Collections Source: https://docs.cloudglue.dev/deep-dives/chat-completion Build intelligent conversations with video content using Media Description Collections and grounded AI responses Video contains a wealth of conversational potential, but extracting meaningful insights through natural language queries requires sophisticated understanding of both spoken content and visual context. While basic transcription gives you raw text, chat completions with Media Description Collections allow you to have intelligent conversations with your video content. With Cloudglue's Chat Completion API and Media Description Collections, you can ask natural language questions about your videos and receive contextually accurate responses grounded in the actual video content—including speech, visual scenes, and on-screen text. Chat completions work with both locally uploaded videos and YouTube content, with multimodal understanding available for uploaded files. ## Understanding Media Description Collections Media Description Collections are specialized collections that combine multiple layers of video understanding: 1. **Speech transcription**: What's being said in the video 2. **Visual scene descriptions**: What's happening visually 3. **On-screen text**: Text and captions visible in the video 4. **Contextual understanding**: How all these elements work together Unlike basic transcription, Media Description Collections create a searchable knowledge base that enables semantic queries across all modalities of your video content. ## When to Use Chat Completions Chat completions with Media Description Collections are ideal when you need to: 1. **Answer specific questions** about video content without watching the entire video 2. **Extract insights** that span multiple segments or videos 3. **Generate summaries** with specific focus areas or perspectives 4. **Find precise information** using natural language rather than keyword search 5. **Build conversational interfaces** that can discuss video content intelligently ## Core Chat Completion Parameters ### Essential Parameters ```typescript theme={null} const response = await client.chat.completions.create({ messages: [ { role: 'user', content: 'What ingredients are needed for the pasta recipe?', }, ], model: 'nimbus-001', collections: [collection.id], force_search: true, include_citations: true, max_tokens: 1000, temperature: 0.3, }); ``` #### Messages Array The `messages` array follows the standard chat completion format: * `role`: "user", "assistant", or "system" * `content`: The message content For multi-turn conversations, include the full conversation history: ```typescript theme={null} const messages = [ { role: 'user', content: "What's the main pasta dish in the video?" }, { role: 'assistant', content: 'The main dish is a traditional Italian carbonara...', }, { role: 'user', content: 'What makes this carbonara authentic?' }, ]; ``` #### Model: nimbus-001 Cloudglue's `nimbus-001` is a specialized model optimized for: * **Multimodal understanding**: Processes speech, visual, and text content together * **Grounded responses**: Answers are based on actual video content, not training data * **Citation support**: Can provide specific timestamps and sources * **Conversational context**: Maintains context across multiple exchanges #### Force Search The `force_search` parameter controls whether the system searches your collections: * `true`: Always searches collections before responding (recommended) * `false`: May respond from general knowledge without searching ```typescript theme={null} // Always search collections first force_search: true; ``` #### Include Citations Citations provide transparency and verifiability: * `true`: Returns timestamps, file references, and content snippets * `false`: Returns only the response text ### Advanced Search Filters For precise control over what content is searched, you can use metadata filters to target specific videos in your collection: ```typescript theme={null} const response = await client.chat.completions.create({ messages: [{ role: 'user', content: 'Show me pasta preparation techniques' }], model: 'nimbus-001', collections: [collection.id], force_search: true, filter: { metadata: [ { path: 'metadata.cuisine_type', operator: 'Equal', valueText: 'Italian', }, { path: 'metadata.difficulty_level', operator: 'In', valueText: 'beginner,intermediate', }, ], }, }); ``` #### Supported Filter Operations The `filter` parameter allows you to constrain searches using file metadata: * **path**: JSON path to the metadata field (e.g., `"metadata.custom_field"` or `"video_info.has_audio"`) * **operator**: Comparison operator to apply * `Equal` / `NotEqual`: Exact match comparison * `LessThan` / `GreaterThan`: Numeric comparison * `In`: Check if value is in a comma-separated list * `ContainsAny` / `ContainsAll`: Array operations (use `valueTextArray`) * **valueText**: Single value for scalar comparisons * **valueTextArray**: Array of values for array operations **Example metadata filter scenarios:** ```typescript theme={null} // Filter by custom video categories filter: { metadata: [ { path: 'metadata.video_category', operator: 'ContainsAny', valueTextArray: ['recipe', 'technique', 'basics'], }, ]; } // Filter by video duration filter: { metadata: [ { path: 'video_info.duration_seconds', operator: 'LessThan', valueText: '600', // Videos under 10 minutes }, ]; } ``` ## Practical Example: Cooking Video Chat Bot Let's build a comprehensive example using cooking videos to demonstrate multi-turn conversations and advanced features. ### Setting Up the Collection ```typescript theme={null} import { Cloudglue } from '@cloudglue/cloudglue-js'; const client = new Cloudglue(); // Create a media description collection for cooking videos const collection = await client.collections.createCollection({ name: 'Italian Cooking Masterclass', collection_type: 'media-descriptions', description: 'Collection of Italian cooking videos for recipe chat', describe_config: { enable_summary: true, enable_speech: true, enable_scene_text: true, enable_visual_scene_description: true, }, }); // Add cooking videos to the collection const cookingVideos = [ 'https://www.youtube.com/watch?v=PASTA_VIDEO_1', 'https://www.youtube.com/watch?v=PASTA_VIDEO_2', 'https://www.youtube.com/watch?v=PASTA_VIDEO_3', ]; for (const videoUrl of cookingVideos) { const result = await client.collections.addVideoByUrl( collection.id, url: videoUrl, ); await client.collections.waitForReady(collection.id, result.file_id); console.log(`Added and processed: ${videoUrl}`); } ``` ### Multi-Turn Conversation Example Now let's demonstrate a realistic conversation about pasta recipes. This example shows how to build a stateful chatbot that maintains conversation history and can handle follow-up questions. The implementation demonstrates proper conversation state management, error handling, and how to structure questions for optimal results from the nimbus-001 model. ```typescript theme={null} import { type ChatCompletionResponse, Cloudglue, } from '@cloudglue/cloudglue-js'; import * as dotenv from 'dotenv'; // Load environment variables from .env file dotenv.config(); if (process.argv.length < 4) { console.error('Usage: ts-node chat-completion.ts '); process.exit(1); } const collectionId = process.argv[2]; const userQuery = process.argv[3]; class CookingChatBot { private client: Cloudglue; private collectionId: string; private conversationHistory: { role: 'user' | 'system' | 'assistant'; content: string; name?: string | undefined; }[] = []; constructor(client: Cloudglue, collectionId: string) { this.client = client; this.collectionId = collectionId; this.conversationHistory = []; } async askQuestion(question: string) { // Add user message to conversation history this.conversationHistory.push({ role: 'user', content: question, }); // Get response from Cloudglue const response = await this.client.chat.createCompletion({ messages: this.conversationHistory, model: 'nimbus-001', collections: [this.collectionId], force_search: true, include_citations: true, temperature: 0.4, }); const assistantMessage = response.choices?.[0]?.message; // Add assistant response to conversation history this.conversationHistory.push({ role: 'assistant', content: assistantMessage?.content || '', }); return { response: assistantMessage?.content || '', citations: response.choices?.[0]?.citations || [], }; } getConversationHistory() { return this.conversationHistory; } } // Example conversation async function runCookingChat(chatBot: CookingChatBot) { console.log('=== Question 1 ==='); const q1 = await chatBot.askQuestion( 'What cooking techniques are used for pasta dishes?', ); console.log('Response:', q1.response); console.log('Citations:', q1.citations.length, 'sources'); console.log('\n=== Question 2 ==='); const q2 = await chatBot.askQuestion( 'What ingredients are needed for the main recipe?', ); console.log('Response:', q2.response); console.log('\n=== Question 3 ==='); const q3 = await chatBot.askQuestion(` "Show me when the chef is plating the food `); console.log('Response:', q3.response); console.log('\n=== Question 4 ==='); const q4 = await chatBot.askQuestion(` Can you provide a complete recipe for one of the pasta dishes in markdown format? Include ingredients, equipment, and step-by-step instructions with approximate timing. `); console.log('Response:', q4.response); } async function main() { // Initialize the Cloudglue client const client = new Cloudglue({ apiKey: process.env.CLOUDGLUE_API_KEY, }); // Initialize the chat bot const chatBot = new CookingChatBot(client, collectionId); console.log('Chat bot initialized'); console.log('Running cooking chat example queries...'); await runCookingChat(chatBot); console.log('\n\nRunning cooking chat with user query...'); const userResponse = await chatBot.askQuestion(userQuery); console.log('User query:', userQuery); console.log('User response:', userResponse.response); console.log('Citations:', userResponse.citations.length, 'sources'); console.log('\n\nConversation TL;DR:'); const userResponse2 = await chatBot.askQuestion( 'Summarize the conversation in a few sentences. Include the user query and the response.', ); console.log('User response:', userResponse2.response); } main().catch(console.error); ``` **Key features demonstrated in this example:** * **Conversation State Management**: The chatbot maintains conversation history across multiple questions * **TypeScript Integration**: Full type safety with proper Cloudglue SDK types * **Multiple Question Types**: Shows different query patterns (techniques, ingredients, visual cues, structured output) * **Citation Handling**: Demonstrates how to access and count citation sources * **Environment Configuration**: Proper setup with environment variables and command-line arguments * **Error Handling**: Safe access to response properties with optional chaining * **Production-Ready Structure**: Modular design that can be extended for real applications ### Example Conversation Output Here's what a realistic conversation might look like: ```text [expandable] theme={null} npm run cooking-chat-bot "Yo what kinds of appliances / kitchenware is used in the video?" > cloudglue-js-examples@1.0.0 cooking-chat-bot > dotenv -e .env -- ts-node cooking-chat-bot.ts Yo what kinds of appliances / kitchenware is used in the video? Chat bot initialized Running cooking chat example queries... === Question 1 === Response: The video segments provided showcase various cooking techniques used for pasta dishes, specifically for a traditional Italian ragu. The initial steps involve preparing the meat, as seen in the segments from 00:20 to 01:00. The chef starts by cutting 500 grams of topside beef and 200 grams of sausages into smaller pieces, seasoning them with salt and pepper. This is followed by browning the meat in a pan, which is an essential step in developing the flavor of the ragu. The next steps involve cooking the onions and adding bay leaves to the pan, as shown in the segment from 00:40 to 01:00. The chef emphasizes the importance of mixing the ingredients well and breaking the bay leaves to release their flavor. The segments from 01:00 to 02:00 demonstrate the technique of sealing the meat properly and cooking the onions until they dissolve. The chef also explains the difference between sweating and burning, highlighting the importance of cooking the ingredients slowly to avoid burning. The addition of wine to the pan is another crucial step, as seen in the segment from 01:20 to 01:40. The chef advises against using cooking wine and instead recommends using a good quality wine that one would drink. The wine is added to the pan and allowed to evaporate, which helps to intensify the flavors. The segments from 02:40 to 03:40 show the addition of tomatoes and basil to the ragu. The chef dissolves the tomatoes in wine and then adds water to the pan, followed by stirring in some basil. This is an important step in creating a rich and flavorful sauce. The final steps involve slowly cooking the tomato sauce for about two hours, as shown in the segment from 03:20 to 03:40. The chef then prepares the pasta dish by adding the cooked pasta to the ragu and finishing it with a drizzle of Parmesan cheese and olive oil. Overall, the cooking techniques used for this pasta dish include browning the meat, cooking the onions, adding aromatics like bay leaves and basil, using wine to intensify the flavors, and slowly cooking the tomato sauce. These techniques come together to create a rich and flavorful ragu that is perfect for serving with pasta. Citations: 7 sources === Question 2 === Response: The video segments provided showcase the preparation of a traditional Italian ragu, and the ingredients needed for the main recipe are mentioned throughout the clips. From the segment [00:20 - 00:40], we can see that the recipe requires 500 grams of topside beef, which can be substituted with brisket. Additionally, 500 grams of spare ribs and 200 grams of sausages are used. The speaker also mentions the importance of seasoning with salt and pepper. In the segment [00:40 - 01:00], two bay leaves are added to the mixture, and the speaker emphasizes the need to break the leaves before adding them. The segment [01:00 - 01:20] shows the preparation of onions, with the speaker mentioning the use of one onion. The onions will eventually dissolve and add to the flavor of the dish. The segment [01:20 - 01:40] highlights the addition of wine to the recipe. The speaker stresses the importance of using a good-quality wine, rather than a cooking wine, which is referred to as "colored water." The segment [01:40 - 02:00] shows the meat and onions being cooked, with the speaker explaining the process of evaporating the wine and allowing the ingredients to sweat, rather than burn. The segment [02:40 - 03:00] mentions the addition of tomatoes, which are dissolved in wine and then filled with water. The segment [03:00 - 03:20] shows the addition of basil, which is stirred into the mixture, and the speaker comments on the lovely aroma. The segment [03:20 - 03:40] explains that the tomato sauce should be cooked slowly for about two hours. Finally, the segments [04:00 - 04:40] and [04:40 - 05:00] show the preparation of the pasta dish, with the addition of Parmesan cheese, olive oil, and the final tasting of the dish. In summary, the ingredients needed for the main recipe are: - 500 grams of topside beef (or brisket) - 500 grams of spare ribs - 200 grams of sausages - Salt and pepper - 2 bay leaves - 1 onion - Wine (good-quality, not cooking wine) - Tomatoes - Water - Basil - Parmesan cheese - Olive oil - Pasta (not specified in the video segments, but implied as the base of the dish) These ingredients come together to create a rich and flavorful Italian ragu, which is then served with pasta. === Question 3 === Response: The video segments provided show the preparation of a traditional Italian ragu by Chef Gennaro Contaldo. To answer your query, "Show me when the chef is plating the food," I found a relevant segment. The chef is plating the food in the segment [04:00 - 04:20] (video_id: 5d47c409-7c0d-45c4-88e3-da1e51f0e7b6, segment_id: 63d29eba-df60-4c49-a7d8-93db717decb2). In this segment, the chef says, "Look at that. As soon as I put the pasta inside, I start the dancing." Although the chef doesn't explicitly mention plating, this segment shows the chef preparing the pasta dish, which implies that the plating process is about to begin or is already underway. However, a more specific moment related to plating can be seen in the segment [04:40 - 05:00] (video_id: 5d47c409-7c0d-45c4-88e3-da1e51f0e7b6, segment_id: d821c770-3151-4247-8458-ffe9e884386b), where the chef tastes the pasta and says, "Do you want some? Bellissimo." This suggests that the dish is now ready to be served, and the chef is presenting it to someone, which can be considered as part of the plating process. While the video segments don't provide a clear shot of the chef plating the food in the classical sense, these two segments show the chef preparing and presenting the dish, which are closely related to the plating process. === Question 4 === Response: ### Family Ragu with Gennaro Contaldo Recipe #### Ingredients: * 500g topside of beef (or brisket) * 500g spare ribs * 200g sausages * Salt * Pepper * 2 bay leaves * 1 onion * Wine (use a good quality wine, not cooking wine) * Tomatoes * Water * Fresh basil * Parmesan cheese * Olive oil * Pasta (not specified in the video, but typically used with ragu) #### Equipment: * Large pot or Dutch oven * Cutting board * Knife * Spoon * Colander (for draining pasta) #### Step-by-Step Instructions: 1. **Prepare the meat** (00:20 - 00:40): Cut the beef, spare ribs, and sausages into desired sizes. Season with salt and pepper. 2. **Sear the meat** (00:40 - 01:00): Heat some oil in the pot and sear the meat until browned on all sides. 3. **Add onion and bay leaves** (01:00 - 01:20): Add the chopped onion and 2 bay leaves to the pot. Break the bay leaves to release their flavor. 4. **Add wine** (01:20 - 01:40): Pour in the wine, making sure to use a good quality wine. Let it simmer and evaporate. 5. **Add tomatoes and water** (02:40 - 03:00): Add the tomatoes and some water to the pot. Stir to combine. 6. **Let it cook** (03:20 - 03:40): Let the sauce cook slowly for about 2 hours, stirring occasionally. 7. **Prepare the pasta** (04:00 - 04:20): Cook the pasta according to the package instructions. Drain and set aside. 8. **Combine pasta and ragu** (04:20 - 04:40): Add the cooked pasta to the pot with the ragu sauce. Toss to combine. 9. **Serve** (04:40 - 05:00): Serve the pasta with ragu, topped with Parmesan cheese and a drizzle of olive oil. #### Approximate Timing: * Preparation: 30 minutes * Cooking: 2 hours * Total: 2 hours 30 minutes Note: The video segments provided do not include exact measurements for some ingredients, such as the amount of onion or tomatoes. The recipe above is based on the information provided in the video segments and may need to be adjusted according to personal preference. Running cooking chat with user query... User query: Yo what kinds of appliances / kitchenware is used in the video? User response: The video segments provided showcase the preparation of a traditional Italian ragu by Chef Gennaro Contaldo. To answer your query, "Yo what kinds of appliances / kitchenware is used in the video?", I found several relevant segments. Throughout the video, Chef Contaldo uses various kitchenware and appliances to prepare the dish. At [00:20 - 00:40], he is shown handling a large cutting board and a knife to chop the meat. He also uses a large pot to cook the ragu, which is visible in several segments, including [01:00 - 01:20] and [02:00 - 02:20]. In [01:20 - 01:40], Chef Contaldo is shown using a wooden spoon to stir the meat and onions, and in [02:40 - 03:00], he uses a can opener to open a can of tomatoes. A colander is also used to drain the pasta, as seen in [04:00 - 04:20]. Additionally, Chef Contaldo uses a stovetop to cook the ragu, which is visible throughout the video. He also uses a large serving spoon to serve the pasta, as seen in [04:20 - 04:40]. Some of the specific kitchenware and appliances used in the video include: * Large cutting board * Knife * Large pot * Wooden spoon * Can opener * Colander * Stovetop * Serving spoon These kitchenware and appliances are used to prepare and cook the traditional Italian ragu, and are essential to the dish's preparation. Citations: 4 sources Conversation TL;DR: User response: The conversation started with a query about cooking techniques used for pasta dishes. The user then asked about the ingredients needed for the main recipe, and later inquired about the appliances and kitchenware used in the video. The user also asked to summarize the conversation in a few sentences, including the user query and the response. To answer the query about cooking techniques, the video segments provided show Chef Gennaro Contaldo preparing a traditional Italian ragu. The techniques used include browning the meat, sweating the onions, and slowly cooking the tomato sauce. The chef also emphasizes the importance of using high-quality ingredients, such as good wine and fresh basil. Regarding the ingredients, the video segments mention the use of topside of beef, spare ribs, sausages, onions, carrots, celery, bay leaves, tomatoes, and basil. The chef also uses olive oil, salt, and pepper to season the dish. As for the appliances and kitchenware, the video segments show the chef using a large pot, a cutting board, a knife, and a wooden spoon. The chef also uses a stove to cook the ragu. To summarize, the conversation covered various aspects of cooking a traditional Italian ragu, including cooking techniques, ingredients, and appliances. The video segments provided a detailed and informative guide to preparing this classic dish. ``` ## Advanced Techniques ### Using System Messages for Specialized Responses You can guide the model's behavior using system messages: ```typescript theme={null} const messages = [ { role: 'system', content: `You are a professional chef's assistant specializing in Italian cuisine. Always provide specific timestamps when referencing techniques, include metric measurements, and explain the culinary science behind each method. Format recipes in clear markdown structure.`, }, { role: 'user', content: 'How should I prepare the guanciale for carbonara?', }, ]; ``` ### Search Optimization with Filters For complex queries, use metadata filters to target specific videos in your collection: ```typescript theme={null} // Search only Italian recipes with specific difficulty levels const italianRecipesResponse = await client.chat.completions.create({ messages: [{ role: 'user', content: 'Show me pasta preparation techniques' }], model: 'nimbus-001', collections: [collection.id], force_search: true, filter: { metadata: [ { path: 'metadata.cuisine_type', operator: 'Equal', valueText: 'Italian', }, { path: 'metadata.difficulty', operator: 'In', valueText: 'beginner,intermediate', }, ], }, }); // Search videos by chef or instructor const specificChefResponse = await client.chat.completions.create({ messages: [{ role: 'user', content: 'What techniques does this chef use?' }], model: 'nimbus-001', collections: [collection.id], force_search: true, filter: { metadata: [ { path: 'metadata.chef_name', operator: 'ContainsAny', valueTextArray: ['Gordon Ramsay', 'Julia Child', 'Anthony Bourdain'], }, ], }, }); ``` ## Understanding Citations When you set `include_citations: true`, the response includes detailed references to the specific video segments that informed the answer. This provides transparency and allows users to verify information or explore the original content. ### Example Citation Response Let's examine what a real citation response looks like for the question "What cooking techniques are used for pasta dishes?": ### Complete JSON Response with Citations ```json [expandable] theme={null} { "id": "13c4c53d-f1b7-48d2-a6b3-b86ced3cb283", "object": "chat.completion", "created_at": 1748039761062, "model": "nimbus-001", "usage": { "prompt_tokens": 69, "completion_tokens": 640, "total_tokens": 1060 }, "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The video segments provided showcase various cooking techniques used to prepare a traditional Italian dish, specifically a family ragu. \n\nFirstly, the chef, Gennaro Contaldo, starts by heating olive oil in a pan, as seen in the segment from 00:00 to 00:20. This is a fundamental technique in many Italian recipes, where olive oil is used as a base for sautéing ingredients.\n\nThe next segment, from 00:20 to 00:40, demonstrates the technique of cutting and preparing meat. The chef cuts 500 grams of topside beef and 200 grams of sausages into smaller pieces, seasoning them with salt and pepper. This technique is essential in ensuring that the meat cooks evenly and absorbs the flavors of the dish.\n\nIn the segment from 00:40 to 01:00, the chef mixes and cooks the meat with bay leaves. The technique of breaking the bay leaves and mixing them with the meat allows the flavors to infuse evenly. The chef also emphasizes the importance of hearing the \"beautiful noise\" of the ingredients cooking together, which suggests that he values the sensory experience of cooking.\n\nThe segment from 01:20 to 01:40 shows the technique of deglazing the pan with wine. The chef adds wine to the pan, stating that one should not use cooking wine, but rather a good-quality wine that they would drink. This technique helps to loosen the browned bits from the bottom of the pan, adding depth and richness to the sauce.\n\nIn the segment from 01:40 to 02:00, the chef cooks the meat and onions, allowing the wine to evaporate and the onions to \"sweat\" rather than burn. This technique is crucial in developing the flavor of the dish, as it prevents the onions from becoming bitter and overpowering.\n\nThe chef explains the difference between \"sweating\" and \"burning\" in the segment from 02:00 to 02:20, using the analogy of jogging and sunburn to illustrate the concept. This technique is essential in cooking, as it helps to preserve the natural flavors and textures of the ingredients.\n\nThe segment from 02:40 to 03:00 demonstrates the technique of cooking with tomatoes. The chef dissolves the tomatoes in wine and then adds water to the pan, creating a rich and flavorful sauce. This technique is a staple of Italian cooking, where tomatoes are often used to add acidity and depth to sauces.\n\nFinally, the segments from 03:00 to 03:20 and 03:20 to 03:40 show the technique of slowly cooking the tomato sauce with basil. The chef stirs the sauce and allows it to simmer, creating a rich and flavorful sauce. The technique of cooking the sauce slowly over low heat helps to develop the flavors and textures of the ingredients, resulting in a delicious and authentic Italian dish.\n\nOverall, the video segments showcase a range of cooking techniques, including sautéing, deglazing, sweating, and slow cooking. These techniques are essential in creating a rich and flavorful Italian ragu, and demonstrate the importance of attention to detail and patience in cooking." }, "citations": [ { "collection_id": "d9c40ba5-19be-4180-817b-249927616c84", "file_id": "5d47c409-7c0d-45c4-88e3-da1e51f0e7b6", "text": "I am Genera Contaldo, and this is my family revel", "segment_id": "2b44d4e4-c1e2-4c37-aae0-dd30c17c1bf5", "start_time": 0, "end_time": 20, "visual_scene_description": [], "scene_text": [], "audio_description": [], "speech": [ { "speaker": "0", "text": "I am Genera Contaldo, and this is my family revel.", "start_time": 10.16, "end_time": 14.32 }, { "speaker": "0", "text": "It's so good.", "start_time": 15.005, "end_time": 16.365 }, { "speaker": "0", "text": "Olive oil goes in.", "start_time": 17.165, "end_time": 18.605 } ] }, { "collection_id": "d9c40ba5-19be-4180-817b-249927616c84", "file_id": "5d47c409-7c0d-45c4-88e3-da1e51f0e7b6", "text": "500 gram topside of beef", "segment_id": "855594b8-145e-4ae1-a1ec-23123339f01f", "start_time": 20, "end_time": 40, "visual_scene_description": [], "audio_description": [], "scene_text": [], "speech": [ { "speaker": "0", "text": "500 gram topside of beef.", "start_time": 20.045, "end_time": 22.845 }, { "speaker": "0", "text": "You can use brisket as well.", "start_time": 23.244999, "end_time": 24.925 }, { "speaker": "0", "text": "It's fantastic.", "start_time": 24.925, "end_time": 25.965 }, { "speaker": "0", "text": "Roughly, this is the size you want.", "start_time": 26.045, "end_time": 27.805 }, { "speaker": "0", "text": "I have a 500 gram of spare ribs, 200 grams of sausages.", "start_time": 28.26, "end_time": 33.54 }, { "speaker": "0", "text": "Just cut them in half.", "start_time": 33.54, "end_time": 35.06 }, { "speaker": "0", "text": "Little salt.", "start_time": 35.06, "end_time": 36.26 }, { "speaker": "0", "text": "Pepper.", "start_time": 37.54, "end_time": 38.26 } ] } ] } ] } ``` ### Key Citation Fields Each citation provides detailed information about the source: * **collection\_id**: ID of the collection containing the video * **file\_id**: Unique identifier for the specific video file * **segment\_id**: ID of the specific segment within the video * **start\_time** / **end\_time**: Precise timestamps in seconds * **text**: Brief description of the segment's relevance * **speech**: Array of transcribed speech with speaker identification and timestamps * **visual\_scene\_description**: Visual content descriptions (when available) * **audio\_description**: Audio content descriptions (when available) * **scene\_text**: On-screen text detected in the segment ### Using Citation Data Citations enable powerful functionality in your applications: ```typescript theme={null} // Process citations to create clickable timestamps function createTimestampLinks(citations) { return citations.map((citation) => ({ fileId: citation.file_id, startTime: citation.start_time, endTime: citation.end_time, description: citation.text, url: `https://your-video-player.com/watch?v=${citation.file_id}&t=${citation.start_time}`, })); } // Extract specific quotes with timestamps function extractQuotes(citations) { const quotes = []; citations.forEach((citation) => { citation.speech?.forEach((speechItem) => { quotes.push({ text: speechItem.text, speaker: speechItem.speaker, timestamp: speechItem.start_time, fileId: citation.file_id, }); }); }); return quotes; } ``` This citation system ensures that every response can be traced back to its original source, providing transparency and enabling users to explore the full context of the information. ## Best Practices ### 1. Design Effective Conversations * **Start broad, then narrow**: Begin with overview questions, then drill into specifics * **Maintain context**: Include relevant conversation history for follow-up questions * **Use specific terminology**: Culinary terms, technique names, and ingredient specifics yield better results ### 2. Optimize Search Parameters * **Use force\_search: true** for accuracy when you need video-specific information * **Apply metadata filters** when you need to target specific subsets of your video collection * **Set appropriate temperature**: Lower (0.1-0.3) for factual responses, higher (0.5-0.7) for creative interpretations ### 3. Structure Complex Requests ```typescript theme={null} // Instead of: "Tell me about pasta" // Use this structured approach: const structuredQuery = ` Analyze the pasta preparation techniques shown in these videos: 1. What are the key steps for each recipe? 2. Which techniques are emphasized by multiple chefs? 3. What timing considerations are mentioned? Please organize your response with clear headings and include specific timestamps. `; ``` ### 4. Handle Multi-Video Collections * **Reference specific videos** when asking comparative questions * **Use metadata filters** to focus on relevant subsets of your collection * **Aggregate insights** by asking for patterns across multiple sources ### 5. Leverage Citation Information * **Verify key facts** by checking citation timestamps * **Cross-reference** information across multiple cited segments * **Guide users** to specific video moments for deeper learning ## Try it out Ready to start building conversational video experiences? Check out our [Chat Completion API](/api-reference/endpoint/chat) to get started with Media Description Collections. Experiment with different question types: * **Factual queries**: "What ingredients are used?" * **Technique analysis**: "How do the chefs differ in their approach?" * **Structured requests**: "Create a recipe in markdown format" * **Comparative questions**: "Which method is more traditional?" Get started on our [platform](https://app.cloudglue.dev) and create your first Media Description Collection today. ## Advanced Integration Patterns For production applications, consider implementing: * **Conversation persistence**: Store and resume chat sessions * **Response caching**: Cache common queries for better performance * **Citation indexing**: Build searchable citation databases for content discovery * **Multi-collection queries**: Search across different video collections simultaneously * **Response validation**: Implement confidence scoring based on citation quality # Description Guide Source: https://docs.cloudglue.dev/deep-dives/description-guide Deep dive into how to describe and analyze videos using our Describe API Video is a treasure trove of information, but developers need a way to get that information out. In some cases, developers may have used speech-to-text APIs to get the speech from a video. However, this doesn't give you the full picture. With Cloudglue, you can describe a video and get the speech **alongside other information like on-screen text and visual scene descriptions**. This allows you to build more powerful applications that can operate on more information from the video, and in a simple and straightforward manner. Cloudglue allows you to either describe locally uploaded files, or videos from YouTube. ## The Basics ### Describe Config The **describe config** is a JSON object that contains the configuration for the description. It dictates what the description will include from the video/audio file. By default, the description will include the speech, and summary of the video/audio file. However, you can also include visual scene descriptions, and on screen text or captions from the video/audio file. #### Example Config Here's an example of the config you can use to describe a video/audio file. ```json theme={null} { // Defaults to true "enable_speech": true, // Defaults to false "enable_scene_text": false, // Defaults to false "enable_visual_scene_description": false, // Defaults to true "enable_summary": true, // Optional free-form guidance — see below "prompt": "Spell product names as Cloudglue and Aviary; call out any pricing discussed." } ``` By altering the config, you can explicitly control what gets generated from the video/audio file. #### Guiding the description with `prompt` The `enable_*` flags decide *what kinds* of description you get. The optional `prompt` decides *what the description pays attention to*: the domain terms, product names, and acronyms you want spelled correctly, the people who tend to appear, or the details you want called out. It applies to every pass — visual, scene text, speech, and the title/summary — and works the same on an on-demand describe and on a Media Description collection's `describe_config`, where it applies to every file in the collection. Two limits worth knowing: * A prompt is context, not a licence to invent. Naming a term or a person does not make the description report them if they aren't in the media. * A prompt does not constrain speaker labels. To restrict transcript speaker names to a known cast, pass `participants` instead (see [On-Demand Describe](/core-concepts/describe#speaker-identification)). The prompt is part of the describe config, so it's part of the cache key too: the same file described under different guidance produces a new job rather than reusing the earlier one. Maximum 2000 characters. ### Generating a Description #### Example All you need is a few lines of code to generate a description with Cloudglue. ```typescript theme={null} // File uri comes from our files API, in the form of cloudglue://file_id const description = await client.describe.createDescribe(fileUri, { enable_speech: true, enable_scene_text: true, enable_visual_scene_description: true, enable_summary: true, }); // ... // Or if you want to use a youtube video const description = await client.describe.createDescribe('https://www.youtube.com/watch?v=AJpK3YTTKZ4', { enable_speech: true, enable_scene_text: true, enable_visual_scene_description: true, enable_summary: true, }); ``` ```python theme={null} # File uri comes from our files API, in the form of cloudglue://file_id description = client.describe.run( url=file_uri, enable_speech=True, enable_scene_text=True, enable_visual_scene_description=True, enable_summary=True ) # ... # Or if you want to use a youtube video description = client.describe.run( url='https://www.youtube.com/watch?v=AJpK3YTTKZ4', enable_speech=True, enable_scene_text=True, enable_visual_scene_description=True, enable_summary=True ) ``` ## Description Outputs Getting the description is as simple as making one call to get the description. ### Getting the Description ### Examples Here's an example of a truncated output you get when you describe a video/audio file. #### JSON Example ```json JSON Output [expandable] theme={null} { "data": { "title": "Maryland Public Service Commission Hearing on Chaberton Energy's Wild Turkey Community Solar Project", "summary": "This video documents a Maryland Public Service Commission hearing held on May 22, 2024, regarding the application for a certificate of public convenience and necessity for the Wild Turkey community solar project by Chaberton Energy. The hearing, presided over by Chief Public Utility Law Judge Chuck McLean, includes introductions of involved attorneys and state agency representatives, followed by a detailed presentation from Chaberton Energy representatives about the project's benefits, community engagement efforts, environmental considerations, and technical design. The project aims to provide renewable energy benefits, local tax revenue, and subscriber savings while ensuring environmental protection and compliance with state regulations. The session concludes with procedural updates and plans for future public hearings.", "speech": [ { "text": "Alright.", "start_time": 2.8799999, "end_time": 3.6 }, { "text": "Good morning, everybody.", "start_time": 3.9199998, "end_time": 5.04 }, { "text": "Everybody hear me okay?", "start_time": 5.04, "end_time": 6.3999996 }, { "text": "Yes,", "start_time": 6.3999996, "end_time": 6.8 }, { "text": "your honor.", "start_time": 8.4, "end_time": 8.96 }, { "text": "Excellent.", "start_time": 8.96, "end_time": 9.679999 }, { "text": "Alright.", "start_time": 9.679999, "end_time": 10.16 }, { "text": "Doug, we're", "start_time": 10.16, "end_time": 10.719999 }, { "text": "ready to go, sir?", "start_time": 10.719999, "end_time": 11.36 }, { "text": "Yes.", "start_time": 12.095, "end_time": 12.575001 }, { "text": "Can you hear me okay, your honor?", "start_time": 12.575001, "end_time": 14.175 }, { "text": "Yes, sir.", "start_time": 14.175, "end_time": 14.895 }, { "text": "Let's go on the record, sir.", "start_time": 14.895, "end_time": 16.335 }, { "text": "We are on the record.", "start_time": 17.535, "end_time": 18.815 }, { "text": "Thank you very much.", "start_time": 18.815, "end_time": 19.695 }, { "text": "Good evening, everyone.", "start_time": 19.695, "end_time": 20.575 }, { "text": "Good evening, everyone.", "start_time": 19.695, "end_time": 20.575 }, { "text": "Today is 05/22/2024, and it's approximately 7PM.", "start_time": 20.575, "end_time": 25.055 }, { "text": "My name is Chuck McLean.", "start_time": 25.055, "end_time": 26.095001 }, { "text": "I'm the chief public utility law judge at the Maryland Public Service Commission, and I am filling in for judge Christine Burke, uh, for the, uh, just this hearing, so you're not gonna be stuck with me at least for the duration of this particular case.", "start_time": 26.095001, "end_time": 38.25 }, { "text": "Now no one, uh, signed up to speak this evening.", "start_time": 69.365005, "end_time": 71.925 }, { "text": "You may also file written comments by sending them directly to the commission.", "start_time": 95.759995, "end_time": 100.245 }, { "text": "Uh, attention, Jamie Bergen, chief clerk, six Saint Paul Street, sixteenth floor, Baltimore, Maryland two one two zero two.", "start_time": 100.565, "end_time": 109.365 }, { "text": "Uh, and please reference case number 9717 so the case, uh, so the comments make it to the correct file.", "start_time": 109.685, "end_time": 115.6 }, ... ], "audio_description": [], "visual_scene_description": [ { "text": "The video begins with a black screen displaying the name \"Doug\".\nThe scene transitions to a man identified as Law Judge. He is wearing a suit jacket. Behind him are framed pictures on a white wall.\nThe scene changes to a man identified as Andy Flavin. He is wearing a suit and tie and glasses. Behind him is a wooden cabinet, a picture, and a telephone.\nThe scene returns to the man identified as Law Judge. He looks down.\nThe scene transitions back to the black screen with the name \"Doug\".\nThe scene returns to the man identified as Law Judge.", "start_time": 0, "end_time": 20 }, { "text": "A man with short graying hair, wearing a light brown suit jacket and a blue shirt, speaks directly to the camera. He is centered in the frame, with a white wall behind him. On the wall are two framed pictures and a display of small items. The lighting is neutral, and the camera remains static throughout the scene.", "start_time": 20, "end_time": 40 }, { "text": "A man in a suit and blue shirt speaks directly to the camera. He is in front of a white wall with framed pictures. The camera angle is a medium shot, focusing on the man's face and upper body.", "start_time": 40, "end_time": 60 }, { "text": "A man with short gray hair and a tan suit speaks directly to the camera. He is indoors with a white wall behind him with two framed pictures.", "start_time": 60, "end_time": 80 }, { "text": "The video shows a man with short, graying hair wearing a tan suit jacket. He is speaking directly to the camera. Behind him is a white wall with two framed pictures.", "start_time": 80, "end_time": 100 }, { "text": "A man with short graying hair wearing a suit jacket and blue shirt is speaking directly to the camera. Behind him is a white wall with two framed pictures. The lighting is bright and the camera angle is a medium close-up.", "start_time": 100, "end_time": 120 }, { "text": "A man with short, graying hair and a light complexion is speaking directly to the camera. He is wearing a tan suit jacket and a light blue shirt. The background is a plain white wall with three framed pictures hanging on it. The lighting is neutral, and the camera angle is a medium close-up, focusing on his face and upper body.", "start_time": 120, "end_time": 140 }, ... ], "scene_text": [ { "text": "Doug\nLaw Judge\nAndy Flavin\nLaw Judge\nDoug\nLaw Judge", "start_time": 0, "end_time": 20 }, { "text": "Law Judge", "start_time": 20, "end_time": 39 }, { "text": "@Chaberton\nAmee Beame - Chab...\nchaberton\nENERGY\nPROJECT\nWhat is\nCommunity\nSolar?\nBENEFITS\nThank You\n7:04 PM\n5/22/2024", "start_time": 273, "end_time": 279 }, { "text": "Law Judge", "start_time": 274, "end_time": 276 }, { "text": "chaberton\nENERGY\nWhat is\nCommunity\nSolar?\nBENEFITS", "start_time": 280, "end_time": 282 }, { "text": "PROPOSED DIVISION\nAREA (APPROX. 25.45\nACRES) (10.30 HECTARES)", "start_time": 420, "end_time": 421 }, { "text": "PROPOSED DIVISION\nAREA (APPROX. 25.45\nACRES) (10.30 HECTARES)\nCLEAR EXISTING\nOR IRRIGATION\nLINE (TYP.)\nPROPOSED SECURITY\nFENCE (TYP.)\nMODULAR GROUND\nMOUNTED SOLAR ARRAY\n(APPROX. 18.15 ACRES)\n(7.35 HECTARES) (TYP.)\nACCESS WITHOUT\nIMPACT TO\nTHE MAIN\nCHANNEL (TYP.)\nEXISTING CULVERT\nTO BE UPGRADED\nPROPOSED 12' WIDE\nACCESS DRIVE WITH TURNING\nRADIUS TO BE UPGRADED\n(TYP.)\nPROPOSED\nDOUBLE GATE\nUTILITY POINT OF\nINTERCONNECTION\nUTILITY METER\nAND MAIN\nSWITCH\nCLEARANCE\n1.5-2METER\nCLEARANCE WITHIN\nACCESS DRIVEWAY\nAND UTILITY ACCESSIBLE\nEASEMENT AREA\nAND DISCONNECT POLE\nPROPOSED\nELECTRIC LINE (TYP.)\nPROPOSED LANDSCAPE\nSCREENING (TYP.)\nPROPOSED SOLAR\nLANDSCAPE\nSCREENING (ASO\n(ASO GRASSING)\nTRANSFORMERS, INVERTERS,\nAND COMPONENTS REQUIRED\nTO CONVERT POWER (TYP.)\nPROPOSED\nPAD FOR\nTRANSFORMER\n(TYP.)\nPROPOSED SOLAR\nLANDSCAPE\nSCREENING (TYP.)\nNOTE:\nBRIGHT SURFACES,\nINCLUDING SOLAR PANELS\nTO BE LOCATED 100M\nFROM FENCE\n@chobertson\nArnie Beame - Chab...", "start_time": 420, "end_time": 439 }, { "text": "@cRobertson", "start_time": 540, "end_time": 560 }, { "text": "If there are any other questions or concerns please contact us\nAmee Bearne, Community Impact Manager\nAmee.Bearne@Chaberton.com\nNatalie Castro, Development Manager\nNatalie.Castro@Chaberton.com\nThank You", "start_time": 592, "end_time": 599 }, { "text": "If there are any other questions or concerns please contact us\nAmee Bearne, Community Impact Manager\nAmee.Bearne@Chaberton.com\nNatalie Castro, Development Manager\nNatalie.Castro@Chaberton.com\nThank You", "start_time": 600, "end_time": 607 }, { "text": "Law Judge", "start_time": 607, "end_time": 608 }, { "text": "Andy Flavin", "start_time": 612, "end_time": 613 }, { "text": "Law Judge", "start_time": 616, "end_time": 617 }, { "text": "Bob Sadzinski", "start_time": 618, "end_time": 619 }, { "text": "Bob Sadzinski", "start_time": 620, "end_time": 621 }, ... ] }, "describe_config": { "enable_speech": true, "enable_visual_scene_description": true, "enable_scene_text": true, "enable_summary": true } } ``` Let's look at the the different descriptions available from the JSON output. * `title`: The generated title of the video based on the descriptions generated. * `summary`: A generated summary of the video based on the descriptions generated. * `speech`: The speech transcription of the video. * `visual_scene_description`: A description of the scene at different timestamps in the video. * `audio_description`: A description of the audio at different timestamps in the video. * `scene_text`: The on-screen text at different timestamps in the video. We also support markdown outputs, ideal for using in LLMs. The document is titled after the media type it describes — `# Video Document`, `# Audio Document` or `# Image Document` — and opens with a `## File` section so a model reading it can tell which file it is looking at and cite it back by URI. Pass `include_metadata=true` to add a `## Metadata` section carrying the file's own metadata: `### User` is the free-form JSON you attached at upload (or via `PUT /files/{file_id}`), and `### Source` is the meeting provenance captured from the upstream connector for Grain, Zoom, Gong and Recall files. Either subsection is omitted when it has nothing to say, and the whole section is absent without the flag — so a prompt only pays for the metadata when it needs it. The same flag populates `file.metadata` and `file.source_metadata` on the JSON responses. #### Markdown Example The following is a truncated example of the markdown output, as returned with `response_format=markdown&include_metadata=true`. ```markdown Markdown Output [expandable] theme={null} # Video Document ## File - **Filename**: wild-turkey-hearing.mp4 - **URI**: cloudglue://files/8f2c1e4a-1a2b-4c3d-9e8f-7a6b5c4d3e2f - **Source**: upload - **Added**: 2024-05-23T14:02:11.000Z - **Duration**: 01:47:32 - **Resolution**: 1920x1080 ## Metadata ### User - **docket**: 9704 - **agency**: Maryland Public Service Commission ## Title Maryland Public Service Commission Hearing on Chaberton Energy's Wild Turkey Community Solar Project ## Summary This video documents a Maryland Public Service Commission hearing held on May 22, 2024, regarding the application for a certificate of public convenience and necessity for the Wild Turkey community solar project by Chaberton Energy. The hearing, presided over by Chief Public Utility Law Judge Chuck McLean, includes introductions of involved attorneys and state agency representatives, followed by a detailed presentation from Chaberton Energy representatives about the project's benefits, community engagement efforts, environmental considerations, and technical design. The project aims to provide renewable energy benefits, local tax revenue, and subscriber savings while ensuring environmental protection and compliance with state regulations. The session concludes with procedural updates and plans for future public hearings. --- ## Scenes ### Scene [00:00 - 00:20] #### Video Conference Call A video conference call between multiple participants. **Visual Content:** - [00:00 - 00:20] - The video begins with a black screen displaying the name "Doug". - The scene transitions to a man identified as Law Judge. - He is wearing a suit jacket. - Behind him are framed pictures on a white wall. - The scene changes to a man identified as Andy Flavin. - He is wearing a suit and tie and glasses. - Behind him is a wooden cabinet, a picture, and a telephone. - The scene returns to the man identified as Law Judge. - He looks down. - The scene transitions back to the black screen with the name "Doug". - The scene returns to the man identified as Law Judge. **Speech:** - [00:02 - 00:03] Alright. - [00:03 - 00:05] Good morning, everybody. - [00:05 - 00:06] Everybody hear me okay? - [00:06 - 00:06] Yes, - [00:08 - 00:08] your honor. - [00:08 - 00:09] Excellent. - [00:09 - 00:10] Alright. - [00:10 - 00:10] Doug, we're - [00:10 - 00:11] ready to go, sir? - [00:12 - 00:12] Yes. - [00:12 - 00:14] Can you hear me okay, your honor? - [00:14 - 00:14] Yes, sir. - [00:14 - 00:16] Let's go on the record, sir. - [00:17 - 00:18] We are on the record. - [00:18 - 00:19] Thank you very much. - [00:19 - 00:20] Good evening, everyone. **On-screen Text:** - [00:00 - 00:20] "Doug" | "Law Judge" | "Andy Flavin" --- ### Scene [00:20 - 00:40] #### Maryland Public Service Commission Hearing Chuck McClean, the Chief Public Utility Law Judge at the Maryland Public Service Commission, fills in for Judge Christine Burke for a hearing on May 22, 2024. **Visual Content:** - [00:20 - 00:40] - A man with short graying hair, wearing a light brown suit jacket and a blue shirt, speaks directly to the camera. - He is centered in the frame, with a white wall behind him. - On the wall are two framed pictures and a display of small items. - The lighting is neutral, and the camera remains static throughout the scene. **Speech:** - [00:19 - 00:20] Good evening, everyone. - [00:20 - 00:25] Today is 05/22/2024, and it's approximately 7PM. - [00:25 - 00:26] My name is Chuck McLean. - [00:26 - 00:38] I'm the chief public utility law judge at the Maryland Public Service Commission, and I am filling in for judge Christine Burke, uh, for the, uh, just this hearing, so you're not gonna be stuck with me at least for the duration of this particular case. **On-screen Text:** - [00:20 - 00:39] "Law Judge" --- ``` Looking for other categories of information from the video? Learn more about our [extraction features](/deep-dives/extraction-guide) and what they can do for you. ## Key Features * **Speech transcription**: Speech to text transcription of video/audio files. * **Scene text descriptions**: On screen text or captions from a video/audio file at different timestamps. * **Visual scene descriptions**: Get descriptions of different scenes in a video, at different timestamps. * **Title + Summary**: Get a generated title and summary of the video/audio file based on all the descriptions we have available. * **Markdown compatible**: Our descriptions are also able to generated with markdown, so you can use them in LLMs right away. ## Try it out Check out our [Describe Video Endpoint](/api-reference/endpoint/describe/post) to get started with building your own video/audio processing with Cloudglue. Get started on our [platform](https://app.cloudglue.dev). ## YouTube At the moment, if you want to describe a video directly from YouTube, we only support generating **speech** transcriptions. If you would like to get the full spectrum of descriptions for a YouTube video, you'll need to download the video and upload it to Cloudglue. # Extraction Guide Source: https://docs.cloudglue.dev/deep-dives/extraction-guide A comprehensive guide to extracting data with Cloudglue Video contains a wealth of information, but developers often need this information in a structured format that their applications can easily consume. While transcription gives you raw information from a video, entity extraction allows you to get precisely the structured data you need. With Cloudglue's Extract API, you can define exactly what information you want to extract and receive it in a format that's ready for your database or application logic. Cloudglue allows you to extract entities from both locally uploaded files and YouTube videos. ## Understanding Entities Entities are structured pieces of information that can be extracted from videos. They come in two extraction modes: 1. **Video-level entities**: Information that applies to the entire video as a whole 2. **Segment-level entities**: Information that appears at specific moments in the video (default) These modes are mutually exclusive - each extraction job can only use one mode. Segment-level extraction is enabled by default. Examples below show video-level and segment-level outputs from separate extraction jobs. For example, if you're analyzing product review videos, you might want to extract: * Video-level: Overall product rating, reviewer name, product category * Segments: Individual features discussed, pros/cons mentioned at different timestamps ## When to Use Entity Extraction While transcription gives you comprehensive raw data from a video, entity extraction is better when you: 1. Need specific structured data rather than raw transcripts 2. Want to populate a database with consistent fields 3. Need to track information across time segments 4. Want type-safe data that's ready for your application ## Defining Extraction Schemas ### Schema Structure An entity schema defines what information you want to extract. You can specify it in two ways: #### 1. Abbreviated Form (Recommended for simple schemas) ```json theme={null} { "people": [ { "name": "string", "description": "string", "gender": "string", "age_group": "string" } ], "vehicles": [ { "make": "string", "model": "string", "color": "string" } ] } ``` #### 2. Full JSON Schema Specification ```json theme={null} { "type": "object", "properties": { "people": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "description": { "type": "string" }, "gender": { "type": "string" }, "age_group": { "type": "string" } } } }, "vehicles": { "type": "array", "items": { "type": "object", "properties": { "make": { "type": "string" }, "model": { "type": "string" }, "color": { "type": "string" } } } } } } ``` ### Using the Schema Builder We provide a graphical tool to help you build and test your schemas: * [Schema Builder Tool](https://app.cloudglue.dev/tools/extract-schema-helper) * [Extract Playground](https://app.cloudglue.dev/home/playground?mode=extract) ### Example: Political Speech Analysis For this example, we'll analyze a political speech about tariffs. While the video below shows the source content, our analysis was performed using a local copy to enable full multimodal understanding: