> ## Documentation Index > Fetch the complete documentation index at: https://docs.cloudglue.dev/llms.txt > Use this file to discover all available pages before exploring further. # Describe Video > Get a comprehensive multimodal description of a video ## OpenAPI ````yaml POST /describe openapi: 3.0.0 info: title: Cloudglue API description: API for Cloudglue license: name: Apache License 2.0 url: https://www.apache.org/licenses/LICENSE-2.0 version: 0.7.10 servers: - url: https://api.cloudglue.dev/v1 security: - bearerAuth: [] paths: /describe: post: tags: - Describe summary: Create a new media description job description: Get a comprehensive multimodal description of a video operationId: createDescribe requestBody: description: Media description job parameters content: application/json: schema: $ref: '#/components/schemas/NewDescribe' required: true responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/Describe' '400': description: Invalid request or missing required url/file_id content: application/json: schema: $ref: '#/components/schemas/Error' '404': description: File not found content: application/json: schema: $ref: '#/components/schemas/Error' '429': description: Chat completion limits reached (monthly or daily) content: application/json: schema: $ref: '#/components/schemas/Error' '509': description: Monthly description jobs limit reached content: application/json: schema: $ref: '#/components/schemas/Error' components: schemas: NewDescribe: allOf: - type: object required: - url properties: url: description: >- Input media URL. Supports URIs of files uploaded to Cloudglue Files endpoint, public YouTube video URLs, public TikTok video URLs, public Loom share URLs, public HTTP URLs (including direct image URLs such as JPEG/PNG/WebP), and files which have been granted access to Cloudglue via data connectors. Note that YouTube videos are currently limited to speech level understanding only. Public TikTok video URLs will be scraped and stored automatically — subject to charges. For files via our Data connectors, see our documentation on data connectors for setup information. type: string enable_summary: type: boolean description: >- Whether to generate video-level and segment-level (moment-level) summaries and titles enable_speech: type: boolean description: Whether to generate speech transcript enable_visual_scene_description: type: boolean description: Whether to generate visual scene description enable_scene_text: type: boolean description: Whether to generate scene text extraction enable_audio_description: type: boolean description: Whether to generate audio description thumbnails_config: $ref: '#/components/schemas/ThumbnailsConfig' include_chapters: type: boolean description: >- Include narrative chapters in the response when segmentation strategy is 'narrative'. Only affects cached completed results; newly created pending jobs do not include chapters. default: false include_shots: type: boolean description: >- Include shot boundaries in the response when segmentation strategy is 'shot-detector'. Only affects cached completed results; newly created pending jobs do not include shots. default: false use_in_default_index: type: boolean description: >- When true, the video's search documents will be added to the account's default index, making them searchable via the Response API and Deep Search API with `source: 'default'`. participants: type: array maxItems: 50 description: >- Known participants on the recording. When provided, speaker naming is constrained to these people: transcript speaker labels will only use one of these names (or a generic "Speaker N"), never an invented name. Intended for uploaded files, which (unlike data-connector files such as Grain) carry no participant metadata; for connector files this is populated automatically and need not be supplied. items: type: object required: - name properties: name: type: string description: Participant display name, e.g. "Alice Smith". scope: type: string description: >- Optional free-form context for the participant, e.g. a role, affiliation, or "internal"/"external". Used only as naming context. - $ref: '#/components/schemas/FileSegmentationConfig' Describe: required: - job_id - status type: object properties: job_id: type: string status: type: string enum: - pending - processing - completed - failed - not_applicable url: type: string description: The URL of the processed video duration_seconds: type: number description: Duration of the video in seconds thumbnail_url: type: string format: uri description: >- URL of the file-level thumbnail for the video. Only present when include_thumbnails=true. created_at: type: integer description: Unix timestamp in milliseconds when the job was created describe_config: type: object description: Configuration for media description from videos properties: enable_summary: type: boolean description: >- Whether the user requested to generate video-level and segment-level (moment-level) summaries and titles enable_speech: type: boolean description: Whether the user requested to generate speech transcript enable_visual_scene_description: type: boolean description: Whether the user requested to generate visual scene description enable_scene_text: type: boolean description: Whether the user requested to generate scene text enable_audio_description: type: boolean description: Whether the user requested to generate audio description participants: type: array maxItems: 50 description: >- Known participants on the recording. When provided, speaker naming is constrained to these people: transcript speaker labels will only use one of these names (or a generic "Speaker N"), never an invented name. Intended for uploaded files, which (unlike data-connector files such as Grain) carry no participant metadata; for connector files this is populated automatically and need not be supplied. items: type: object required: - name properties: name: type: string description: Participant display name, e.g. "Alice Smith". scope: type: string description: >- Optional free-form context for the participant, e.g. a role, affiliation, or "internal"/"external". Used only as naming context. use_in_default_index: type: boolean description: >- Whether this describe job's search documents are included in the default index. data: allOf: - type: object properties: content: type: string description: >- Content string returned based on formatting, e.g. set to markdown text when response_format=markdown is requested title: type: string description: >- Generated title of the video; for YouTube videos, this is the title of the video as it appears on YouTube summary: type: string description: >- Generated video level summary; for YouTube videos, this is the summary of the video as it appears on YouTube segment_summary: type: array description: >- Array of summary information for each segment of the video. Only available when enable_summary is set to true in the describe configuration. items: type: object properties: title: type: string description: Generated segment-level title summary: type: string description: Generated segment-level summary start_time: type: number description: Start time of segment in seconds end_time: type: number description: End time of segment in seconds thumbnail_url: type: string format: uri description: URL of the segment thumbnail, when available. - $ref: '#/components/schemas/DescribeOutput' error: type: string description: Error message if status is 'failed' segmentation_id: type: string format: uuid description: >- The ID of the segmentation job if the describe was run with a segmentation chapters: type: array description: >- Array of narrative chapters (only present when include_chapters=true and segmentation strategy is 'narrative') items: type: object properties: index: type: integer minimum: 0 start_time: type: number minimum: 0 end_time: type: number minimum: 0 description: type: string required: - index - start_time - end_time - description shots: type: array description: >- Array of shot boundaries (only present when include_shots=true and segmentation strategy is 'shot-detector') items: type: object properties: index: type: integer minimum: 0 start_time: type: number minimum: 0 end_time: type: number minimum: 0 required: - index - start_time - end_time total_chapters: type: integer minimum: 0 description: >- Total number of chapters (only present when include_chapters=true and segmentation strategy is 'narrative') total_shots: type: integer minimum: 0 description: >- Total number of shots (only present when include_shots=true and segmentation strategy is 'shot-detector') Error: required: - error type: object properties: error: type: string ThumbnailsConfig: type: object required: - enable_segment_thumbnails properties: enable_segment_thumbnails: type: boolean description: >- Whether to enable segment thumbnails. If not provided will use default to false. Cannot be used together with segmentation_id. FileSegmentationConfig: type: object properties: segmentation_id: type: string format: uuid description: >- Segmentation job id to use. If not provided will use default to uniform 20s segmentation. Cannot be provided together with segmentation_config. segmentation_config: $ref: '#/components/schemas/SegmentationConfig' description: >- Configuration for video segmentation. Cannot be provided together with segmentation_id. DescribeOutput: type: object properties: visual_scene_description: type: array description: Array of visual descriptions items: $ref: '#/components/schemas/DescribeOutputPart' scene_text: type: array description: Array of scene text extractions items: $ref: '#/components/schemas/DescribeOutputPart' speech: type: array description: Array of speech transcriptions items: $ref: '#/components/schemas/SpeechOutputPart' audio_description: type: array description: Array of audio descriptions items: $ref: '#/components/schemas/DescribeOutputPart' SegmentationConfig: type: object description: >- Configuration for video segmentation. **Choose a strategy and provide ONLY the corresponding config:** • **uniform**: Provide `uniform_config`, do NOT provide other configs • **shot-detector**: Provide `shot_detector_config`, do NOT provide other configs • **manual**: Provide `manual_config`, do NOT provide other configs • **narrative**: Provide `narrative_config`, do NOT provide other configs Optionally specify `start_time_seconds` and `end_time_seconds` to limit segmentation to a portion of the video. required: - strategy properties: strategy: type: string enum: - uniform - shot-detector - manual - narrative description: Segmentation strategy - determines which config you must provide uniform_config: $ref: '#/components/schemas/SegmentationUniformConfig' description: >- 🎯 **REQUIRED when strategy = 'uniform'** - Configuration for uniform segmentation. shot_detector_config: $ref: '#/components/schemas/SegmentationShotDetectorConfig' description: >- 🎯 **REQUIRED when strategy = 'shot-detector'** - Configuration for shot detection segmentation. manual_config: $ref: '#/components/schemas/SegmentationManualConfig' description: >- 🎯 **REQUIRED when strategy = 'manual'** - Configuration for manual segmentation. narrative_config: $ref: '#/components/schemas/NarrativeConfig' description: >- 🎯 **REQUIRED when strategy = 'narrative'** - Configuration for narrative-based chapter segmentation using AI analysis. keyframe_config: $ref: '#/components/schemas/KeyframeConfig' description: >- When provided, will be used to extract keyframes for the segmentation. start_time_seconds: type: number minimum: 0 description: >- Optional: The start time of the video in seconds to start segmenting from end_time_seconds: type: number minimum: 0 description: 'Optional: The end time of the video in seconds to stop segmenting at' DescribeOutputPart: type: object properties: text: type: string description: Text content start_time: type: number description: Start time in seconds end_time: type: number description: End time in seconds SpeechOutputPart: type: object properties: speaker: type: string description: Identified speaker text: type: string description: Transcribed speech text start_time: type: number description: Start time of speech in seconds end_time: type: number description: End time of speech in seconds words: type: array description: >- Word-level timestamps. Only present when include_word_timestamps=true. Not available for YouTube sources. items: $ref: '#/components/schemas/WordTimestamp' SegmentationUniformConfig: type: object required: - window_seconds properties: window_seconds: type: number minimum: 1 maximum: 120 description: >- The duration of each segment in seconds. Must be between 1 and 120 seconds. hop_seconds: type: number minimum: 1 maximum: 120 description: >- The offset between the start of new windows. This means there can be overlap between segments. If not provided, defaults to window_seconds. Must be between 1 and 120 seconds. SegmentationShotDetectorConfig: type: object required: - detector properties: threshold: type: number nullable: true description: >- Detection sensitivity threshold - lower values create more segments: • **content**: Sensitivity to visual differences (default: 27.0). Lower values detect smaller changes in color/lighting. • **adaptive**: Does not support threshold parameter - uses internal adaptive algorithm. min_seconds: type: number minimum: 0.6 maximum: 600 nullable: true description: The minimum length of a shot in seconds. Defaults to 0.6. max_seconds: type: number minimum: 1 maximum: 600 nullable: true description: The maximum length of a shot in seconds. Defaults to 300. detector: type: string enum: - adaptive - content description: >- The detector strategy to use: • **adaptive**: Designed for dynamic footage with camera movement, panning, or action. Examples: sports broadcasts, drone footage, handheld documentaries, action movies, live events. • **content**: Optimized for controlled footage with clear visual transitions. Examples: studio interviews, corporate videos, educational content, product demos, scripted content. fill_gaps: type: boolean description: >- When true, gaps between detected shots are filled: gaps greater than or equal to min_seconds become their own segments (split by max_seconds if needed), and shorter gaps are merged into the nearest adjacent segment. Defaults to true. SegmentationManualConfig: type: object required: - segments properties: segments: type: array items: type: object properties: start_time: type: number description: The start time of the segment in seconds end_time: type: number description: The end time of the segment in seconds description: Array of segments NarrativeConfig: type: object properties: prompt: type: string description: >- Optional custom prompt to guide the narrative segmentation analysis. This will be incorporated into the main segmentation prompt as additional guidance. strategy: type: string enum: - comprehensive - balanced - transcript description: >- Narrative segmentation strategy: • **comprehensive**: Uses a VLM to deeply analyze logical segments of video. Only available for video files (not YouTube or audio). • **balanced** (default): Balanced analysis approach using multiple modalities. Supports YouTube URLs and audio files. • **transcript**: Cheap and fast speech-transcript-based segmentation. Requires a transcript; returns an error for silent or visual-only content (use `balanced` instead, or `comprehensive` for non-YouTube/non-audio video files). **Note**: YouTube URLs and audio files only support the **balanced** and **transcript** strategies; **comprehensive** will be rejected with an error. number_of_chapters: type: integer minimum: 1 description: >- Optional target number of chapters to generate. If provided, min_chapters and max_chapters will be calculated automatically if not specified. If only target is provided, the AI will attempt to generate exactly this number of chapters. min_chapters: type: integer minimum: 1 description: >- Optional minimum number of chapters to generate. If provided along with number_of_chapters and max_chapters, validates that min <= number_of_chapters <= max. If only number_of_chapters is provided, min and max are calculated automatically. max_chapters: type: integer minimum: 1 description: >- Optional maximum number of chapters to generate. If provided along with number_of_chapters and min_chapters, validates that min <= number_of_chapters <= max. If only number_of_chapters is provided, min and max are calculated automatically. KeyframeConfig: type: object required: - frames_per_segment description: >- Configuration for keyframe extraction. When provided, will be used to extract keyframes for the segmentation. This is not supported for YouTube videos. properties: frames_per_segment: type: number minimum: 0 maximum: 8 description: The number of key frames to extract per segment max_width: type: number minimum: 144 maximum: 4320 description: The maximum width of the key frames in pixels WordTimestamp: type: object description: A single word with its timing information properties: word: type: string description: The word text start_time: type: number description: Start time of the word in seconds end_time: type: number description: End time of the word in seconds required: - word - start_time - end_time securitySchemes: bearerAuth: type: http scheme: bearer ````