Bulk Imports

A bulk import lists a connected data source and brings everything matching your filters into a collection — instead of calling the API once per file. It is a saved definition you can re-run, so “keep this collection in sync with that Zoom account” is a durable thing rather than a script you have to babysit.

Two kinds of import

What an import ingests follows the collection you create it on. You never choose this directly — it is reported back as import_type.

Collection type	`import_type`	What a run does	Cost
`metadata`	`metadata`	Indexes whatever metadata the source exposes. No media is downloaded.	Free
`rich-transcripts`, `media-descriptions`, `entities`, `face-analysis`	`media`	Ingests and processes the actual files, exactly like adding each one by hand.	Billed per file

Which fields you get depends entirely on the connector — a metadata import stores what that source happens to publish about each file, and sources differ a lot. A Zoom recording carries things like a topic, host, start time and duration; a Gong call adds parties and its own call analysis; Grain contributes AI summaries and highlights; a Dropbox or Drive file may be little more than a name, size and timestamps; iconik brings whatever custom metadata your workspace has configured. Anything present is indexed and becomes searchable and filterable.

The two compose into the workflow most teams want: bulk-index a whole source into a metadata collection for free, search it to find the files actually worth processing, then run a media import with narrower filters against a processing collection to ingest only those.

Create an import

curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports" \
  -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "All sales calls",
    "connector_id": "'"$CONNECTOR_ID"'",
    "max_files": 500
  }'

The first run starts immediately unless you pass start: false. The response echoes the definition plus latest_run.

Filtering what gets imported

filters is an array of listing passes, mirroring the parameters you would pass when browsing a connector. Each set runs as its own pass and results are deduplicated, so overlapping sets are safe.

{
  "filters": [
    { "from": "2026-07-01", "to": "2026-09-30", "title_search": "customer" }
  ]
}

Which keys a connector honors depends on the connector — folder_id for Google Drive, path and recursive for Dropbox, team and meeting_type for Grain, title_search and date ranges for most. Unsupported keys are rejected when you create the import rather than silently ignored, because a saved definition re-runs forever and a quietly dropped filter would import the whole source.

Zoom and Gong default to a six-month lookback when no from is given. That date is frozen into the definition at creation so re-runs keep listing the same window — otherwise a refresh + delete_missing run would remove files that had merely aged out of the window.

Watch a run

curl "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID" \
  -H "Authorization: Bearer $CLOUDGLUE_API_KEY"

files_queued is the denominator; files_imported (media) or files_indexed (metadata) plus files_failed settle against it.

{
  "import_type": "media",
  "runs": [
    {
      "status": "processing",
      "progress": {
        "files_listed": 180,
        "files_queued": 154,
        "files_imported": 96,
        "files_skipped": 26,
        "files_failed": 2
      }
    }
  ]
}

Large runs take as long as the work takes — hundreds of long videos is normal for hours. A run keeps going in the background; you can close the connection and poll later.

Re-running: append, refresh, and mirroring

Every run picks a mode, defaulting to the definition’s default_mode.

append — add what's new

Imports files the collection does not have yet and retries previously failed ones. Files already imported are skipped: not re-downloaded, not re-billed. This is the mode to re-run on a schedule.

refresh — also re-sync existing files

Additionally re-fetches source metadata for files this import already brought in. For media imports the bytes are never re-downloaded — only the metadata is refreshed.

refresh + delete_missing — mirror the source

After a complete listing, removes files this import previously added that the source no longer returns. Files added by a different import, files you added yourself, and files this import has never listed are all left alone. Only the file’s place in this collection is removed — the file itself, and its use in any other collection, is untouched. A run that stopped at max_files skips this step entirely, because a truncated listing cannot prove anything is missing.To keep a file regardless of what the source does, add it to a collection this import does not target.

curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID/runs" \
  -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{ "mode": "refresh", "delete_missing": true }'

When a run stops early

Media imports are billed per file, so a run can stop on something account-wide — running out of credits, or hitting a usage limit. When that happens the run ends with a failed status and a plain-language error, and everything already imported is kept. Recovery is always the same move: fix the cause, then re-run in append mode. Already-imported files are skipped and only the remainder is attempted.

curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID/runs" \
  -H "Authorization: Bearer $CLOUDGLUE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{ "mode": "append" }'

Problems with a single file — too long, unsupported type, no longer in the source — fail only that file. The run continues and reports it in files_failed.

Cancelling

curl -X POST "https://api.cloudglue.dev/v1/collections/$COLLECTION_ID/imports/$IMPORT_ID/runs/$RUN_ID/cancel" \
  -H "Authorization: Bearer $CLOUDGLUE_API_KEY"

Files already imported stay in the collection. Deleting the import definition likewise removes only the definition and its run history — imported content is never removed.

Things to know

One active run per collection. Starting a second while one is in flight returns a 409; creating a definition still succeeds, it just comes back with latest_run: null so you can run it once the current one finishes.
Definitions are immutable. To change filters or connector, delete and recreate.
max_files caps the files a run takes from the listing, and a capped run never removes anything.
Runs use the workspace’s default API key to read the connector, so the workspace needs one.

Getting Started

Connect Your Data

Core Concepts

Deep Dives

Two kinds of import

Create an import

Filtering what gets imported

Watch a run

Re-running: append, refresh, and mirroring

When a run stops early

Cancelling

Things to know

Reference

Create a bulk import

Run a bulk import

Get a bulk import

Cancel a run

​Two kinds of import

​Create an import

​Filtering what gets imported

​Watch a run

​Re-running: append, refresh, and mirroring

​When a run stops early

​Cancelling

​Things to know

​Reference

Create a bulk import

Run a bulk import

Get a bulk import

Cancel a run

Two kinds of import

Create an import

Filtering what gets imported

Watch a run

Re-running: append, refresh, and mirroring

When a run stops early

Cancelling

Things to know

Reference