In this tutorial, we’ll walk through how to build a video Q&A chatbot using the CloudGlue API for a YouTube video. This chatbot will allow us to ask questions about the video, and get relevant answers related to the video.

​
Prerequisites

  • A Cloudglue account + API key (Get one here)
  • We’ll be using this YouTube video for this tutorial.
    • For the purpose of this tutorial, we’ve downloaded the YouTube video locally, to enable multimodal results.

​
Video Context

This particular video, is a public hearing for a 5 MW solar facility in Frederick County, Maryland, case number 9717, before the Maryland Public Service Commission.

The hearing is conducted by the Maryland Public Service Commission, in front of a judge. It features a presentation about the project’s location, size, and design, as well as its potential benefits and impact on the environment.

​
Preparation

​
Uploading the video

First, we’ll need to upload the video to Cloudglue. We can do this using the Cloudglue SDK.

import { CloudGlue } from '@aviaryhq/cloudglue-js';

const cloudGlue = new CloudGlue({
  apiKey: 'YOUR_API_KEY',
});

const fileName = 'PATH_TO_VIDEO_FILE'

const videoPath = path.join(os.homedir(), fileName);
const videoFile = new File([fs.readFileSync(videoPath)], fileName);

const file = await cloudGlue.files.uploadFile({
  file: videoFile,
});

​
Working with collections

Now that we have the video uploaded onto Cloudglue, we will be adding this file to a Rich Transcripts collection, to allow us to chat over the video. To learn more about these collections, check out the Rich Transcripts documentation.

Let’s create a collection first.

We’ll be using the transcribe_config to enable the summary, speech, scene_text, and visual_scene_description features, to maximize the amount of information that we can use to answer questions about the video.

const richTranscriptsCollection = await cloudGlue.collections.createCollection({
  name: 'Rich Transcripts Collection',
  collection_type: 'rich-transcripts',
  transcribe_config: {
    enable_summary: true,
    enable_speech: true,
    enable_scene_text: true,
    enable_visual_scene_description: true,
  },
});

Now that we have the collection created, we can add the video to it.

cloudGlue.collections.addVideo(richTranscriptsCollection.id, file.id); ```
</CodeGroup>

When we add a video to a `Rich Transcripts` collection, Cloudglue generate transcriptions for the video, based on the `transcribe_config` that we've assigned to the collection.

## Chatting

Now that we have the video in the collection, and we have transcriptions, we can start chatting over the video.

We'll be using the the Clouglue `Chat Completion` API to chat over the video.

Let's get started asking our first question about the video.

<CodeGroup>
```typescript JS SDK
const chatCompletion = await cloudGlue.chat.createChatCompletion({
  model: 'nimbus-001',
  messages: [{ role: 'user', content: 'What is the video about?' }],
  force_search: 'true',
  include_citations: 'true',
});

console.log(chatCompletion);

The above message, generates the response below.

{
"id": "57c99ec5-a4df-4413-b051-126182a31029",
"object": "chat.completion",
"created": 1747940508268,
"model": "nimbus-001",
"usage": {
  "prompt_tokens": 69,
  "completion_tokens": 450,
  "total_tokens": 870
},
"choices": [
  {
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "The video appears to be a recording of a public comment hearing for a 5 MW solar facility in Frederick County, Maryland, case number 9717, before the Maryland Public Service Commission. \n\nAt the beginning of the video, a man, Chuck McLean, introduces himself as the chief public utility law judge at the Maryland Public Service Commission and explains that he is filling in for Judge Christine Burke for this hearing. He states that the hearing is being held to receive public comments on the application for the solar facility.\n\nThe video then transitions to a presentation by Chaberton Energy, the company proposing the solar project. The presenter discusses the benefits of the project, including the immediate environmental benefits and the replacement of non-renewable sources of energy. The presenter also explains that the project will include planting pollinator habitats to increase farming production in the local area and support the local bee and insect population.\n\nThe video includes various slides and diagrams showing the layout of the solar panel project, including the location of the solar panels, security fencing, and landscaping. The presenter explains that the project will be located on a 12-acre parcel on Old Frederick Road in Rocky Ridge and will include a 5 MW solar array.\n\nThe video also includes a discussion of the project's design and construction, including the use of existing access roads and the installation of security fencing and landscaping. The presenter explains that the project will be connected to Potomac Edison's existing 34 kV circuit and that the company will extend the overhead circuit northeast along Old Frederick Road for approximately 5,000 feet.\n\nThroughout the video, there are also interactions between the law judge, Chuck McLean, and various participants, including representatives from the Office of People's Counsel and the commission staff. The law judge asks if anyone has any questions or comments, and the participants respond accordingly.\n\nOverall, the video provides an overview of the proposed solar facility and the public comment hearing process, including the presentation by Chaberton Energy and the discussions between the law judge and the participants."
    },
    "citations": [
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Introduction by Chuck McLean, chief public utility law judge",
        "segment_id": "153d9b48-66bb-467e-9c56-c025e354c683",
        "start_time": 20,
        "end_time": 40,
        "visual_scene_description": [
          {
            "text": "A man with short graying hair, wearing a light brown suit jacket and a blue shirt, speaks directly to the camera. He is centered in the frame, with a white wall behind him. On the wall are two framed pictures and a display of small items. The lighting is neutral, and the camera remains static throughout the scene.",
            "start_time": 20,
            "end_time": 40
          }
        ],
        "scene_text": [
          {
            "text": "Law Judge",
            "start_time": 20,
            "end_time": 39
          }
        ],
        "speech": [
          {
            "speaker": "0",
            "text": "Good evening, everyone.",
            "start_time": 19.695,
            "end_time": 20.575
          },
          {
            "speaker": "0",
            "text": "Today is 05/22/2024, and it's approximately 7PM.",
            "start_time": 20.575,
            "end_time": 25.055
          },
          {
            "speaker": "0",
            "text": "My name is Chuck McLean.",
            "start_time": 25.055,
            "end_time": 26.095001
          },
          {
            "speaker": "0",
            "text": "I'm the chief public utility law judge at the Maryland Public Service Commission, and I am filling in for judge Christine Burke, uh, for the, uh, just this hearing, so you're not gonna be stuck with me at least for the duration of this particular case.",
            "start_time": 26.095001,
            "end_time": 38.25
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Presentation by Chaberton Energy, including benefits of the project",
        "segment_id": "5e83cda2-99fc-462d-a984-4c7f06ad290b",
        "start_time": 40,
        "end_time": 60,
        "visual_scene_description": [
          {
            "text": "A man in a suit and blue shirt speaks directly to the camera. He is in front of a white wall with framed pictures. The camera angle is a medium shot, focusing on the man's face and upper body.",
            "start_time": 40,
            "end_time": 60
          }
        ],
        "scene_text": [
          {
            "text": "Law Judge",
            "start_time": 40,
            "end_time": 60
          }
        ],
        "speech": []
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Discussion of pollinator habitats and support for local bee and insect population",
        "segment_id": "3f50f935-97cb-413c-b850-dd0199fb4358",
        "start_time": 60,
        "end_time": 80,
        "visual_scene_description": [
          {
            "text": "A man with short gray hair and a tan suit speaks directly to the camera. He is indoors with a white wall behind him with two framed pictures.",
            "start_time": 60,
            "end_time": 80
          }
        ],
        "scene_text": [
          {
            "text": "Law Judge",
            "start_time": 60,
            "end_time": 80
          }
        ],
        "speech": [
          {
            "speaker": "0",
            "text": "Now no one, uh, signed up to speak this evening.",
            "start_time": 69.365005,
            "end_time": 71.925
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Slides and diagrams showing the layout of the solar panel project",
        "segment_id": "3fa84a28-8030-432f-b67c-2ef0e691b64f",
        "start_time": 100,
        "end_time": 120,
        "visual_scene_description": [
          {
            "text": "A man with short graying hair wearing a suit jacket and blue shirt is speaking directly to the camera. Behind him is a white wall with two framed pictures. The lighting is bright and the camera angle is a medium close-up.",
            "start_time": 100,
            "end_time": 120
          }
        ],
        "scene_text": [
          {
            "text": "Law Judge",
            "start_time": 100,
            "end_time": 119
          }
        ],
        "speech": [
          {
            "speaker": "0",
            "text": "Uh, attention, Jamie Bergen, chief clerk, six Saint Paul Street, sixteenth floor, Baltimore, Maryland two one two zero two.",
            "start_time": 100.565,
            "end_time": 109.365
          },
          {
            "speaker": "0",
            "text": "Uh, and please reference case number 9717 so the case, uh, so the comments make it to the correct file.",
            "start_time": 109.685,
            "end_time": 115.6
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Location of the solar panels, security fencing, and landscaping",
        "segment_id": "699d58aa-5612-4a88-8a35-492ebda74d0c",
        "start_time": 120,
        "end_time": 140,
        "visual_scene_description": [
          {
            "text": "A man with short, graying hair and a light complexion is speaking directly to the camera. He is wearing a tan suit jacket and a light blue shirt. The background is a plain white wall with three framed pictures hanging on it. The lighting is neutral, and the camera angle is a medium close-up, focusing on his face and upper body.",
            "start_time": 120,
            "end_time": 140
          }
        ],
        "scene_text": [
          {
            "text": "Law Judge",
            "start_time": 120,
            "end_time": 121
          }
        ],
        "speech": [
          {
            "speaker": "0",
            "text": "Additionally, the application is available, uh, to the public for an inspection at the Frederick County Department of Development Review and Planning at 30 North Market Street in Frederick County, Maryland.",
            "start_time": 128.30501,
            "end_time": 139.66501
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Project design and construction, including use of existing access roads",
        "segment_id": "6d5b9833-5ad3-4fa1-a11f-c063c8f3337c",
        "start_time": 140,
        "end_time": 160,
        "visual_scene_description": [
          {
            "text": "A man wearing a suit jacket and blue shirt is centered in the frame. He is speaking directly to the camera. Behind him is a white wall with two framed pictures hanging on it. To the left of the frame, slightly behind the man, is a clear plastic container with a bag of Fritos inside.",
            "start_time": 140,
            "end_time": 160
          }
        ],
        "scene_text": [
          {
            "text": "Law Judge",
            "start_time": 140,
            "end_time": 159
          }
        ],
        "speech": [
          {
            "speaker": "0",
            "text": "In terms of where this case sits procedurally, the applicant's direct testimony was filed on May 2024.",
            "start_time": 139.905,
            "end_time": 147.8
          },
          {
            "speaker": "0",
            "text": "In the power plant research program, the office of people's council, and the commission staff's testimony is due to be filed on August 2024.",
            "start_time": 147.87999,
            "end_time": 158.255
          },
          {
            "speaker": "0",
            "text": "Excuse me.",
            "start_time": 158.255,
            "end_time": 158.975
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Installation of security fencing and landscaping",
        "segment_id": "e98b8e43-df4d-43e1-b168-b399d8b305e0",
        "start_time": 340,
        "end_time": 360,
        "visual_scene_description": [
          {
            "text": "The video displays a title graphic for Chaberton Energy, with a group of people holding paintings of solar panels. The paintings are displayed on a brick wall. Text overlays provide information about the company, while a presenter is visible in a small window in the top right corner.",
            "start_time": 340,
            "end_time": 360
          }
        ],
        "scene_text": [
          {
            "text": "chaberton\nENERGY\nImpact &\nEngagement\nCommunity\nFeedback in\nOur Process\n100+ Projects\nin Current\nDevelopment\nHeadquartered in Maryland\n~45 employees\nFounded in 2020\nOver 300 combined years of solar experience",
            "start_time": 340,
            "end_time": 359
          }
        ],
        "speech": [
          {
            "speaker": "3",
            "text": "There will also be an immediate environmental benefit consistent with Maryland's renewable energy and sustainability goals, including replacing nonrenewable sources of energy and planting pollinator habitat that can increase farming production in the local area and support the local bee and insect population.",
            "start_time": 339.25497,
            "end_time": 357.49
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Connection to Potomac Edison's existing 34 kV circuit",
        "segment_id": "8c8ab690-8f43-4eed-a204-fa0869e0190c",
        "start_time": 360,
        "end_time": 380,
        "visual_scene_description": [
          {
            "text": "The video begins with a slide from Chaberton Energy. The slide shows a large group of people holding paintings of solar panels. Text on the slide provides information about the company, including that it is headquartered in Maryland, has approximately 45 employees, and was founded in 2020. Amie Beame is in the upper right corner of the screen.",
            "start_time": 360,
            "end_time": 380
          }
        ],
        "scene_text": [
          {
            "text": "chaberton\nENERGY\nImpact &\nEngagement\nCommunity\nFeedback in\nOur Process\n100+ Projects\nin Current\nDevelopment\nHeadquartered in Maryland\n~45 employees\nFounded in 2020\nOver 300 combined years of solar experience",
            "start_time": 360,
            "end_time": 379
          }
        ],
        "speech": [
          {
            "speaker": "3",
            "text": "This allows Marylanders to be sure of the savings and benefits they will receive from the project.",
            "start_time": 363.17502,
            "end_time": 369.09503
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Extension of the overhead circuit northeast along Old Frederick Road",
        "segment_id": "4df1407f-c4e2-4729-aec3-a890aeb65d3c",
        "start_time": 380,
        "end_time": 400,
        "visual_scene_description": [
          {
            "text": "A large group of people stand together, each holding a painting of solar panels in a field. The Chaberton Energy logo is displayed above them, with text about the company's headquarters, founding year, number of employees, and combined years of solar experience. A woman is visible in a small window in the upper right corner.\nA slide with a grid of blue squares resembling solar panels appears. Text boxes with titles like \"What is Community Solar?\", \"Benefits\", and \"Thank You\" are overlaid on the grid. The Chaberton Energy logo remains in the upper left corner, and the woman is still visible in the small window.\nThe slide changes to show a site plan of a solar project. The Chaberton Energy logo and the woman's video feed remain in the upper corners. Text labels and images related to the project are displayed.\nThe slide changes to a detailed aerial view of a solar panel array within a fenced area. The Chaberton Energy logo and the woman's video feed remain in the upper corners. The aerial view shows surrounding greenery and other details of the landscape.",
            "start_time": 380,
            "end_time": 400
          }
        ],
        "scene_text": [
          {
            "text": "chaberton\nENERGY\nImpact & Engagement\nCommunity\nFeedback in\nOur Process\n100+ Projects\nin Current\nDevelopment\nHeadquartered in Maryland\n~45 employees\nFounded in 2020\nOver 300 combined years of solar experience",
            "start_time": 380,
            "end_time": 385
          },
          {
            "text": "chaberton\nENERGY\nPROJECT\nBENEFITS\nWhat is\nCommunity\nSolar?\nThank You",
            "start_time": 385,
            "end_time": 386
          },
          {
            "text": "chaberton\nENERGY\nPROJECT\nTotal parcel: 112 acres\nProject area: 53 acres\nSite Plan\nMaryland CPCN\nProcess\nCompliance\nFrom Construction to\nDecommissioning to",
            "start_time": 386,
            "end_time": 389
          },
          {
            "text": "Zoom In",
            "start_time": 394,
            "end_time": 395
          }
        ],
        "speech": [
          {
            "speaker": "3",
            "text": "Project Wild Turkey is a five megawatt solar project located in Potomac Edison service territory on a 12 acre parcel on Old Frederick Road in Rocky Ridge.",
            "start_time": 384.87,
            "end_time": 397.855
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Interactions between the law judge and participants, including representatives from the Office of People's Counsel",
        "segment_id": "75fd6037-9e1e-49cb-8bba-4bd196a69cbe",
        "start_time": 420,
        "end_time": 440,
        "visual_scene_description": [
          {
            "text": "The video begins with a medium shot of a woman speaking in the upper right corner of the screen. To the left of her, a detailed aerial plan of a solar panel project is displayed. The plan includes outlines of the proposed solar panel area, access roads, and fencing. The background is blurred, with hints of pink and green, possibly from flowers or foliage.\nThe camera zooms in on the aerial plan of the solar panel project. The plan details the layout of solar panels within a fenced perimeter, access roads, and surrounding landscape features. The solar panels are arranged in rows, and the plan includes annotations and labels for various components such as security fencing, landscape screening, and electric lines.",
            "start_time": 420,
            "end_time": 440
          }
        ],
        "scene_text": [
          {
            "text": "PROPOSED DIVISION\nAREA (APPROX. 25.45\nACRES) (10.30 HECTARES)",
            "start_time": 420,
            "end_time": 421
          },
          {
            "text": "PROPOSED DIVISION\nAREA (APPROX. 25.45\nACRES) (10.30 HECTARES)\nCLEAR EXISTING\nOR IRRIGATION\nLINE (TYP.)\nPROPOSED SECURITY\nFENCE (TYP.)\nMODULAR GROUND\nMOUNTED SOLAR ARRAY\n(APPROX. 18.15 ACRES)\n(7.35 HECTARES) (TYP.)\nACCESS WITHOUT\nIMPACT TO\nTHE MAIN\nCHANNEL (TYP.)\nEXISTING CULVERT\nTO BE UPGRADED\nPROPOSED 12' WIDE\nACCESS DRIVE WITH TURNING\nRADIUS TO BE UPGRADED\n(TYP.)\nPROPOSED\nDOUBLE GATE\nUTILITY POINT OF\nINTERCONNECTION\nUTILITY METER\nAND MAIN\nSWITCH\nCLEARANCE\n1.5-2METER\nCLEARANCE WITHIN\nACCESS DRIVEWAY\nAND UTILITY ACCESSIBLE\nEASEMENT AREA\nAND DISCONNECT POLE\nPROPOSED\nELECTRIC LINE (TYP.)\nPROPOSED LANDSCAPE\nSCREENING (TYP.)\nPROPOSED SOLAR\nLANDSCAPE\nSCREENING (ASO\n(ASO GRASSING)\nTRANSFORMERS, INVERTERS,\nAND COMPONENTS REQUIRED\nTO CONVERT POWER (TYP.)\nPROPOSED\nPAD FOR\nTRANSFORMER\n(TYP.)\nPROPOSED SOLAR\nLANDSCAPE\nSCREENING (TYP.)\nNOTE:\nBRIGHT SURFACES,\nINCLUDING SOLAR PANELS\nTO BE LOCATED 100M\nFROM FENCE\n@chobertson\nArnie Beame - Chab...",
            "start_time": 420,
            "end_time": 439
          }
        ],
        "speech": [
          {
            "speaker": "3",
            "text": "To gain access to the site, we will use the existing access entrance located on the property.",
            "start_time": 420.255,
            "end_time": 425.935
          },
          {
            "speaker": "3",
            "text": "The project's access to the site will be controlled via a locked gate with seven foot high agricultural style fencing around the perimeter of the project.",
            "start_time": 426.43997,
            "end_time": 435.88
          }
        ]
      }
    ]
  }
]
}

We can see that the response includes a message, with the answer to our question, and a list of citations. The citations are ways for the user to verify the answer that the chat completion is based on.

If we don’t need the citations, we can remove the include_citations parameter from the request.

const chatCompletion = await cloudGlue.chat.createChatCompletion({
  model: 'nimbus-001',
  messages: [{ role: 'user', content: 'What is the video about?' }],
  force_search: 'true',
  include_citations: 'false',
});

console.log(chatCompletion);


{
"id": "0aa6e0f5-8c3d-4e02-ad93-e30946467131",
"object": "chat.completion",
"created": 1747944451665,
"model": "nimbus-001",
"usage": {
  "prompt_tokens": 69,
  "completion_tokens": 364,
  "total_tokens": 784
},
"choices": [
  {
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "The video segments provided appear to be from a public hearing for a 5 MW solar facility in Frederick County, as indicated by the case number 9717. The hearing is conducted by a law judge, who explains the procedures for making public comments and introduces the parties involved, including representatives from the Department of Natural Resources and the Office of People's Council.\n\nThe video segments also feature a presentation by a woman, likely a representative of the solar project, who discusses the project's design and layout. She explains that the project will utilize a modular ground-mounted solar array, with a proposed security fence and landscape screening to minimize the visual impact. The project will also include equipment banks for inverters and transformers, and a proposed underground electric line.\n\nThe presentation includes detailed diagrams and maps of the project site, showing the layout of the solar panels, access roads, and surrounding landscape features. The woman also discusses the project's proximity to environmentally sensitive areas, such as wetlands and floodplains.\n\nThe law judge also mentions that a second public hearing will be held in person, with the time and location to be determined, and an evidentiary hearing is currently set for September 10, 2024.\n\nOverall, the video segments suggest that the hearing is focused on reviewing and discussing the proposed 5 MW solar facility in Frederick County, with a emphasis on the project's design, layout, and potential environmental impacts.\n\nTo directly answer the query \"what is this about,\" this is about a public hearing for a proposed 5 MW solar facility in Frederick County, specifically Case No. 9717, where the project's design, layout, and potential environmental impacts are being reviewed and discussed."
    }
  }
]
}

Now that we’ve tested that the chat completion works, let’s ask some more complicated questions that require searching through the video.

We’ll ask a question that requires looking at different aspects of the video, whether through speech, visual scene description, or scene text.

Specifically, we want to find out what the positive benefits of this project are to the local community.

const chatCompletion = await cloudGlue.chat.createChatCompletion({
  model: 'nimbus-001',
  messages: [{ role: 'user', content: 'What are some positive benefits that this project can bring to the local community' }],
  force_search: 'true',
  include_citations: 'true',
});

console.log(chatCompletion);


{
"id": "b17f3249-db24-46d6-a80b-472bfe59bd5b",
"object": "chat.completion",
"created": 1747945287087,
"model": "nimbus-001",
"usage": {
  "prompt_tokens": 69,
  "completion_tokens": 252,
  "total_tokens": 672
},
"choices": [
  {
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "The project in question is a community solar project proposed by Chaberton Energy, which aims to provide various benefits to the local community. \n\nAs stated in the video segment from 05:20 to 05:40, this community solar project will benefit the local community in many ways, including generating tax revenue for the state and county. Additionally, subscribers to the project can expect an average of $300 in annual savings.\n\nFurthermore, as mentioned in the segment from 05:40 to 06:00, there will be an immediate environmental benefit consistent with Maryland's renewable energy and sustainability goals. This includes replacing non-renewable sources of energy and planting pollinator habitat, which can increase farming production in the local area and support the local bee and insect population.\n\nThe project's design also incorporates features that minimize its impact on the surrounding environment. For example, as shown in the segment from 07:00 to 07:20, the project includes a 25-foot wide landscaping buffer along the remaining border of the project, featuring a double row of plantings.\n\nOverall, the project is expected to have a positive impact on the local community, providing economic benefits, environmental benefits, and contributing to the state's renewable energy goals."
    },
    "citations": [
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Community solar project benefits, including generating tax revenue and providing average annual savings of $300",
        "segment_id": "035fe6de-f253-4c5b-913b-6cf1f2f147d1",
        "start_time": 320,
        "end_time": 340,
        "visual_scene_description": [
          {
            "text": "The video starts with a visual screen that shows a group of people holding solar panel paintings. In the top left corner is the Chaberton Energy logo. In the top right corner is a small video of a woman speaking. Below the group of people, in the bottom left corner, is the text \"Headquartered in Maryland\", \"Founded in 2020\". To the right of that, in the bottom right corner, is the text \"~45 employees\", and \"Over 300 combined years of solar experience\".",
            "start_time": 320,
            "end_time": 340
          }
        ],
        "scene_text": [
          {
            "text": "chaberton\nENERGY\nImpact &\nEngagement\nCommunity\nFeedback in\nOur Process\n100+ Projects\nin Current\nDevelopment\nHeadquartered in Maryland\n~45 employees\nFounded in 2020\nOver 300 combined years of solar experience",
            "start_time": 320,
            "end_time": 339
          }
        ],
        "speech": [
          {
            "speaker": "3",
            "text": "This community solar project will benefit the local community in many ways, including tax revenue to the state and county and an average of $300 annual savings for subscribers to the project.",
            "start_time": 327.31998,
            "end_time": 338.615
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "Immediate environmental benefits, including replacing non-renewable energy sources and planting pollinator habitats",
        "segment_id": "e98b8e43-df4d-43e1-b168-b399d8b305e0",
        "start_time": 340,
        "end_time": 360,
        "visual_scene_description": [
          {
            "text": "The video displays a title graphic for Chaberton Energy, with a group of people holding paintings of solar panels. The paintings are displayed on a brick wall. Text overlays provide information about the company, while a presenter is visible in a small window in the top right corner.",
            "start_time": 340,
            "end_time": 360
          }
        ],
        "scene_text": [
          {
            "text": "chaberton\nENERGY\nImpact &\nEngagement\nCommunity\nFeedback in\nOur Process\n100+ Projects\nin Current\nDevelopment\nHeadquartered in Maryland\n~45 employees\nFounded in 2020\nOver 300 combined years of solar experience",
            "start_time": 340,
            "end_time": 359
          }
        ],
        "speech": [
          {
            "speaker": "3",
            "text": "There will also be an immediate environmental benefit consistent with Maryland's renewable energy and sustainability goals, including replacing nonrenewable sources of energy and planting pollinator habitat that can increase farming production in the local area and support the local bee and insect population.",
            "start_time": 339.25497,
            "end_time": 357.49
          }
        ]
      },
      {
        "collection_id": "7479d770-7efa-4965-aacb-01d8a2d6d0d2",
        "file_id": "5d607259-5621-4da5-b137-e06929990463",
        "text": "25-foot wide landscaping buffer along the project border, featuring a double row of plantings",
        "segment_id": "aec02cf3-dc74-41ca-84c2-d566fa891083",
        "start_time": 400,
        "end_time": 420,
        "visual_scene_description": [
          {
            "text": "The video opens with a background of blurred pink and white flowers. Superimposed on the background is a Zoom call. The main visual is a detailed aerial map of a proposed solar panel project, outlining the area with a green dotted line. A blue line indicates a river or stream running nearby. In the top right corner, a small inset video shows a woman with blonde hair, speaking about the project.",
            "start_time": 400,
            "end_time": 420
          }
        ],
        "scene_text": [
          {
            "text": "@choberton",
            "start_time": 400,
            "end_time": 419
          }
        ],
        "speech": []
      }
    ]
  }
]
}

The chat completion has given us its response, based off its search of the video.

The project in question is a community solar project proposed by Chaberton Energy, which aims to provide various benefits to the local community.
As stated in the video segment from 05:20 to 05:40, this community solar project will benefit the local community in many ways, including generating tax revenue for the state and county.
Additionally, subscribers to the project can expect an average of $300 in annual savings.
Furthermore, as mentioned in the segment from 05:40 to 06:00, there will be an immediate environmental benefit consistent with Maryland's renewable energy and sustainability goals.
This includes replacing non-renewable sources of energy and planting pollinator habitat, which can increase farming production in the local area and support the local bee and insect population.
The project's design also incorporates features that minimize its impact on the surrounding environment.
For example, as shown in the segment from 07:00 to 07:20, the project includes a 25-foot wide landscaping buffer along the remaining border of the project, featuring a double row of plantings.
Overall, the project is expected to have a positive impact on the local community, providing economic benefits, environmental benefits, and contributing to the state's renewable energy goals.

Fantastic, we can see from the response, that there is financial savings for residents, plus some benefits to the local wildlife. The answer has also referenced some timestamps to give better insight of where they refer to this.

That looks great, but how can we be sure that the information is correct?

We can look at the citations, and see what sources the chat completion used to answer the question.

In this case, we can see that we’ve picked out the expected annual savings that residents would expect to get from the project, based on the first citation.

The second citation also provides some insight into the environmental benefits of the project.

​
Conclusion

In this tutorial, we’ve seen how to do the following:

  • Upload a video to Cloudglue
  • Create a Rich Transcripts collection to allow us to chat over the video
  • Add a video to a Rich Transcripts collection
  • Use chat completion to answer questions about our video