---
updatedAt: 2026-09-15T10:54:45.000Z
agentTools:
projectIndex: https://docs.d-id.com/llms.txt
---
# Agents SDK Overview
{`
Welcome to the Agents SDK documentation!
Here, you’ll find everything you need to know to get started with the SDK, understand its core concepts, utilize built-in methods, and access additional resources.
This guide is designed to help you integrate the Agents SDK into your projects effectively and efficiently.
`}
## Introduction
The D-ID Agents SDK provides a seamless integration pathway for embedding your created Agents or real-time streaming avatars into web applications.
With a streamlined and user-friendly workflow, you can easily harness the capabilities of the D-ID Agents and Streams API right out of the box.
The SDK supports three avatar types:
* **Talks (V2)** — Photo-based presenters using WebRTC streaming.
* **Clips (V3)** — Pre-built presenter avatars using WebRTC streaming.
* **Expressives (V4)** — Next-generation avatars using LiveKit-based streaming, supporting microphone input and always-on fluent mode.
**Please note:** This SDK is designed for front-end development only. The creation of Agents and Knowledge bases should be handled through the [Agents API](https://docs.d-id.com/docs/agent-quickstart) or directly within the [ D-ID Studio](https://studio.d-id.com/agents).
> 📚 **Looking for a specific method, callback or type?** The full **[Agents SDK API reference](https://sdk.d-id.com/)** is generated from the SDK source on every release and always matches the published npm version.
## Getting Started
### ➤Prerequisites
Follow these steps:
1. Log in to the [D-ID Studio](http://studio.d-id.com)
2. Create a new Agent with the required options - Image, voice, etc.
3. In the [Agents gallery](https://studio.d-id.com/agents), hover with your mouse over the created Agent, then click on the `[...]` button
4. Click on ` Embed` button
5. Set the list of allowed domains for your Agent, for example: `http://localhost`
This is an additional security measurement: your Agent can be accessed only from the domains allowed by you.
6. In the code snippet section, fetch the `data-client-key` and the `data-agent-id`, these will be used later to access your Agent.
### ➤Installation
In your front-end application folder, install the Agents SDK library using `npm`.
```shell
npm i @d-id/client-sdk
```
Alternatively, you can clone the SDK from its [GitHub repository](https://github.com/de-id/agents-sdk).
### ➤Initialization
In your front-end application,
1. Import the Agents SDK library
2. Paste the `data-agent-id` obtained in the prerequisites step in the `agentId` variable
3. Paste the `data-client-key` obtained in the prerequisites step in the `auth.clientKey` variable
4. Define an object called `callbacks`.
This will be explained in the [Usage section](#➤-%EF%B8%8F-callback-functions) in this guide, and documented in full in the [`AgentManagerCallbacks` reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html).
5. Define an object called `streamOptions` \[optional]
This will be explained in the [Usage section](#➤-%EF%B8%8F-stream-options) in this guide, and documented in full in the [`StreamOptions` reference](https://sdk.d-id.com/interfaces/StreamOptions.html).
6. Create an instance of the `createAgentManager` object called `agentManager` with the values created above.
This will be explained later in the [Usage section](#➤-%EF%B8%8F-agent-manager) in this guide. See [`createAgentManager()`](https://sdk.d-id.com/functions/createAgentManager.html) and [`AgentManager`](https://sdk.d-id.com/interfaces/AgentManager.html) in the API reference.
Example:
```javascript
// 1. Import the Agents SDK library
import * as sdk from '@d-id/client-sdk';
// 2. Paste the `data-agent-id' in the 'agentId' variable
let agentId = 'agt_fumf1234';
// 3. Paste the 'data-client-key' in the 'auth.clientKey' variable
let auth = {
type: 'key',
clientKey: 'Z3123asdaczxSXSAasdcxzcashDY6MGSASFsafxSDdfASY2k0TUhPcEVsTnBR',
};
// 4. Define the SDK callbacks functions in this object
const callbacks = {
onSrcObjectReady(value) {
videoElement.srcObject = value;
srcObject = value;
return srcObject;
},
//....Rest of the Callback functions here....//
//...................................//
};
// 5. Define the Stream Options object (Optional — v2/v3 avatars only)
let streamOptions = { compatibilityMode: 'auto', streamWarmup: true };
//....Rest of the APP's code here....//
//...................................//
// 6. Create the 'agentManager' instance with the values created above
let agentManager = await sdk.createAgentManager(agentId, {
auth,
callbacks,
streamOptions,
});
```
## Usage
### ➤Agent Manager
#### **Built-in Properties**
The `agentManager` object created during initialization has several built-in parameters that might come in handy.
* **[`agentManager.agent`](https://sdk.d-id.com/interfaces/AgentManager.html#agent)**
Displaying all of the Agent's saved information (Same as the following [endpoint](/reference/agent-get))
* **[`agentManager.starterMessages`](https://sdk.d-id.com/interfaces/AgentManager.html#startermessages)**
Displaying the Agent's defined Starter Messages.
#### **Built-in Methods**
The `agentManager` object created during initialization has several built-in methods that allow you to interact with your Agent.
* **[`agentManager.connect()`](https://sdk.d-id.com/interfaces/AgentManager.html#connect)**
Method to create a new connection with an Agent (new WebRTC connection, web-socket, new Agent chat ID)
* **[`agentManager.speak({type, input})`](https://sdk.d-id.com/interfaces/AgentManager.html#speak)**
Method to make your Agent stream back a video based on a text or audio file.
(Similar to [Talks Streams](https://docs.d-id.com/reference/talks-streams-overview) / [Clips Streams API](https://docs.d-id.com/reference/clips-streams-overview))
```javascript Text - JavaScript
let speak = agentManager.speak({
type: 'text',
input: "Hi! I'm Alice!",
});
```
```javascript Audio File - JavaScript
let speak = agentManager.speak(
{
type: "audio",
audio_url: "http://www.yourwebsite.com/audio.mp3"
}
);
```
Text scripts also accept two optional fields, supported for Expressive (V4) agents only:
* `sentiment` - the sentiment name to speak with, for example: `"friendly"`, `"excited"`, `"professional"`, `"empathetic"`, `"frustrated"`. If the requested sentiment is not supported by the agent, the default sentiment is used.
* `should_queue_speaks` - when `true`, the speak is queued behind the currently playing speech instead of interrupting it. Defaults to `false`.
```javascript Text with Sentiment - JavaScript
let speak = agentManager.speak({
type: 'text',
input: "Hi! I'm Alice!",
sentiment: 'friendly',
});
```
```javascript Queued Speak - JavaScript
let speak = agentManager.speak({
type: 'text',
input: "And here is a follow-up line.",
should_queue_speaks: true,
});
```
* **[`agentManager.chat(string)`](https://sdk.d-id.com/interfaces/AgentManager.html#chat)**
Method to send a message to your Agent and get a streamed video based on its answer (LLM)
```javascript JavaScript
let chat = agentManager.chat('What is the distance to the moon?');
```
* **[`agentManager.rate(messageID, score)`](https://sdk.d-id.com/interfaces/AgentManager.html#rate)**
Method to rate the Agent's answer in the chat - for future analytics and insights.
* **[`agentManager.reconnect()`](https://sdk.d-id.com/interfaces/AgentManager.html#reconnect)**
Method to reconnect to the Agent when the session expires and continue the conversation on the same chat ID.
* **[`agentManager.disconnect()`](https://sdk.d-id.com/interfaces/AgentManager.html#disconnect)**
Method to close the existing connection and chat with the Agent.
* **[`agentManager.interrupt(interrupt)`](https://sdk.d-id.com/interfaces/AgentManager.html#interrupt)**
Method to interrupt the current video stream mid-playback. It never throws — it does nothing when interrupting is not available or not currently allowed.
Supported for Fluent streams (V3 Pro Avatars) and all Expressive (V4) agents.
* **[`agentManager.publishMicrophoneStream(stream)`](https://sdk.d-id.com/interfaces/AgentManager.html#publishmicrophonestream)**
**Supported only with Expressive (V4) agents.**
Method to publish a microphone audio track to the session. Call after `connect()` to enable voice input.
```javascript JavaScript
const micStream = await navigator.mediaDevices.getUserMedia({ audio: true });
await agentManager.publishMicrophoneStream(micStream);
```
* **[`agentManager.unpublishMicrophoneStream()`](https://sdk.d-id.com/interfaces/AgentManager.html#unpublishmicrophonestream)**
**Supported only with Expressive (V4) agents.**
Method to stop and remove the currently published microphone track from the session.
```javascript JavaScript
await agentManager.unpublishMicrophoneStream();
```
### ➤Callback Functions
Callback functions enable you to manage various events throughout the SDK lifecycle. Each function is linked to one or more methods within the built-in `agentManager` and triggers automatically to handle specific events efficiently
\[*MANDATORY for using the SDK*] - Linking the Streamed video and audio to the HTML element.
The `value` of this callback function is passed to the HTML video element in the following function.
Triggered when `agentManager.connect(), agentManager.reconnect(), agentManager.disconnect()` are called.
[Full reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html#onsrcobjectready)
```javascript
onSrcObjectReady(value) {
videoElement.srcObject = value
srcObject = value
return srcObject
}
```
Displaying the state of the streamed video, used for switching the HTML element's source between the idle and streamed videos.
Triggered when `agentManager.chat() and agentManager.speak()` are called.
[Full reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html#onvideostatechange)
```javascript
onVideoStateChange(state) {
console.log("onVideoStateChange(): ", state)
if (state == "STOP") {
videoElement.srcObject = undefined
videoElement.src = agentManager.agent.presenter.idle_video
}
else {
videoElement.src = ""
videoElement.srcObject = srcObject
connectionLabel.innerHTML = "Online"
}
}
```
Displaying the different connection states with the Agent's WebRTC stream connection
Triggered when `agentManager.connect(), agentManager.reconnect(), agentManager.disconnect()` are called.
[Full reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html#onconnectionstatechange)
```javascript
onConnectionStateChange(state) {
console.log("onConnectionStateChange(): ", state)
if (state == "connected") {
console.log("I'm ready to go!")
}
}
```
```javascript Example Values
state: ['new', 'fail', 'connecting', 'connected', 'disconnected', 'closed'];
```
Displaying the chat messages array when a new message is sent to the chat.
`type`: `answer` indicates the full answer replied in the streamed video.
`role`: `user`, `assistant`(Agent)
Triggered when `agentManager.chat()` is called:
[Full reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html#onnewmessage)
```javascript
onNewMessage(messages, type) {
console.log(messages, type)
}
```
```javascript Example Values
type: ['partial', 'answer', 'user'];
messages: [
{
role: 'assistant',
content: "Hi! I'm an Agent. How can I help you?",
created_at: '2024-07-08T08:35:54.503Z',
id: '35113960da531',
},
{
role: 'user',
content: 'What is the distance to the moon?',
created_at: '2024-07-08T08:36:48.036Z',
id: 'f82377af5a9c4',
},
{
role: 'assistant',
content:
"The average distance to the moon is about 238,855 miles (384,400 kilometers). That's about 30 times the diameter of the Earth!",
id: '49b86saf2aff8',
created_at: '2024-07-08T08:36:48.037Z',
matches: [],
},
];
```
Triggered when the user's internet connectivity state changes. For Talks (V2) and Clips (V3) agents this is derived from jitter-buffer delay and freeze count on the video stream; for Expressive (V4) agents it is LiveKit's reported connection quality.
[Full reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html#onconnectivitystatechange)
```javascript
onConnectivityStateChange(state) {
console.log("onConnectivityStateChange(): ", state)
}
```
```javascript Example Values
state: ['STRONG', 'WEAK', 'UNKNOWN'];
```
Throwing an error and displaying the error message when things go badly.
[Full reference](https://sdk.d-id.com/interfaces/AgentManagerCallbacks.html#onerror)
```javascript
onError(error, errorData) {
console.log("Error:", error, "Error Data", errorData)
}
```
### ➤Stream Options
> **Note:** `streamOptions` apply only to Talks (V2) and Clips (V3) agents. Expressive (V4) avatars manage transport settings automatically and do not use these options. See the full [`StreamOptions` reference](https://sdk.d-id.com/interfaces/StreamOptions.html).
* **`compatibilityMode`**:
Defines the video codec to be used in the stream.
When set to `"on"`: VP8 will be used. **Recommended**
When set to `"off"`: H264 will be used
When set to `"auto"` - the codec will be selected according to the browser.
* **`streamWarmup`**:
Allowed values:
`true` - warmup video will be streamed when the connection is established.
`false` - no warmup video (default)
* **`sessionTimeout`**:
**Can only be used with proper permissions**
Maximum duration (in seconds) between messages before the session times out.
Max value: `300`
* **`outputResolution`**:
**Supported only with Talk presenters (photo-based).**
The output resolution sets the maximum height or width pixels of the streamed video.
When resolution is not configured, it defaults to the agent output resolution.
Allowed values: `150 - 1080`
* **`fluent`**:
**Supported with Agents created with V3 Pro Avatars. Always enabled for V4 Avatars.**
Allowed values:
`true` - Fluent streaming (one video for Idle/Talking states
`false` - Legacy streaming mode (2 video elements)
See the Demo Repository below for more details
## See it in Action
Explore our demo repository on GitHub to see the Agents SDK in action!
This repository features a sample project crafted in Vanilla JavaScript and Vite, utilizing the Agents SDK to help you get started swiftly.
[GitHub Demo Repository](https://github.com/de-id/Agents-SDK-Demo)
## Support