Custom LLMs
Comprehensive guide for integrating your own LLM implementation with D-ID agents. Learn how to create an OpenAI-compatible endpoint, handle authentication, optimize for production, and implement streaming responses.
Note: LLMs must stream tokens to minimize latency between turns.
Overview
Custom LLM integration enables you to:
- Use proprietary or fine-tuned models with D-ID agents
- Route requests through your own infrastructure
- Maintain full control over model selection and parameters
Example CodeYou can start from our reference implementation on GitHub.
Responsibilities
D-ID's Responsibility
- Securely store and encrypt your API keys/credentials
- Send properly formatted requests to your endpoint
- Handle streaming and non-streaming responses
- Include metadata headers for context
Your Responsibility
- Implement an OpenAI compatible endpoint
- Authenticate requests via API key or OAuth2
- Optimize for low latency (TTFT 200–500ms, under 1000ms p95)
- Scale infrastructure for production load
Usage
Custom LLM Configuration Options
| FIELD | TYPE | REQUIRED | DESCRIPTION |
|---|---|---|---|
| type | string | ✓ Yes | Authentication type: basic or oauth2 |
| url | string | ✓ Yes | Your LLM endpoint URL that receives POST requests |
| streaming | boolean | Optional | Enable streaming for production (Default: false) |
| max_messages | number | Optional | Maximum conversation history messages sent to your LLM |
| headers | object | Optional | Additional custom headers to include in requests |
LLM Behavior (Optional)
Custom LLMs support the same behavior configuration as OpenAI:
| FIELD | TYPE | DESCRIPTION |
|---|---|---|
| instructions | string | Defines what the Agent does and how it should behave |
| prompt_customization | object | Advanced prompt control (role, personality, topics_to_avoid, etc.) |
Authentication
Basic
| FIELD | TYPE | REQUIRED | DESCRIPTION |
|---|---|---|---|
| key | string | ✓ Yes | API key sent in x-api-key header |
OAuth2
| FIELD | TYPE | REQUIRED | DESCRIPTION |
|---|---|---|---|
| token_url | string | ✓ Yes | token endpoint for client credentials |
| client_id | string | ✓ Yes | client identifier |
| client_secret | string | ✓ Yes | client secret |
FAQ
Updated 4 months ago