When to use Openai Realtime Dart
The **Openai Realtime Dart** package brings the power of OpenAI's Realtime API to Flutter and pure Dart projects. By encapsulating the low‑level WebSocket protocol, it delivers a high‑level, stateful interface that emits typed events for audio, transcription, tool calls, and conversation updates. Developers can subscribe to these streams, react to changes, and push new user inputs without worrying about socket lifecycle, reconnection logic, or JSON parsing. The package is deliberately lightweight, exposing only the essentials needed to integrate voice‑driven assistants, live transcription, or collaborative AI agents into mobile, web, or desktop apps.
When to use this library? It shines in scenarios where latency‑sensitive, bidirectional communication with OpenAI is required—think real‑time voice assistants, interactive tutoring bots, or live captioning tools. Because the API is event‑based, you can update UI instantly as the model streams audio or text, creating a seamless conversational experience. The package fits naturally into a Flutter architecture that separates concerns: the client lives in a repository or service layer, while UI widgets consume its streams via providers, Riverpod, Bloc, or any other state‑management solution. This separation keeps your UI declarative and testable, while the client handles networking and state synchronization.
Getting started is straightforward. After adding the dependency with `flutter pub add openai_realtime_dart`, instantiate `OpenAiRealtimeClient` with your API key and optional configuration such as model name, voice settings, and custom tool definitions. The client exposes `onAudio`, `onTranscript`, `onToolCall`, and `onError` streams that you can listen to using `StreamBuilder` or your preferred state manager. Sending user audio is as simple as feeding a `Uint8List` to `client.sendAudio()`. The package also provides helpers for managing session lifecycle—`client.connect()`, `client.disconnect()`, and automatic reconnection on network interruptions—so you can focus on the conversational flow rather than socket plumbing.
Production considerations are important because the Realtime API is still in beta. Rate limits, token usage, and model availability can change, so you should implement graceful degradation (e.g., fallback to a standard HTTP completion request) and monitor the `onError` stream for quota or connectivity issues. Secure storage of the API key is mandatory; use `flutter_secure_storage` or platform‑specific keychains. Additionally, be aware that WebSocket support varies across platforms—mobile and desktop are fully supported, while web may require additional CORS configuration on the OpenAI side. Testing with mock streams is recommended to avoid hitting the live API during CI runs.
For beginners, a minimal example can be built with a `ChangeNotifier` that forwards the client’s streams to UI widgets. A simple screen with a microphone button can start the session, display live transcription in a `Text` widget, and play back the model’s audio response using the `just_audio` package. This pattern demonstrates how the client integrates with Flutter’s reactive model while keeping networking code isolated. As you grow, you can extend the client with custom tool definitions, multi‑turn dialogues, or integrate it with other back‑ends like Firebase for user authentication and analytics. In short, Openai Realtime Dart provides a robust foundation for any Flutter app that needs real‑time AI interaction, while remaining flexible enough to fit into a wide range of architectural styles.
Pros
- high‑level abstraction over WebSocket
- typed event streams
- built‑in reconnection logic
- compatible with major state‑management solutions
Watch outs
- beta API may change
- web requires CORS setup
- limited to OpenAI's realtime models
Setup notes
Add the dependency with: ``` flutter pub add openai_realtime_dart ``` Import the package: ```dart import 'package:openai_realtime_dart/openai_realtime_dart.dart'; ``` Create an instance, connect, and start listening to the provided streams. Remember to store your OpenAI API key securely and call `client.disconnect()` when the widget is disposed.
Requires Dart SDK >=2.17.0 and Flutter >=3.0.0. Works on iOS, Android, macOS, Windows, Linux, and web (subject to CORS configuration). The package relies on `dart:io` WebSocket for mobile/desktop and `html` WebSocket for web, so ensure the target platform supports WebSocket connections.
```dart
import 'package:flutter/material.dart';
import 'package:openai_realtime_dart/openai_realtime_dart.dart';
class RealtimeChatScreen extends StatefulWidget {
const RealtimeChatScreen({Key? key}) : super(key: key);
@override
State<RealtimeChatScreen> createState() => _RealtimeChatScreenState();
}
class _RealtimeChatScreenState extends State<RealtimeChatScreen> {
late final OpenAiRealtimeClient _client;
final List<String> _transcripts = [];
@override
void initState() {
super.initState();
_client = OpenAiRealtimeClient(
apiKey: const String.fromEnvironment('OPENAI_API_KEY'),
model: 'gpt-4o-realtime',
);
_client.connect();
_client.onTranscript.listen((event) {
setState(() => _transcripts.add(event.text));
});
_client.onError.listen((e) => debugPrint('Realtime error: $e'));
}
@override
void dispose() {
_client.disconnect();
super.dispose();
}
void _sendSampleAudio() async {
// Replace with real audio bytes from a microphone plugin.
final Uint8List fakeAudio = Uint8List.fromList([0, 1, 2]);
await _client.sendAudio(fakeAudio);
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(title: const Text('Realtime Chat')),
body: ListView.builder(
itemCount: _transcripts.length,
itemBuilder: (_, i) => ListTile(title: Text(_transcripts[i])),
),
floatingActionButton: FloatingActionButton(
onPressed: _sendSampleAudio,
child: const Icon(Icons.mic),
),
);
}
}
```