Mastering AI Agents in Flutter with flutter_agent_harness

What is Flutter Agent Harness?

Flutter Agent Harness (flutter_agent_harness) is a lightweight, cross‑platform library that abstracts the agent loop, tool execution, and session persistence for AI‑driven features. It lets you focus on UI and business logic while the harness streams partial responses, manages context, and stores conversation state on mobile, web, and desktop.

When to Choose flutter_agent_harness

• Building chat‑bot interfaces, voice assistants, or any feature that requires multi‑turn conversations.
• Needing real‑time streaming of AI responses to reduce perceived latency.
• Wanting built‑in tool adapters so the agent can call external services (e.g., fetch data, run calculations) without writing boilerplate.
• Requiring automatic session persistence across app restarts.

Installation

flutter pub add flutter_agent_harness

After the command finishes, run flutter pub get to fetch the package.

Getting Started: A Minimal Chat Bot

The example below demonstrates a complete, runnable Flutter app that connects to OpenAI, creates an AgentSession, and streams chat messages to the UI.

import 'package:flutter/material.dart';
import 'package:flutter_agent_harness/flutter_agent_harness.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  // 1️⃣ Configure a provider – replace with your own API key.
  final provider = OpenAiProvider(apiKey: 'YOUR_API_KEY');

  // 2️⃣ Create a session. You can attach tool adapters here if needed.
  final session = AgentSession(provider: provider);

  runApp(MyApp(session: session));
}

class MyApp extends StatelessWidget {
  final AgentSession session;
  const MyApp({Key? key, required this.session}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Agent Harness Demo',
      home: Scaffold(
        appBar: AppBar(title: const Text('AI Agent Demo')),
        body: ChatScreen(session: session),
      ),
    );
  }
}

class ChatScreen extends StatefulWidget {
  final AgentSession session;
  const ChatScreen({Key? key, required this.session}) : super(key: key);

  @override
  _ChatScreenState createState() => _ChatScreenState();
}

class _ChatScreenState extends State {
  final TextEditingController _controller = TextEditingController();
  final List _messages = [];
  bool _isSending = false;

  Future _send() async {
    final prompt = _controller.text.trim();
    if (prompt.isEmpty) return;

    setState(() {
      _messages.add('You: $prompt');
      _isSending = true;
    });
    _controller.clear();

    // 3️⃣ Use the session to send a prompt. The response is streamed.
    final stream = widget.session.sendPrompt(prompt);
    final StringBuffer buffer = StringBuffer();
    await for (final chunk in stream) {
      buffer.write(chunk);
      setState(() {
        // Show the partial response while it arrives.
        if (_messages.isNotEmpty && _messages.last.startsWith('Agent:')) {
          _messages[_messages.length - 1] = 'Agent: ${buffer.toString()}';
        } else {
          _messages.add('Agent: ${buffer.toString()}');
        }
      });
    }

    setState(() => _isSending = false);
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children: [
        Expanded(
          child: ListView.builder(
            padding: const EdgeInsets.all(12),
            itemCount: _messages.length,
            itemBuilder: (context, index) => Text(_messages[index]),
          ),
        ),
        if (_isSending)
          const LinearProgressIndicator(minHeight: 2),
        Padding(
          padding: const EdgeInsets.symmetric(horizontal: 8, vertical: 4),
          child: Row(
            children: [
              Expanded(
                child: TextField(
                  controller: _controller,
                  onSubmitted: (_) => _send(),
                  decoration: const InputDecoration(
                    hintText: 'Ask the agent...',
                  ),
                ),
              ),
              IconButton(
                icon: const Icon(Icons.send),
                onPressed: _isSending ? null : _send,
              ),
            ],
          ),
        ),
      ],
    );
  }
}

This snippet showcases the three core steps:

• Provider configuration – choose any streaming‑compatible LLM provider supported by the package.
• Session creation – the AgentSession holds conversation state and optional tool adapters.
• Prompt sending – session.sendPrompt() returns a Stream that you can render incrementally.

Key Concepts to Remember

• Streaming providers: The harness works best with providers that support token‑by‑token streaming (e.g., OpenAI's gpt‑4o‑mini with stream: true).
• Tool adapters: If your agent needs to call external APIs, register a ToolAdapter when constructing the session.
• Session persistence: By default, the harness stores the conversation in local storage. Verify the storage location for each platform (SharedPreferences on mobile, IndexedDB on web, etc.).

Best Practices & Common Pitfalls

Tip: Keep the API key out of source control. Use flutter_dotenv or platform‑specific secret management to inject the key at runtime.

• Secure storage: The session cache may contain sensitive user prompts. Consider encrypting the storage layer if you handle personal data.
• Bandwidth awareness: Streaming can increase network usage on limited connections. Provide a user setting to toggle streaming on/off.
• Isolate management: Tool execution runs in the same Dart isolate by default. For heavy computations, off‑load work to a background isolate to keep the UI responsive.
• Token budgeting: The harness automatically compacts context, but very long conversations can still exceed model limits. Periodically truncate or summarize older messages.

Where to Find More Information

• Package page: flutter_agent_harness on pub.dev
• Full API docs: pub.dev documentation
• GitHub repository (if available) for issue tracking and community examples.

Next Steps

After the basic chat works, experiment with:

• Adding custom ToolAdapter implementations to fetch weather data, perform calculations, or call your backend.
• Persisting session IDs across user accounts for personalized assistants.
• Integrating voice input/output using speech_to_text and flutter_tts alongside the harness.

Frequently Asked Questions

Do I need an OpenAI account to use flutter_agent_harness?

The harness itself is provider‑agnostic, but the example uses OpenAI. You can plug in any streaming‑compatible LLM provider that implements the required interface.

How does session persistence work on web?

On web, the package stores the conversation JSON in IndexedDB. Verify the size limits for your use case and consider manual cleanup if needed.

Can I run tool adapters in a background isolate?

Yes. By default tools run in the main isolate, but you can create a custom <code>ToolAdapter</code> that spawns a separate isolate for heavy work.

Is the package compatible with Flutter 3.24 and later?

flutter_agent_harness targets the stable Flutter SDK. Check the pub.dev page for the exact SDK constraints and update accordingly.

What should I do if streaming causes high data usage?

Provide a user setting to disable streaming and fall back to a single‑response mode, or limit the maximum token count per request.