Getting Started with flutter_local_agent_kit: Offline‑First AI for Flutter

Why Choose flutter_local_agent_kit?

The flutter_local_agent_kit (FLAK) package brings a lightweight large‑language‑model (LLM) inference engine directly onto a Flutter device. It’s ideal for scenarios where latency, privacy, or intermittent connectivity are critical, such as field‑service tools, remote‑learning apps, or personal assistants that must stay offline.

Tip: If your app targets web browsers, verify the current web support status on the package’s pub.dev page, as the library primarily targets mobile and desktop.

Installation

Add the package to your pubspec.yaml using the Flutter CLI:

flutter pub add flutter_local_agent_kit

After the dependency resolves, run flutter pub get to fetch the assets.

Preparing Model Assets

FLAK requires a TensorFlow Lite model file (e.g., mini_llm.tflite) placed in your assets/models directory. Update pubspec.yaml to include the asset:

flutter:
  assets:
    - assets/models/mini_llm.tflite

Make sure the model you choose matches the device’s memory constraints; larger models increase APK size and RAM usage.

Basic Usage – A Minimal Chat App

The following example demonstrates the quickest way to spin up a Material 3 chat UI powered by the local agent.

import 'package:flutter/material.dart';
import 'package:flutter_local_agent_kit/flutter_local_agent_kit.dart';

Future main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await FlutterLocalAgentKit.initialize(
    modelPath: 'assets/models/mini_llm.tflite',
    ragEnabled: true, // Enable Retrieval‑Augmented Generation
  );
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Local Agent Demo',
      theme: ThemeData(useMaterial3: true),
      home: const Scaffold(
        body: SafeArea(
          child: ChatScreen(), // Provided by flutter_local_agent_kit
        ),
      ),
    );
  }
}

The ChatScreen widget is part of the package and already implements a Material 3 chat interface. You can customize its appearance via standard Flutter theming.

Customizing the Chat UI

If you need a bespoke UI, the package also exposes lower‑level APIs:

• FlutterLocalAgentKit.sendPrompt(String prompt) – returns a
  CodeCopy

  Future<String>

  with the model’s response.
• FlutterLocalAgentKit.addDocument(String id, String content) – registers a document for RAG.

Below is a simple example that sends a prompt manually and displays the response in a ListView:

class ManualChat extends StatefulWidget {
  const ManualChat({super.key});
  @override
  State<ManualChat> createState() => _ManualChatState();
}

class _ManualChatState extends State<ManualChat> {
  final List<String> _messages = [];
  final TextEditingController _controller = TextEditingController();

  Future _send() async {
    final prompt = _controller.text.trim();
    if (prompt.isEmpty) return;
    setState(() => _messages.add('You: $prompt'));
    final response = await FlutterLocalAgentKit.sendPrompt(prompt);
    setState(() => _messages.add('Agent: $response'));
    _controller.clear();
  }

  @override
  Widget build(BuildContext context) {
    return Column(
      children: [
        Expanded(
          child: ListView.builder(
            itemCount: _messages.length,
            itemBuilder: (ctx, i) => Padding(
              padding: const EdgeInsets.symmetric(vertical: 4, horizontal: 8),
              child: Text(_messages[i]),
            ),
          ),
        ),
        Padding(
          padding: const EdgeInsets.all(8.0),
          child: Row(
            children: [
              Expanded(
                child: TextField(
                  controller: _controller,
                  decoration: const InputDecoration(
                    hintText: 'Ask the local agent...',
                  ),
                  onSubmitted: (_) => _send(),
                ),
              ),
              IconButton(
                icon: const Icon(Icons.send),
                onPressed: _send,
              ),
            ],
          ),
        ),
      ],
    );
  }
}

Enabling Retrieval‑Augmented Generation (RAG)

RAG lets the LLM reference external documents stored on the device. After initializing the kit with ragEnabled: true, add documents like this:

await FlutterLocalAgentKit.addDocument(
  'doc_001',
  'Flutter is Google’s UI toolkit for building natively compiled applications.',
);

When you later send a prompt that mentions “Flutter”, the model can retrieve the stored snippet to improve answer relevance.

Best Practices for RAG

• Keep document IDs short and unique.
• Chunk large texts into 200‑500 word pieces to stay within the model’s context window.
• Remove or replace outdated documents when the app updates.

Mistakes to Avoid

• Skipping asset registration: Forgetting to list the .tflite file in pubspec.yaml will cause a runtime file‑not‑found error.
• Loading the kit after
  CodeCopy

  runApp()

  : Initialization must happen before the UI builds; otherwise you’ll see “FlutterLocalAgentKit not initialized” exceptions.
• Using a model too large for the target device: Devices with < 2 GB RAM may crash when loading heavyweight models. Test on the lowest‑spec hardware you intend to support.
• Assuming web support: The current package has limited or no Web compatibility. Verify the latest status on pub.dev before targeting browsers.

Performance & Memory Tips

Tip: Call FlutterLocalAgentKit.dispose() in dispose() of a top‑level widget when you know the agent is no longer needed. This releases native buffers and reduces memory pressure.

• Prefer the smallest model that meets your quality needs (e.g., mini_llm.tflite vs. a 100 MB model).
• Enable ragEnabled only when you truly need document lookup; it adds an extra index in memory.
• Profile with devtools to monitor RAM usage during inference.

Where to Find More Information

For the most up‑to‑date API reference, examples, and troubleshooting, visit the package’s page on pub.dev:

• flutter_local_agent_kit – pub.dev
• Documentation and issue tracker are linked from the same page.

Conclusion

The flutter_local_agent_kit Flutter package empowers developers to embed AI directly into their apps, delivering instant, privacy‑first responses without relying on external servers. By following the installation steps, initializing early, and respecting device constraints, you can create robust offline assistants, document search tools, and more.

Frequently Asked Questions

Do I need an internet connection to use flutter_local_agent_kit?

No. The package runs a TensorFlow Lite LLM entirely on the device, so it works offline after the model file is bundled with the app.

Which platforms are supported?

flutter_local_agent_kit is primarily built for Android, iOS, macOS, and Windows. Web support is limited; check the latest pub.dev notes before targeting browsers.

How large is the model file and does it affect app size?

Model sizes vary. A typical “mini” model is around 30‑50 MB, which adds to the APK/IPA size. Larger models improve answer quality but increase download size and memory usage.

Can I use my own TensorFlow Lite model?

Yes. Provide the path to any compatible <code>.tflite</code> model in the <code>initialize</code> call. Ensure the model follows the expected input/output schema documented on pub.dev.

How do I free resources when the agent is no longer needed?

Call <code>FlutterLocalAgentKit.dispose()</code> (usually in the <code>dispose()</code> method of a top‑level widget) to release native buffers and reduce memory consumption.