Installation Guide

Set up your router in minutes

I want to:

Before you start

Install SyftUI from syftbox.net and add the syft-llm-router app

syft-llm-router app

https://github.com/OpenMined/syft-llm-router
Installing syft-llm-router in SyftUI

Add the syft-llm-router app to manage router creation, data indexing, and publication.

Bring your data to the network

Step 1 of 4
1

Create router

Set up your router service type

~2 minutes
  1. Click New router in the dashboard
  2. Choose router mode:
    🚀 Default (Recommended)

    Ready-to-use solution: Includes all necessary components out-of-the-box: for chat services, runs TinyLlama locally on your machine; for Search services: uses a local vector database.

    ⚙️ Custom (Advanced)

    Bring your own infrastructure: Creates only the template code for the router, where you can write custom code to plug your own vector databases, AI models, or cloud services you want to integrate.

  3. Select service type:
    • Search: Returns relevant quotes from your data
    • Chat: Local LLM that can speak to your data
  4. Router is created as a draft (not yet public)

Router creation workflow - automatically cycles through steps

2

Add data

Upload and index your documents

  1. Start your draft router
  2. Navigate to local-rag in SyftUI apps
  3. Upload and index your data

Supported formats:

.md .pdf .json .yaml .txt

Need custom formats? Reach out to OpenMined for support.
Dynamic data? Create a Syft app or local cronjob to fetch data from platforms like WordPress, Silverchair, Ghost, Medium, Slack, Notion, or any REST API-compatible platform into a folder watched by local RAG.

Add data interface

Upload and index your documents for search and chat

3

Publish

Make your router discoverable to others

  1. Proceed back to syft-llm-router, where you manage your public services
  2. Click Publish for your router
  3. Add metadata (title, description, tags)
  4. Set pricing (can start free)
  5. Confirm publication
Publish metadata

Configure metadata and pricing for your router

Step 4 of 4
4

Try it out

Test your router in the chat interface

Ready to use!
  1. Try it out in the chat interface!
  2. If you are a provider, monitor usage metrics in the dashboard
Try it out interface

Test your router with the built-in chat interface

Success!

Your router is now live and discoverable by others on the network.