Using Retrieval-Augmented Generation (RAG)

Enhance your agents with knowledge from external sources using RAG. This allows your agents to answer questions and complete tasks based on your own documents and data.

Overview

RAG (Retrieval-Augmented Generation) allows your agents to access and reference specific information from your documents, databases, and knowledge sources. When a user asks a question, the agent can search through your uploaded content to provide accurate, contextual responses based on your actual data.

Prerequisites

Before using RAG, ensure you have:

1. Database and OpenAI API configured - RAG requires a database for storing embeddings and an OpenAI API key for generating embeddings
2. Agent with searchKnowledgeBase tool enabled - This is the critical step most users miss

Enable the searchKnowledgeBase Tool

IMPORTANT: You must enable the searchKnowledgeBase tool for your agent to access the knowledge base.

Via Admin UI (Recommended)

1. Navigate to the Admin Console → Agents
2. Select or create your agent
3. Go to the Tools section
4. Enable the searchKnowledgeBase tool
5. Configure RAG sources in the Knowledge Base section
6. Save your changes

Via Configuration File

Alternatively, manually edit your agent’s config.json:

{
  "toolsAllEnabled": false,
  "toolsEnabled": ["searchKnowledgeBase", "otherTools..."],
  "ragAllEnabled": true,
  "ragEnabled": []
}

Without enabling this tool, your agent cannot access the knowledge base, even if RAG sources are configured.

Setting Up Knowledge Sources

Creating Resource Groups

Resource groups are collections of related documents that you can selectively enable for different agents:

1. Navigate to the Admin Console → Knowledge Base
2. Click “Create Resource Group”
3. Fill in the details:
   • Name: company-policies
   • Description: Employee handbook and company policies
4. Save the resource group

Uploading Documents

Upload documents to your resource groups through the Admin Console:

1. Go to your Resource Group
2. Click “Upload Documents”
3. Select your files (drag & drop or file picker)
4. Wait for processing to complete

Supported file types:

• PDF documents (.pdf)
• Text files (.txt)
• Markdown files (.md, .mdx)
• Word documents (.docx)
• CSV files (.csv)
• JSON files (.json)

Processing Pipeline

When you upload documents:

1. Text Extraction: Content is extracted from files
2. Chunking: Documents are split into manageable chunks
3. Embedding Generation: Each chunk is converted to vector embeddings using OpenAI’s text-embedding-3-small
4. Storage: Embeddings and metadata are stored in your database
5. Indexing: Content becomes searchable by your agents

Configuring Agents for RAG

Option 1: Enable All Knowledge Sources

Allow your agent to access all available knowledge:

{
  "ragAllEnabled": true,
  "ragEnabled": [],
  "toolsEnabled": ["searchKnowledgeBase"]
}

Option 2: Selective Knowledge Access

Limit your agent to specific resource groups:

{
  "ragAllEnabled": false,
  "ragEnabled": ["company-policies", "technical-docs"],
  "toolsEnabled": ["searchKnowledgeBase"]
}

Real-World Examples

HR Assistant with Company Policies

{
  "id": "hr-assistant-001",
  "name": "HR Assistant",
  "chatDisplayName": "HR Bot",
  "description": "Helps employees understand company policies and procedures",
  "prompt": "You are an HR assistant. Use the searchKnowledgeBase tool to find relevant company policies and procedures when answering employee questions. Always cite the specific source when providing policy information.",
  "toolsEnabled": ["searchKnowledgeBase"],
  "ragAllEnabled": false,
  "ragEnabled": ["employee-handbook", "benefits-guide", "hr-policies"]
}

Technical Support Agent

{
  "id": "tech-support-001",
  "name": "Technical Support",
  "chatDisplayName": "TechBot",
  "description": "Provides technical support using product documentation",
  "prompt": "You are a technical support agent. Search the knowledge base for relevant troubleshooting guides, product documentation, and known solutions before responding to technical issues.",
  "toolsEnabled": ["searchKnowledgeBase", "createTicket"],
  "ragAllEnabled": false,
  "ragEnabled": ["product-docs", "troubleshooting-guides", "api-docs"]
}

How RAG Search Works

Automatic Search Behavior

The searchKnowledgeBase tool is designed to be used automatically by your agent when:

• Users ask questions that might reference specific information
• The agent needs context to provide accurate answers
• Users mention documents, policies, or specific topics
• The agent is unsure about factual information

Search Process

1. Query Analysis: User’s question is analyzed and reformulated as a search query
2. Embedding Generation: Query is converted to a vector embedding
3. Similarity Search: System finds the most relevant content chunks using cosine similarity
4. Relevance Filtering: Only content with similarity > 0.5 is returned
5. Agent Integration: Retrieved content is provided to the agent for response generation

Search Results

Each search returns up to 4 most relevant content chunks with:

{
  "content": "The actual text content from your documents",
  "similarity": 0.85,
  "source": {
    "filename": "employee-handbook.pdf",
    "fileType": "pdf",
    "metadata": {
      "page": 15,
      "section": "Vacation Policy"
    }
  }
}

Best Practices

1. Document Organization

Structure your knowledge base logically:

Resource Groups:
├── company-policies/
│   ├── employee-handbook.pdf
│   ├── code-of-conduct.pdf
│   └── benefits-guide.pdf
├── technical-docs/
│   ├── api-documentation.md
│   ├── troubleshooting-guide.pdf
│   └── installation-manual.pdf
└── procedures/
    ├── onboarding-checklist.pdf
    ├── expense-policy.pdf
    └── security-guidelines.pdf

2. Agent Prompt Engineering

Include RAG usage instructions in your agent prompts:

{
  "prompt": "You are a customer service agent. When users ask questions about policies, products, or procedures, always use the searchKnowledgeBase tool first to find the most current and accurate information. Cite your sources and provide specific page numbers or section references when available."
}

3. Content Quality

Ensure high-quality source material:

• Clear Structure: Use headings, bullet points, and clear formatting
• Current Information: Keep documents up-to-date
• Comprehensive Coverage: Include all relevant topics
• Consistent Terminology: Use consistent language across documents

4. Testing RAG Performance

Test your knowledge base regularly:

1. Ask specific questions about content you know exists
2. Test edge cases and ambiguous queries
3. Verify accuracy of retrieved information
4. Check source attribution in agent responses

5. Monitoring and Optimization

Monitor RAG performance:

• Search Success Rate: How often relevant content is found
• Response Accuracy: Quality of agent answers
• User Satisfaction: Feedback on knowledge-based responses
• Content Gaps: Topics where no relevant content is found

Troubleshooting

Agent Not Using Knowledge Base

Check these common issues:

1. Tool Not Enabled: Ensure searchKnowledgeBase is in toolsEnabled array
2. No RAG Access: Verify ragAllEnabled: true or specific sources in ragEnabled
3. Empty Knowledge Base: Confirm documents are uploaded and processed
4. Permissions: Check agent has access to the resource groups

Poor Search Results

Improve search quality:

1. Content Quality: Ensure documents are well-structured and readable
2. Query Refinement: Test different ways of asking questions
3. Chunk Size: Consider document chunking strategy
4. Similarity Threshold: Adjust if needed (default is 0.5)

Performance Issues

Optimize for speed:

1. Selective Access: Use specific ragEnabled instead of ragAllEnabled
2. Content Volume: Monitor total content size
3. Database Performance: Ensure proper indexing
4. Embedding Cache: Embeddings are cached for efficiency

Advanced Configuration

Custom Search Behavior

For advanced use cases, you can extend the search functionality by modifying the searchKnowledgeBase tool or creating custom tools that integrate with the embedding system.

Integration with External Systems

RAG can be combined with external APIs and services to provide comprehensive information access:

{
  "toolsEnabled": [
    "searchKnowledgeBase",
    "searchExternalAPI",
    "checkLiveData"
  ]
}

Multi-Language Support

The embedding system supports multiple languages. Upload documents in different languages and the search will work across language boundaries to some extent.

Summary

RAG transforms your agents from general AI assistants into knowledgeable experts on your specific domain. By properly configuring the searchKnowledgeBase tool and organizing your knowledge sources, you can create agents that provide accurate, contextual responses based on your actual data and documentation.

Remember: Always enable the searchKnowledgeBase tool in your agent configuration - this is the key that unlocks your knowledge base for your agents.