RAG with Built-in Tools

Create powerful AI agents that combine organizational knowledge with specialized tools like web browsing, image generation, and code interpretation

RAG (Retrieval Augmented Generation) agents with built-in tools represent the most powerful no-code agents available on the Prisme.ai platform. These agents combine access to your organization’s knowledge with specialized capabilities that extend beyond conversation, enabling them to perform complex tasks, access external information, generate media, and process data.

Understanding RAG + Tools Agents

These advanced agents integrate three key capabilities:

Foundation Models

Powerful language models providing reasoning and generation capabilities

Knowledge Retrieval

Access to your organization’s documents and information

Specialized Tools

Built-in capabilities that extend functionality beyond conversation

This combination creates versatile, capable agents that can:

• Answer questions using your internal knowledge
• Retrieve up-to-date information from the web
• Generate visual content based on requirements
• Perform data analysis and calculations
• Create structured documents and outputs
• Execute multi-step processes with diverse requirements

Available Built-in Tools

Prisme.ai provides several powerful tools that can be activated without writing code:

Web Browsing

Description: Access current information from the internet using Serper integration

Key Capabilities:

• Search for current information beyond the model’s knowledge cutoff
• Access specific websites for reference
• Verify facts and retrieve up-to-date data
• Gather content from public sources

Best For:

• Research and information gathering
• Validating time-sensitive information
• Supplementing internal knowledge with public data
• Competitive research and market intelligence

Configuration Options:

• Search domains to include/exclude
• Number of results to retrieve
• Safe search settings
• Result depth control

Image Generation

Description: Create images based on text descriptions

Key Capabilities:

• Generate custom images from textual descriptions
• Create visual content in various styles
• Produce illustrations, diagrams, and visualizations
• Export in multiple formats and resolutions

Best For:

• Content creation and marketing
• Visual explanations and instructions
• Concept visualization
• Custom illustrations for documentation

Configuration Options:

• Image style and aesthetic
• Resolution settings
• Content safety filters
• Visual style guidelines

Code Interpreter

Description: Run code to perform calculations, data analysis, and visualizations

Key Capabilities:

• Execute Python code in a secure environment
• Perform complex calculations and data analysis
• Generate charts, graphs, and visualizations
• Process and transform structured data

Best For:

• Data analysis and reporting
• Scientific and mathematical calculations
• Visual data presentation
• Custom data transformations

Configuration Options:

• Available libraries and packages
• Execution time limits
• Memory allocation
• File handling permissions

When to Use RAG + Tools Agents

These advanced agents are ideal for scenarios that require a combination of:

Knowledge Access

Situations requiring specific organizational information

Current Information

Needs for up-to-date data beyond internal documentation

Visual Content

Requirements for custom images or visualizations

Data Processing

Calculations, analysis, or data transformation tasks

Multi-capability Workflows

Processes that combine different types of tasks

Complex Problem Solving

Scenarios requiring multiple approaches and resources

Example Use Cases

Research Assistant

Purpose: Help users research topics by combining internal knowledge with current information and analysis.

Tool Utilization:

• Retrieves relevant internal research and documentation
• Searches the web for current information and trends
• Generates data visualizations to compare findings
• Creates structured research summaries

Example Interaction:

User: I need to understand the competitive landscape for cloud storage solutions, focusing on security features.

Agent: I'll help you research this topic. Let me check our internal knowledge first, then supplement with current market information.

[Accesses knowledge base for internal research on cloud storage security]

Based on our internal analysis from Q1 this year, the key security features in cloud storage include:
- End-to-end encryption
- Zero-knowledge architecture
- Multi-factor authentication
- SOC 2 compliance
- Data residency options

Let me check the current market landscape to see if there have been any recent developments.

[Uses web browsing to search for current information]

I've found some updated information. In the last three months, several major providers have introduced:
- Quantum-resistant encryption options
- Biometric authentication integration
- Enhanced ransomware protection features

Would you like me to create a visualization comparing the top competitors based on these security features?

User: Yes, please create a comparison chart.

[Uses code interpreter to generate a visual comparison]

Here's a visual comparison of the top cloud storage providers and their security features:
[Image: Comparative chart of cloud storage security features]

Would you like me to focus on any specific aspect of this competitive landscape?

Technical Support

Purpose: Provide advanced technical assistance by combining product documentation with analytical capabilities.

Tool Utilization:

• Retrieves relevant product documentation and troubleshooting guides
• Generates diagrams to explain technical concepts
• Runs code to demonstrate solutions or analyze logs
• Searches for updates or known issues

Example Interaction:

User: I'm getting a 503 error when trying to access the admin dashboard, but only when using Chrome. Firefox works fine.

Agent: I'll help you troubleshoot this issue. Let me check our knowledge base for relevant information.

[Accesses knowledge base for information on 503 errors and browser compatibility]

According to our troubleshooting guide, a 503 error specific to one browser could indicate:
1. Browser cache issues
2. Extension conflicts
3. Security settings incompatibility

Let me check if there are any known issues with Chrome specifically.

[Uses web browsing to search for recent Chrome issues]

I've found that Chrome released version 115.0.5790.110 three days ago, which has some known issues with certain web applications. Let me create a diagram to explain what might be happening.

[Uses image generation to create a technical diagram]

Here's a visualization of how the Chrome request might be failing:
[Image: Diagram showing the request flow and error point]

I recommend trying these steps:
1. Clear your Chrome cache
2. Temporarily disable extensions
3. Try using Chrome in incognito mode

If those don't work, I can help you with a more advanced troubleshooting process. Would you like to try these steps first?

Content Creator

Purpose: Assist in creating rich content by combining knowledge, research, and media generation.

Tool Utilization:

• Retrieves brand guidelines and existing content
• Searches for current trends and information
• Generates custom images and visualizations
• Analyzes content performance data

Example Interaction:

User: I need to create a blog post about sustainable manufacturing for our company website.

Agent: I'll help you create a comprehensive blog post on sustainable manufacturing. Let me start by checking our brand guidelines and existing content on this topic.

[Accesses knowledge base for brand guidelines and related content]

Based on our brand guidelines, sustainability content should:
- Focus on measurable impact
- Include specific examples from our operations
- Use a professional but conversational tone
- Incorporate our commitment to the UN Sustainable Development Goals

Let me check for current trends in sustainable manufacturing.

[Uses web browsing to research current sustainability trends]

Recent trends in sustainable manufacturing include:
- Circular economy implementation
- Carbon-neutral production targets
- AI-optimized resource utilization
- Blockchain for supply chain transparency

I'd like to suggest a structure for your blog post with these elements. I can also generate some custom images to illustrate key concepts.

[Generates outline and sample images]

Here's a suggested outline for your blog post:
[Structured blog post outline]

And here's a custom image illustrating the circular economy principle in manufacturing:
[Image: Custom illustration of circular economy in manufacturing]

Would you like me to expand any section of this outline, or would you prefer different imagery?

Financial Analyst

Purpose: Help analyze financial data by combining documentation, calculations, and visualizations.

Tool Utilization:

• Retrieves financial policies and procedures
• Runs calculations on financial data
• Creates visual representations of financial trends
• Searches for market benchmarks and comparisons

Example Interaction:

User: Can you help me analyze our Q2 revenue against projections? I've uploaded the spreadsheet.

Agent: I'll help you analyze the Q2 revenue performance against projections using the spreadsheet you've provided. Let me first check our financial analysis procedures.

[Accesses knowledge base for financial analysis guidelines]

According to our financial analysis framework, we should examine:
- Variance analysis by product line
- Key performance indicators against targets
- Seasonal adjustment factors
- Year-over-year comparison

Let me analyze the spreadsheet you've uploaded.

[Uses code interpreter to perform financial analysis]

I've analyzed the data and found several key insights:

1. Overall revenue is 3.2% below projection, primarily due to underperformance in the Enterprise product line
2. Consumer segment exceeded projections by 7.8%
3. The variance expanded from 1.1% in April to 5.4% in June, indicating a growing trend

Let me create some visualizations to help understand these patterns.

[Generates financial charts and graphs]

Here's a visualization of the quarterly performance:
[Image: Revenue vs. Projection chart]

And here's a breakdown by product line:
[Image: Product line performance comparison]

Would you like me to perform any additional analysis, such as identifying the root causes of the Enterprise segment underperformance?

Creating RAG + Tools Agents

Building these advanced agents involves a structured process:

1. Define Agent Purpose and Requirements

   Clearly specify what your agent needs to accomplish.

   Key considerations:

   • Primary use cases and scenarios
   • Required knowledge sources
   • Necessary tools and capabilities
   • User expectations and needs
   • Success criteria and metrics
2. Prepare Knowledge Resources

   Assemble and organize the information your agent needs.

   Activities include:

   • Document collection and curation
   • Content organization and structuring
   • Metadata creation for improved retrieval
   • Validation of information accuracy
   • Knowledge base configuration
3. Select and Configure Tools

   Determine which built-in tools your agent requires.

   For each tool:

   • Enable required capabilities
   • Configure settings and parameters
   • Define usage boundaries and limitations
   • Set up authentication if required
   • Test individual tool functionality
4. Create System Instructions

   Develop comprehensive guidance for your agent.

   Key components:

   • Agent role and purpose definition
   • Knowledge utilization guidelines
   • Tool selection and usage criteria
   • Response formatting requirements
   • Error handling and limitations
5. Develop Tool Usage Instructions

   Provide specific guidance for when and how to use each tool.

   Include for each tool:

   • When to use (clear criteria)
   • How to use effectively
   • Result interpretation guidance
   • Error handling approaches
   • User communication during tool use
6. Test and Refine

   Validate and iteratively improve your agent.

   Testing approach:

   • Scenario-based evaluation
   • Tool usage verification
   • Knowledge retrieval assessment
   • Edge case testing
   • Real user feedback collection

Implementation in AI Knowledge

Creating a RAG + Tools agent in Prisme.ai’s AI Knowledge is straightforward:

1. Create New Agent

Start by creating a new agent in the AI Store.

Steps:

1. Navigate to AI Knowledge
2. Click “Create New Agent”
3. Provide name and basic description

2. Connect Knowledge Base

Link your agent to relevant knowledge sources.

Configuration options:

• Select existing knowledge bases
• Configure retrieval settings
• Set relevance thresholds
• Configure enhance User Query and Post Query if needed

3. Enable Built-in Tools

Activate and configure the tools your agent needs.

For each tool:

• Toggle tool activation
• Configure enabling manual tool selection
• Test individual functionality

4. Create System Instructions

Define comprehensive guidance for your agent.

Key elements to include:

• Agent purpose and identity
• Knowledge base usage guidelines
• Tool selection criteria
• Response formatting requirements
• Error handling procedures
• Limitations and constraints

5. Test and Deploy

Validate and make your agent available to users.

Final steps:

• Run comprehensive tests via Auto Test Feature
• Make adjustments based on results
• Define access permissions
• Publish to intended audience
• Set up monitoring and feedback collection

Tool-Specific Instructions

To maximize the effectiveness of each built-in tool, include specific guidance in your system instructions:

Web Browsing Tool Instructions

When to Use:

Use the web browsing tool when:
1. The user asks for current information beyond our internal documentation
2. You need to verify or update potentially outdated internal information
3. The user explicitly requests external research
4. You need specific details not available in our knowledge base

Always tell the user when you're using web browsing and cite the sources of information retrieved.

How to Use Effectively:

When using the web browsing tool:
1. Form specific, focused search queries
2. Prioritize authoritative sources
3. Cross-reference information from multiple sources when possible
4. Summarize key findings rather than providing entire web pages
5. Extract only relevant information to the user's query
6. Always cite sources with title and URL

Example Prompt Section:

For web browsing:
- Formulate clear, specific search queries focused on the user's information need
- Prioritize results from authoritative sources (.gov, .edu, established industry sites)
- Synthesize information from multiple sources rather than relying on a single result
- Always cite your sources by including "According to [source name] ([URL])"
- When information conflicts between sources, acknowledge this and explain which source seems more reliable and why
- If search results are insufficient, try reformulating your query or tell the user what information you were unable to find

Image Generation Tool Instructions

When to Use:

Use the image generation tool when:
1. The user explicitly requests a visual representation
2. A complex concept would benefit from visual explanation
3. Creating diagrams, charts, or illustrations would enhance understanding
4. Visual examples would supplement textual information

Always confirm with the user before generating images unless explicitly requested.

How to Use Effectively:

When generating images:
1. Create detailed, specific prompts for the image generator
2. Align visual style with our brand guidelines (professional, clean, modern)
3. Ensure images are accessible and clear in their meaning
4. Generate appropriate alternatives if the user is not satisfied
5. Focus on informative rather than decorative images

Example Prompt Section:

For image generation:
- Create detailed, specific prompts that describe both content and style
- Follow our brand style guidelines: clean, professional, with our color palette (blue #1A73E8, gray #5F6368, green #34A853)
- For diagrams and charts, prioritize clarity and readability over aesthetic complexity
- Ensure generated images are appropriate and professional for business use
- Always ask the user if they'd like modifications to the generated image
- If the user requests a specific style, artistic approach, or reference, incorporate these elements explicitly in your prompt

Code Interpreter Tool Instructions

When to Use:

Use the code interpreter tool when:
1. Performing calculations or data analysis is required
2. The user has provided data that needs processing
3. Creating data visualizations would be valuable
4. Converting between data formats is needed
5. The user explicitly requests analysis or computation

Explain your analytical approach before executing code.

How to Use Effectively:

When using code interpreter:
1. Write clean, well-commented code for transparency
2. Explain your analysis approach in non-technical terms
3. Present results with clear interpretations
4. Create visualizations that highlight key insights
5. Handle errors gracefully with meaningful explanations

Example Prompt Section:

For code interpretation and data analysis:
- Write clean, well-documented Python code with comments explaining key steps
- Before executing code, explain your analytical approach in non-technical terms
- When analyzing data, first examine its structure and quality before drawing conclusions
- Create informative visualizations with clear titles, labeled axes, and appropriate chart types
- Always interpret results in business terms, focusing on insights rather than just numbers
- When errors occur, provide clear explanations and alternative approaches
- For data visualizations, use our brand colors (#1A73E8 blue, #34A853 green, #FBBC04 yellow, #EA4335 red)

Best Practices for RAG + Tools Agents

Clear Tool Selection Logic

Provide explicit criteria for when to use each tool

Example:

Use web browsing when:
- Information needed is likely to change frequently
- The query explicitly mentions current events
- Internal knowledge is insufficient or outdated

Use code interpreter when:
- Calculations are required
- Data analysis is needed
- Visualization would enhance understanding

User Communication During Tool Use

Keep users informed about tool usage and processes

Example:

Before using a tool:
"I'll need to search the web to find the most current information on this topic."

During tool use:
"I'm analyzing the data you provided. This will take a moment."

After tool use:
"Based on the web search results from [sources], I found that..."

Integrate Knowledge and Tool Outputs

Combine information from different sources coherently

Example:

First present information from internal knowledge:
"According to our documentation, our current process is..."

Then supplement with web information:
"Based on recent industry updates from [source], there are new approaches..."

Finally, provide analysis:
"Analyzing these approaches against your requirements shows that..."

Handle Tool Failures Gracefully

Provide alternative approaches when tools don’t work as expected

Example:

"I tried to search for current information on this topic, but couldn't find specific results. Instead, let me:
1. Share what we know from our internal documentation
2. Suggest some alternative approaches
3. Recommend reliable sources you might check directly"

Troubleshooting Common Issues

Knowledge Retrieval Problems

Symptoms:

• Agent fails to find relevant information
• Responses ignore available knowledge
• Retrieved information is incorrect or outdated

Solutions:

• Verify knowledge base configuration and content
• Improve document chunking and organization
• Enhance retrieval instructions in system prompt
• Add more specific examples of knowledge utilization
• Test with focused queries to validate retrieval

Inappropriate Tool Selection

Symptoms:

• Agent uses tools when unnecessary
• Agent fails to use tools when needed
• Consistent selection of suboptimal tools

Solutions:

• Refine tool selection criteria in system instructions
• Provide explicit examples of when to use each tool
• Add clear guidance for choosing between similar tools
• Include decision tree for tool selection logic
• Test with scenarios that require tool selection decisions

Poor Tool Usage

Symptoms:

• Ineffective web searches returning irrelevant results
• Low-quality image generation
• Code that errors or produces incorrect results

Solutions:

• Improve tool-specific instructions and examples
• Add more detailed guidance on effective tool usage
• Provide templates for common tool operations
• Include error handling instructions for each tool
• Test each tool independently with focused scenarios

Response Integration Issues

Symptoms:

• Disjointed responses that fail to synthesize information
• Tool outputs presented without context or interpretation
• Failure to connect retrieved knowledge with tool results

Solutions:

• Enhance instructions for information synthesis
• Add examples of well-integrated responses
• Include specific guidance on combining knowledge and tool outputs
• Provide templates for presenting multi-source information
• Test with scenarios requiring information integration

Performance Issues

Symptoms:

• Slow response times
• Tool execution timeouts
• Memory errors during complex operations

Solutions:

• Optimize tool usage guidelines for efficiency
• Add instructions for breaking complex tasks into manageable steps
• Provide fallback options for resource-intensive operations
• Include guidelines for managing large datasets
• Test with progressively complex scenarios to identify limits

Advanced Configuration Techniques

For organizations seeking to maximize the effectiveness of their RAG + Tools agents, consider these advanced approaches:

Tool Chaining

Configure agents to use tools in sequence for complex workflows

Implementation:

For market analysis requests:
1. First use knowledge retrieval to access internal market research
2. Then use web browsing to gather current market trends
3. Next use code interpreter to analyze combined data
4. Finally use image generation to create visualizations of findings

Tool chaining enables sophisticated multi-step processes that combine different capabilities.

Contextual Tool Configuration

Adapt tool behavior based on user type or request context

Implementation:

When supporting technical users:
- Include more technical detail in responses
- Show code snippets and raw data when relevant
- Use technical terminology in explanations

When supporting business users:
- Focus on insights rather than technical details
- Emphasize visualizations over raw data
- Use business terminology in explanations

Contextual configuration improves relevance for different user groups.

Hybrid Knowledge Approach

Strategically combine internal knowledge with web information

Implementation:

For product information:
- Always use internal knowledge for specifications and features
- Use web browsing only for competitor comparisons
- Supplement with internal competitive analysis when available
- Clearly distinguish between internal and external information

This approach ensures accurate internal information while benefiting from external context.

Progressive Disclosure

Structure responses to present essential information first

Implementation:

When providing analysis results:
1. Start with key findings and recommendations
2. Follow with supporting evidence and context
3. Offer detailed analysis for those who want to explore further
4. Provide visualization options that users can request

Progressive disclosure improves usability by preventing information overload.

Enterprise Integration Considerations

Governance and Compliance

When deploying RAG + Tools agents in enterprise environments, consider:

• Usage Policies: Define clear guidelines for appropriate tool usage
• Content Review: Establish processes for reviewing agent outputs
• Audit Trails: Implement logging for tool usage and information sources
• Access Controls: Limit specific tools to appropriate user groups
• Compliance Checks: Ensure tool outputs meet regulatory requirements

Implementation Example:

This agent's web browsing capability is restricted to approved domains and automatically logs all external information sources. Image generation follows brand guidelines and content policies. All data processing complies with our data handling procedures and privacy requirements.

Security Considerations

Protect sensitive information when using tools:

• Data Handling: Establish clear rules for handling sensitive information
• Tool Boundaries: Define what data can be processed by each tool
• Output Scrubbing: Implement checks for sensitive information in outputs
• Authentication: Manage service credentials securely
• User Permissions: Align tool access with user authorization levels

Implementation Example:

Code interpreter will not process files containing PII or confidential information. Web browsing is restricted to public information only. All generated content is screened for sensitive information before delivery to users.

Scalability Planning

Prepare for widespread adoption and usage:

• Resource Management: Monitor and optimize tool usage patterns
• Usage Quotas: Implement appropriate limits for resource-intensive tools
• Caching Strategies: Cache common tool results when appropriate
• Load Balancing: Distribute tool processing across resources
• Performance Monitoring: Track response times and resource utilization

Implementation Example:

This agent implements smart caching for web search results and common calculations. Image generation is limited to 10 images per user per day. Resource-intensive code execution has a 60-second timeout with graceful degradation to simpler analysis for complex requests.

Next Steps

Ready to create your own RAG + Tools agent? Continue with these resources:

AI Store Guide

Learn how to use the AI Store platform effectively

Testing Best Practices

Learn how to thoroughly test and validate your agents