> ## Documentation Index
> Fetch the complete documentation index at: https://prismeai-legacy.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Analytics
> Measuring and optimizing your AI Knowledge agents performance and resource usage
AI Knowledge Analytics provides comprehensive visibility into your AI agents' performance, usage patterns, and resource consumption. These powerful dashboards help you optimize costs, identify improvement opportunities, and ensure your AI solutions deliver maximum value to your organization.
## Key Analytics Features
Track conversations, users, and document generation across all agents
Monitor input and output token usage with detailed breakdowns
Track expenditure with detailed cost attribution by agent and model
Identify usage patterns and performance changes over time
Compare effectiveness and efficiency across your AI agent portfolio
Analyze data across flexible time periods from 1 day to 12 months
## Analytics Dashboards
AI Knowledge Analytics offers multiple dashboards to help you understand different aspects of your AI deployment.
**The Global Analytics section** provides administrators a comprehensive view of platform-wide metrics, divided into four main areas: Dashboard, Costs, Usages, and Carbon Footprint.
### Main Dashboard for Admins
* **Total Users, Sessions, and Added Documents**: Concerning interactions with existing agents, as well as agents created during the selected period (default is 7 days).
* **Generated Responses, Global Token Count, and Estimated Cost in USD (\$)**: Based on the cost per million tokens of each model specified under the AI Knowledge configuration.
* **Classification on Number of Tokens and Messages per Agent**: As well as a Number of Tokens per day per Agent.
* **Charts**: Displaying aggregations on the number of messages, users, and tokens per day per agent.
* **Model-Specific Performances**: Such as tokens and messages per model and the delay to the first token of response per model in milliseconds.
Each metric includes an evolution value, showing the change over the selected time period compared to the previous equivalent period (e.g., comparing the last 7 days to the 7 days before).
Additionally, this section highlights the most frequently referenced documents, detailing the total number of citations, along with daily aggregates of answers, users, and tokens for the agent.
### Cost Dashboard for Admins
The Cost Dashboard provides insights into the financial aspects of AI operations:
* **Estimated Cost in USD (\$)**: Displayed in a main card, showing the evolution compared to the previous selected period.
* **Charts**: Visualize costs in various dimensions:
* Cost Per Agent
* Cost Per Model
* Cost Per Provider
* Cost Per Feature
* **Aggregations**: Line charts display cost per model and per agent.
The data displayed here can be filtered by model to specify it.
**The Usage Analytics dashboard** delves into the technical aspects of agent usage, presenting detailed statistics.
### Key Metrics
* **Average Number of Requests per User**: Provides total, minimum, and maximum requests made by individual users during interactions with the agent.
* **Average Number of Conversations per User**: Shows total, minimum, and maximum conversations initiated by each user.
* **Average Number of Requests per Conversation**: Details total, minimum, and maximum requests made in a single conversation.
* **Number of Dropouts**: Represents users who have not reused the agent since the previous corresponding period.
Token consumption is also analyzed:
* **Total Tokens**: Total tokens expended during interactions with the agent.
* **Input Tokens**: Tokens from users' requests.
* **Output Tokens**: Tokens from the agent's processing and responses.
Charts are available to visualize:
* Number of requests per day.
* Average number of requests per day of the week.
* Most popular tools.
Data is filtered by default to include all tools or Features used within the product. Current Features include:
* **genericQuery**: Standard query to an AI model for an answer.
* **toolRouting**: Request for an LLM to determine the tool to use (*Deprecated as of 14-10-2025*).
* **toolCalling**: Request from the LLM for tool execution results.
* **vectorizeDocument**: Call to an embedding for document vectorization.
* **contextualCompressWithLLM**: Queries to an LLM for context compression.
* **postLLMTasks**: Queries to the Model after receiving the initial answer.
* **rephraseUserQuery**: Query to an LLM to reformulate the user's question.
* **chat-completion**: Queries to the chat-completion endpoint, accessible via API.
* **enhanceQuery**: Queries to enhance the user's question before using prompts and documents.
* **Custom filters**: Any other treatments introduced and injected into the database.
By default, the displayed data is unfiltered.
**The Carbon Footprint** dashboard calculates environmental impact using methods from the [Ecologits Calculator](https://ecologits.ai).
### Key reference factors include:
* **Energy Consumed**: The electrical energy required to power and operate the model, including servers and IT infrastructure. This is crucial due to its environmental impact and associated costs.
* **Global Warming Potential (GWP)**: Measures the relative impact of a greenhouse gas on global warming compared to CO₂ over 100 years, allowing for comparison of different gases' effects on the climate.
* **GPU Energy**: Energy consumed by GPUs in kWh.
* **Servers Energy**: Energy consumed by servers in kWh.
* **Power Usage Effectiveness (PUE)**: An indicator of data center energy efficiency, important for sustainable IT.
* **Emission Factor**: The carbon footprint generated by using a large-scale language model, measuring CO₂ emissions per unit of computation or query processed. This KPI is essential for tracking and reducing AI-related environmental impact.
The dashboard also features a chart aggregating average emissions per day.
You can find **Agent specific analytics** by going to the Analytics section of an specific AI Knowledge project.
## General Statistics Dashboard
The General Statistics dashboard offers a snapshot of key metrics related to agent interactions:
* **Generated Answers**: The total number of responses generated by the agent.
* **Users**: The number of individuals who have interacted with the agent.
* **Sessions**: The number of sessions initiated for potential interaction with the agent.
* **Added Documents**: The number of documents added to the AI Knowledge project related to the agent.
* **User Satisfaction**: Feedback from users, categorized as positive or negative.
Each metric includes a comparison with the previous period to show trends over time. This section also highlights the most frequently referenced documents and provides daily aggregates of answers, users, and tokens.
## Usage Analysis Dashboard
The Usage Analysis dashboard provides detailed insights into how agents are used:
* **Requests per User**: Average, minimum, and maximum number of requests made by users.
* **Conversations per User**: Average, minimum, and maximum number of conversations initiated by users.
* **Requests per Conversation**: Average, minimum, and maximum number of requests within a single conversation.
* **Dropouts**: Users who have not returned to use the agent after the previous corresponding period.
This dashboard also analyzes token consumption, including total, input, and output tokens. Visual charts display requests per day, average requests per day of the week, and the most popular tools used.
## Carbon Footprint Dashboard
The Carbon Footprint dashboard assesses the environmental impact of using AI models:
* **Energy Consumption**: The electrical energy required to power and operate the models.
* **Global Warming Potential (GWP)**: The impact of greenhouse gases compared to CO₂.
* **GPU and Server Energy**: Energy consumed by GPUs and servers.
* **Power Usage Effectiveness (PUE)**: A measure of data center energy efficiency.
* **Emission Factor**: The carbon footprint per unit of computation.
Charts in this section aggregate average emissions per day.
## Using Analytics Effectively
Navigate to the Analytics section from your AI Knowledge dashboard.
Choose your desired timeframe for analysis using the date selectors.
Options include standard periods (1 day, 7 days, 30 days, 12 months) or custom date ranges.
Examine the main performance indicators for your agents.
Pay special attention to significant changes or trends in usage and costs.
Click on individual agents to see detailed performance metrics.
Compare agents to identify best practices and improvement opportunities.
## Best Practices for Analytics
Schedule weekly or monthly analytics reviews to track performance trends
Compare similar agents to establish performance benchmarks
Identify and optimize high token consumption scenarios
Connect analytics data with user feedback for deeper insights
Use analytics to allocate AI costs to appropriate departments
Implement regular optimizations based on analytics insights
## Token Optimization Strategies
Based on analytics insights, consider these strategies to optimize token usage and costs:
Streamline knowledge bases to include only the most relevant information.
Refine system prompts and instructions to be more efficient.
Choose the most cost-effective model for each use case.
Optimize how much context is included in each interaction.
## Custom usage analytics
Both LLM and embeddings usage are tracked by `usage` events, persisted with [`an aggPayload custom mapping`](/products/ai-builder/advanced-topics#implementing-event-mapping) to enable numeric aggregations in Elasticsearch/Opensearch requests.
**`Example usage :`**
```json theme={null}
{
"type": "usage",
"aggPayload": {
"channel": "api",
"api": "completions",
"tool": "genericQuery",
"usage": {
"firstTokenDuration": 804,
"completion_tokens": 92,
"prompt_tokens": 2515,
"total_tokens": 2607,
"duration": 3953,
"cost": 0.00720749999999999951
},
"projectId": "project id",
"model": "gpt-4o",
"provider": "openai",
"messageId": "message id",
"userId": "user id",
"finishReasons": [
"stop"
]
}
}
```
**Example ES/OS aggregations :**
```json Search request body expandable theme={null}
{
"size": 0,
"query": {
"bool": {
"must": {
"range": {
"@timestamp": {
"gte": "2025-04-09T14:27:41.684Z",
"lt": "2025-04-16T14:27:41.684Z"
}
}
},
"filter": [
{
"terms": {
"type": [
"usage"
]
}
},
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"term": {
"payload.api": "embeddings"
}
}
]
}
}
]
}
},
{
"terms": {
"payload.projectId": [
""
]
}
}
]
}
},
"aggs": {
"totalUsers": {
"cardinality": {
"field": "aggPayload.userId"
}
},
"totalMessages": {
"filter": {
"term": {
"type": "usage"
}
}
},
"tokens": {
"sum": {
"field": "aggPayload.usage.total_tokens"
}
},
"completionTokens": {
"sum": {
"field": "aggPayload.usage.completion_tokens"
}
},
"promptTokens": {
"sum": {
"field": "aggPayload.usage.prompt_tokens"
}
}
}
}
```
## Next Steps
Explore more detailed guides for AI Knowledge Analytics:
Accessing analytics data programmatically
# Overview of Dashboards
This document provides an introduction to the three main dashboards available in our platform: General Statistics, Usage Analysis, and Carbon Footprint. These dashboards are designed to offer insights into both specific agent activities and overall platform performance.
## General Statistics Dashboard
The General Statistics dashboard offers a snapshot of key metrics related to agent interactions:
* **Generated Answers**: The total number of responses generated by the agent.
* **Users**: The number of individuals who have interacted with the agent.
* **Sessions**: The number of sessions initiated for potential interaction with the agent.
* **Added Documents**: The number of documents added to the AI Knowledge project related to the agent.
* **User Satisfaction**: Feedback from users, categorized as positive or negative.
Each metric includes a comparison with the previous period to show trends over time. This section also highlights the most frequently referenced documents and provides daily aggregates of answers, users, and tokens.
## Usage Analysis Dashboard
The Usage Analysis dashboard provides detailed insights into how agents are used:
* **Requests per User**: Average, minimum, and maximum number of requests made by users.
* **Conversations per User**: Average, minimum, and maximum number of conversations initiated by users.
* **Requests per Conversation**: Average, minimum, and maximum number of requests within a single conversation.
* **Dropouts**: Users who have not returned to use the agent after the previous corresponding period.
This dashboard also analyzes token consumption, including total, input, and output tokens. Visual charts display requests per day, average requests per day of the week, and the most popular tools used.
## Carbon Footprint Dashboard
The Carbon Footprint dashboard assesses the environmental impact of using AI models:
* **Energy Consumption**: The electrical energy required to power and operate the models.
* **Global Warming Potential (GWP)**: The impact of greenhouse gases compared to CO₂.
* **GPU and Server Energy**: Energy consumed by GPUs and servers.
* **Power Usage Effectiveness (PUE)**: A measure of data center energy efficiency.
* **Emission Factor**: The carbon footprint per unit of computation.
Charts in this section aggregate average emissions per day.
## Global Administration Analytics
The Global Administration Analytics section provides a comprehensive view of platform-wide metrics, divided into four main areas: Dashboard, Costs, Usages, and Carbon Footprint.
### Main Dashboard
This section displays:
* **Total Users, Sessions, and Added Documents**: Metrics for all agents, including new agents created in the selected period.
* **Generated Responses and Global Token Count**: Overall platform activity.
* **Cost Estimates**: Based on token usage and model configurations.
* **Performance Metrics**: Tokens and messages per model, and response times.
### Cost Dashboard
The Cost Dashboard breaks down financial data:
* **Estimated Costs**: Displayed with comparisons to previous periods.
* **Cost Analysis**: Charts show costs per agent, model, provider, and feature.
* **Aggregations**: Line charts for cost per model and agent.
The data displayed here can be filtered by model for more specific insights.
This document aims to provide a clear and concise understanding of the platform's analytics capabilities, making it accessible to users unfamiliar with the system.