Grafana Dashboard

Pre-built Grafana dashboards for monitoring PromptRails AI agents, prompts, costs, and execution trends.

Grafana Dashboard

PromptRails provides pre-built Grafana dashboards for monitoring your AI agents, prompts, and costs. The dashboards connect to the PromptRails API using the Infinity datasource plugin and provide real-time visibility into execution trends, error rates, cost breakdowns, and more.

The source code is available on GitHub.

Dashboards

Overview

Provides a high-level view of your PromptRails platform:

Cost Analysis

Detailed cost breakdown across agents and models:

Quick Start (Docker)

The fastest way to get started is with Docker. This setup auto-provisions dashboards and the datasource.

# 1. Clone
git clone https://github.com/promptrails/grafana-dashboard
cd grafana-dashboard
 
# 2. Configure
cp .env.example .env
# Edit .env and set your PROMPTRAILS_API_KEY
 
# 3. Start
docker-compose up -d
 
# 4. Open
open http://localhost:3333
# Login: admin / admin

Dashboards are auto-provisioned in the PromptRails folder. The Docker setup also re-applies the datasource on startup so changes to PROMPTRAILS_API_KEY don't get stuck in Grafana's persisted volume.

Manual Import (Existing Grafana)

If you already have a Grafana instance, you can import the dashboards manually.

1. Install Infinity Datasource

grafana-cli plugins install yesoreyeram-infinity-datasource

Or install from Grafana UI: Configuration > Plugins > Search "Infinity"

2. Create Datasource

1. Go to Connections > Data Sources > Add data source
2. Search for Infinity
3. Create an Infinity datasource with your preferred name
4. Under Custom HTTP Headers, add:
   • Header: X-API-Key
   • Value: Your PromptRails API key
5. Under Security > Allowed Hosts, add: api.promptrails.ai
6. Click Save & Test

3. Import Dashboards

1. Go to Dashboards > Import
2. Upload JSON files from the exports/ folder in the GitHub repository:
   • overview.json -- Main overview dashboard
   • cost-analysis.json -- Cost breakdown dashboard
3. When Grafana asks for PromptRails, select the Infinity datasource you created in the previous step

Dashboard Variables

Requirements

• Grafana 10+ (tested with 11.4)
• Infinity Datasource Plugin
• PromptRails API key (see API Keys & Scopes)

Troubleshooting

requested URL not allowed

If Query Inspector shows a responseCodeFromServer: 401, the URL allowlist is probably already correct and the actual problem is authentication.

Check:

1. Connections > Data Sources > your Infinity datasource > Security > Allowed Hosts contains api.promptrails.ai
2. Connections > Data Sources > your Infinity datasource > Custom HTTP Headers has X-API-Key
3. The header value is your current PROMPTRAILS_API_KEY

If you imported from exports/, also confirm the dashboard was mapped to the correct Infinity datasource during import.

For Docker installs, re-run:

docker-compose up -d

Grafana keeps datasource state in the grafana-data volume, so an old API key can survive container restarts unless the datasource is updated again.

Related Topics

• Executions -- Execution lifecycle and monitoring
• Tracing -- Distributed tracing for agent executions
• Cost Tracking -- Aggregated cost analysis
• Scoring & Evaluation -- Scoring and evaluation metrics