Spend dashboards
Performance monitoring
Evaluator quality
Guardrail enforcement
Endpoint
POSThttps://my.orq.ai/v2/reporting
- All requests require a
Bearertoken. See API Keys for how to generate one. - The workspace and project scope are derived from the API key itself.
See the API Reference
POST /v2/reporting.Quickstart
Request
groupBy in TypeScript, group_by in Python).grain sets the time bucket: auto, minute, hour, or day. auto, or an omitted field, lets the server choose the bucket. The response reports the applied bucket in meta.effective_grain.
mode: the default timeseries mode returns one row per time bucket. Set "mode": "scalar" to aggregate the whole window instead: with group_by the response carries one row per dimension combination ordered by value (a top list, capped by limit; flip the order with "sort": "asc"), and without group_by exactly one row (a single aggregate). Scalar rows omit the timestamp field.
Response
Every successful response follows the same envelope. Thedata array contains time-ordered buckets. Each bucket carries:
- a
dimensionsmap: thegroup_byvalues for that row (see Dimensions) - a
metricsmap: shape depends on the metric requested
genai.usage) return a full set of related fields in one call.
totals is present only when include_totals: true is set in the request; otherwise the field is omitted. Metric values are rounded to 10 decimal places server-side.
Metric catalog
- Usage
- Evaluator
- Guardrail
genai.latency.p*, genai.ttft.p*) carry a +/-1-2% error
band typical of t-digest estimation at high cardinality. genai.ttft.*
metrics cover streaming requests only and do not support entity dimensions
such as agent or tool.Dimensions
Entity dimensions (agent, tool, and so on) are available for usage metrics and break a metric down by a product entity.
billing_billable is available as a filter field for usage metrics, but not
as a group_by dimension. To break down Orq.ai-managed versus customer-owned credentials, use credential_type instead. The values orq_managed and customer_byok are derived at query time from the workspace billing setup, not stored on each event.- Usage dimensions
- Entity dimensions
- Evaluator and guardrail dimensions
genai.requests, genai.tokens, genai.cost, genai.errors, genai.latency.*, or genai.usage.Examples by use case
Spend: cost by day (bundle)
Spend: cost by day (bundle)
Spend: cost by model per hour
Spend: cost by model per hour
Spend: BYOK vs Orq.ai-managed credentials
Spend: BYOK vs Orq.ai-managed credentials
dimensions.credential_type as "orq_managed" or "customer_byok".Entity dimension: cost by Agent
Entity dimension: cost by Agent
Entity dimension: cost by Tool with model filter
Entity dimension: cost by Tool with model filter
Latency: p95 by model
Latency: p95 by model
metrics["genai.latency.p95"] is reported in milliseconds.Latency: p50, p95, p99 in parallel
Latency: p50, p95, p99 in parallel
Errors: count by provider
Errors: count by provider
Errors: error rate by model (client-side ratio)
Errors: error rate by model (client-side ratio)
Evaluator: pass rate by Evaluator and version
Evaluator: pass rate by Evaluator and version
Evaluator: average score
Evaluator: average score
Evaluator: runs by result label
Evaluator: runs by result label
Guardrail: block rate by Evaluator and stage
Guardrail: block rate by Evaluator and stage
Guardrail: triggers by Policy and stage
Guardrail: triggers by Policy and stage
Guardrail: filter by origin
Guardrail: filter by origin
Multi-tenant: cost per project per day
Multi-tenant: cost per project per day
Multi-tenant: usage per API key
Multi-tenant: usage per API key
Identity: cost per end-user Identity
Identity: cost per end-user Identity
Identity: requests filtered to a single end-user
Identity: requests filtered to a single end-user
Time zone: day buckets in America/New_York
Time zone: day buckets in America/New_York
High resolution: requests per minute, last hour
High resolution: requests per minute, last hour
Cross-dimension: cost by credential type and model
Cross-dimension: cost by credential type and model
Errors
Errors come back as JSON envelopes with a numericcode, a human-readable message, and an empty details array.