> ## Documentation Index > Fetch the complete documentation index at: https://docs.orq.ai/llms.txt > Use this file to discover all available pages before exploring further. # Retrieve an Evaluator > Retrieve a single evaluator by ID with more detail than the list endpoint: full type-specific config, owner, domain_id, metadata and enabled. **Related guide**: Evaluators guide. See the [Evaluators guide](/ai-studio/optimize/evaluators) for a walkthrough with examples. ## OpenAPI ````yaml get /v2/evaluators/{id} openapi: 3.1.0 info: title: orq.ai API version: '2.0' description: orq.ai API documentation servers: - url: https://my.orq.ai security: - ApiKey: [] tags: - name: Chunking description: Split text into smaller chunks for retrieval and generation workflows. - name: File Systems description: >- Create and manage persistent file systems that agents and MCP clients read from and write to. - name: Knowledge Bases description: Create and manage knowledge bases used by agents and retrieval workflows. - name: Memory Stores description: Create and manage memory stores, memories, and memory documents. - name: Evals description: Run an evaluator against a conversation and its result - name: Logs description: >- OpenTelemetry log query API. Search, filter, aggregate, and facet log records ingested via OTLP. - name: Reporting description: >- GenAI reporting API over canonical analytics rollups. Accepts a metric name, time range, grain, group-by, and filters; returns a typed time series and optional totals. - name: Traces description: >- Query and inspect ingested trace data: search trace summaries, aggregate metrics, and read individual traces and their spans. - description: List models available through the AI Router. name: Models - name: Policies - name: Alerts description: >- Alerts evaluate a Reporting API metric on a fixed interval and fire notifications through notifiers when the value breaches a threshold. Each breach opens a trigger that tracks the incident until the value recovers. - name: Annotation Queues description: Annotation queues collect spans for human review. - name: API keys description: >- API keys authenticate programmatic access to the workspace. They expose opaque tokens, per-domain access grants, and budget and rate-limit constraints. - name: Audit Logs description: Audit logs record workspace entity changes and access-relevant events. - name: Budgets description: >- Budgets govern spend, token usage, and request rate across six scopes: workspace, project, identity, API key, provider, and model. Every applicable budget is enforced, and the most restrictive limit applies per dimension. - name: Files description: File upload and retrieval operations. - name: Guardrail Rules description: >- Guardrail Rules conditionally enforce evaluators and plugins for AI Gateway traffic. Rules may be scoped to a project or the whole workspace. - name: Hub description: Hub items are reusable templates available to a workspace. - name: Identities description: >- Identities represent end users from your system for usage and engagement tracking. - name: Management keys description: >- Management keys are workspace-scoped credentials that authenticate programmatic access to workspace administration surfaces (API keys, budgets). Unlike project-scoped API keys, a management key always operates at the workspace level. - name: MCP Gateway description: >- Register upstream MCP servers, discover and sync their tools, and assemble gateways that expose a curated tool surface to MCP clients. - name: Model Catalog description: >- Browse the orq.ai model catalog: every model orq offers, across every provider, with pricing, capabilities and benchmark data. List endpoints only return models that are not deprecated. This API is public, requires no authentication, and is rate limited to 120 requests per minute per IP. Responses carry a 5-minute cache-control max-age. - name: Notifiers description: Notifier destinations used to send delivery and workflow notifications. - name: Projects description: Projects organize resources within a workspace - name: Routing Rules description: >- Routing Rules conditionally select models and enforce request plugins for AI Gateway traffic. Rules are evaluated by ascending priority and may be scoped to a project or the whole workspace. - name: Threads description: Threads group related trace invocations and their aggregate usage - name: Skills description: >- Skills are modular instructions you can use to codify processes and conventions - name: Smart Routers description: >- Create and manage workspace Smart Routers. A Smart Router selects a model from an eligible pool for each request according to a quality, balanced, or cost profile. - name: Webhooks description: >- Create and manage webhooks that deliver workspace events to external HTTPS endpoints. - name: Workspaces description: >- A workspace is the tenant. Create is called from a user session during onboarding; Get, List, and Update are the public management surface. - name: Workspace Security description: >- Workspace-level domain verification and IP allowlist controls. These operations are restricted to workspace administrators. - name: Workspace Settings description: >- Workspace-level settings managed with a workspace credential. A workspace is the tenant, so these settings are a singleton — there is nothing to create or delete, only read and update. - name: Responses - description: Run agents on a cron cadence. Minimum firing interval is 1 hour. name: Agent Schedules - name: Embeddings - name: Telemetry description: >- Unified query envelope for traces, metrics, and logs. One request shape, one filter dialect, and one response shape per source, validated by a per-source registry. - description: Beta. Run typed classification questions against a classify model. name: Classify - description: Search Gateway with managed credits or BYOK. name: Web Search externalDocs: url: https://docs.orq.ai description: orq.ai Documentation paths: /v2/evaluators/{id}: get: tags: - Evals summary: Retrieve an Evaluator description: >- Retrieve a single evaluator by ID with more detail than the list endpoint: full type-specific config, owner, domain_id, metadata and enabled. operationId: GetEval parameters: - name: id in: path required: true schema: type: string responses: '200': description: The evaluator. The response shape depends on the evaluator `type`. content: application/json: schema: oneOf: - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string description: The type of output expected from the evaluator type: type: string enum: - llm_eval mode: type: - string - 'null' enum: - single - jury - null repetitions: type: - integer - 'null' minimum: 1 maximum: 3 model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id jury: type: - object - 'null' properties: judges: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id retry: type: object properties: count: type: integer minimum: 1 maximum: 5 default: 2 on_codes: type: array items: type: integer minimum: 100 maximum: 599 minItems: 1 default: - 429 - 500 - 502 - 503 - 504 fallbacks: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id required: - model default: [] required: - model minItems: 2 replacement_judges: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id retry: type: object properties: count: type: integer minimum: 1 maximum: 5 default: 2 on_codes: type: array items: type: integer minimum: 100 maximum: 599 minItems: 1 default: - 429 - 500 - 502 - 503 - 504 fallbacks: type: array items: type: object properties: model: type: object properties: id: type: string integration_id: type: - string - 'null' model_parameters: type: object properties: temperature: type: number max_tokens: type: number top_k: type: number top_p: type: number frequency_penalty: type: number presence_penalty: type: number reasoning_effort: type: string budget_tokens: type: number required: - id required: - model default: [] required: - model default: [] min_successful_judges: type: integer minimum: 2 default: 2 tie_value: type: string enum: - Tie default: Tie required: - judges prompt: type: string categories: items: type: string type: - array - 'null' categorical_labels: type: - array - 'null' items: type: object properties: value: type: string minLength: 1 description: type: string required: - value dataset_id: type: - string - 'null' required: - metadata - _id - display_name - description - owner - domain_id - type - prompt title: LLM - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string type: type: string enum: - function_eval function_params: oneOf: - type: object properties: type: type: string enum: - contains value: type: string required: - type - value - type: object properties: type: type: string enum: - contains_none keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - contains_all keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - contains_any keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - contains_email required: - type - type: object properties: type: type: string enum: - contains_url required: - type - type: object properties: type: type: string enum: - contains_valid_link required: - type - type: object properties: type: type: string enum: - start_with value: type: string required: - type - value - type: object properties: type: type: string enum: - ends_with value: type: string required: - type - value - type: object properties: type: type: string enum: - exact_match required: - type - type: object properties: type: type: string enum: - length_less_than value: type: number required: - type - value - type: object properties: type: type: string enum: - length_between min: type: number max: type: number required: - type - min - max - type: object properties: type: type: string enum: - length_greater_than value: type: number required: - type - value - type: object properties: type: type: string enum: - one_line required: - type - type: object properties: type: type: string enum: - regex pattern: type: string required: - type - pattern - type: object properties: type: type: string enum: - is_valid_json required: - type - type: object properties: type: type: string enum: - moderations_openai required: - type - type: object properties: type: type: string enum: - moderations_google required: - type - type: object properties: type: type: string enum: - bert_score required: - type - type: object properties: type: type: string enum: - bleu_score required: - type - type: object properties: type: type: string enum: - rouge_n required: - type - type: object properties: type: type: string enum: - meteor_score required: - type - type: object properties: type: type: string enum: - cosine_similarity required: - type - type: object properties: type: type: string enum: - levenshtein_distance required: - type - type: object properties: type: type: string enum: - keywords_match keywords: type: array items: type: string required: - type - keywords - type: object properties: type: type: string enum: - most_repeated_words required: - type - type: object properties: type: type: string enum: - flesch_reading_ease required: - type - type: object properties: type: type: string enum: - gse_english_level required: - type - type: object properties: type: type: string enum: - words_count required: - type - type: object properties: type: type: string enum: - sentences_count required: - type - type: object properties: type: type: string enum: - lexical_repetition required: - type - type: object properties: type: type: string enum: - grammar_diversity required: - type required: - metadata - _id - display_name - description - owner - domain_id - output_type - type - function_params title: Function - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - number type: type: string enum: - ragas model: type: object properties: id: type: string integration_id: type: - string - 'null' required: - id ragas_metric: type: string enum: - context_precision - context_recall - context_entities_recall - harmfulness - maliciousness - coherence - correctness - conciseness - response_relevancy - faithfulness - summarization - noise_sensitivity required: - metadata - _id - display_name - description - owner - domain_id - type - model - ragas_metric title: Ragas - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean type: type: string enum: - json_schema schema: type: string required: - metadata - _id - display_name - description - owner - domain_id - type - schema title: JSON - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string description: The type of output expected from the evaluator type: type: string enum: - http_eval url: type: string method: type: string enum: - GET - POST headers: type: object additionalProperties: type: string payload: type: object additionalProperties: {} required: - metadata - _id - display_name - description - owner - domain_id - type - url - method - headers - payload title: HTTP - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - number code: type: string type: type: string enum: - python_eval required: - metadata - _id - display_name - description - owner - domain_id - code - type title: Python - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean - categorical - number - string description: The type of output expected from the evaluator code: type: string type: type: string enum: - typescript_eval required: - metadata - _id - display_name - description - owner - domain_id - code - type title: Typescript - type: object properties: enabled: type: boolean default: true metadata: type: object properties: required_model_with_tools_support: type: boolean required_retrieval_context: type: boolean required_expected_output: type: boolean supported_on_input_type: type: boolean supported_on_output_type: type: boolean support_use_as_guardrail: type: boolean _id: type: string display_name: type: string description: type: string owner: type: string created: type: string default: '2026-08-05T15:12:51.947Z' updated: type: string default: '2026-08-05T15:12:51.947Z' created_by_id: type: - string - 'null' updated_by_id: type: - string - 'null' domain_id: type: string project_id: type: string description: >- Unique identifier of the project owning this evaluator. guardrail_config: anyOf: - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - boolean value: type: boolean required: - type - value title: Boolean - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - categorical values: type: array items: type: string minLength: 1 required: - type - values title: Categorical - type: object properties: enabled: type: boolean default: true alert_on_failure: type: boolean default: false type: type: string enum: - number value: type: number operator: type: string enum: - eq - ne - gt - gte - lt - lte required: - type - value - operator title: Number - type: 'null' x-speakeasy-type-override: any output_type: type: string enum: - boolean type: type: string enum: - bedrock_eval guardrail_identifier: type: string minLength: 1 guardrail_version: type: string minLength: 1 default: DRAFT guardrail_region: type: string minLength: 1 integration_id: type: - string - 'null' required: - metadata - _id - display_name - description - owner - domain_id - type - guardrail_identifier title: Bedrock Guardrail '404': description: >- No evaluator with this id exists in the authenticated workspace, or the request carries no workspace. content: application/json: schema: type: object properties: message: type: string example: Evaluator not found required: - message x-code-samples: - lang: typescript label: Node.js - Retrieve an evaluator source: | import { Orq } from '@orq-ai/node'; const orq = new Orq({ apiKey: process.env.ORQ_API_KEY }); const evaluator = await orq.evals.get({ id: '01KT1FCSA8N3YD1K8YBPVTAV9E', }); console.log(evaluator); components: securitySchemes: ApiKey: type: http scheme: bearer bearerFormat: JWT ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.