Skip to main content

AI Gateway

Route Claude calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for the use case.

Observability

Send the Anthropic SDK’s native OpenTelemetry spans to Orq.ai to trace every request and tool use.

Claude Agent SDK

Building programmable agents with multi-turn conversations, tool use, and MCP servers? Use the Claude Agent SDK instead.

Anthropic Messages API

Endpoint details, authentication, and cache_control breakpoints on the Anthropic-compatible endpoint.

AI Gateway

Overview

The Anthropic SDK (@anthropic-ai/sdk and anthropic) is the native client for the Claude Messages API. Connect it to Orq.ai’s AI Gateway with a single base_url change and no other modifications to existing code, unlocking 500+ models across 30+ providers, fallbacks, caching, and cost tracking.
For programmable agents with multi-turn conversations, tool use, and MCP servers, see the Claude Agent SDK integration.

Key Benefits

Orq.ai’s AI Gateway enhances the Anthropic SDK with:

Complete Observability

Track every API call, token usage, and model interaction with detailed traces and analytics

Built-in Reliability

Automatic fallbacks, retries, and load balancing for production resilience

Cost Optimization

Real-time cost tracking and spend management across all AI operations

Multi-Provider Access

Access 500+ LLMs and 30+ providers through a single, unified integration

Prerequisites

Before integrating the Anthropic SDK with Orq.ai, ensure the following are in place:
  • An Orq.ai account and API Key
  • Python 3.8+ or Node.js 18+ with TypeScript support
  • Anthropic SDK installed
To set up an API key, see API keys & Endpoints.

Installation

Configuration

Set the Base URL to the AI Gateway and authenticate with an Orq.ai API key to route calls through the API without changing any other part of the code.
base_url: https://my.orq.ai/v3/anthropic

Text Generation

Basic text generation with the Anthropic SDK through Orq.ai:

Streaming

Stream responses for real-time output:

Tool Use

Define tools and let the model decide when to call them. The AI Gateway forwards tool definitions and returns tool_use blocks unchanged:

Model Selection

The anthropic/ prefix routes to Claude models. Switch to any supported model from 30+ providers by changing the model value:
Browse every available model in Supported Models.

Observability

Getting Started

The Anthropic SDK emits OpenTelemetry spans natively: one anthropic.messages.create span per request, and one span per tool use run by the tool runner. No instrumentation library is needed. Register a tracer provider with an OTLP exporter pointing at Orq.ai, and the SDK picks it up without changes to existing calls.
SDK tracing is in beta. Span names, attributes, options, and environment variables can change between SDK versions. Pin the SDK version to control when changes are adopted. See Set up OpenTelemetry with the Anthropic SDK.
Calls through the AI Gateway base_url are already traced by Orq.ai, so this section is only needed to nest model calls under application spans. With SDK tracing also on, the gateway adds a second root span in the same trace, linked to the SDK span (it nests instead when the caller is already an orq-traced span). To avoid the duplicate, turn SDK spans off with open_telemetry=False (TypeScript: openTelemetry: false). See Configure the SDK.

Prerequisites

  • An Orq.ai account and API Key
  • Python 3.10+ or Node.js 18.19+ / 20.6+
  • An Anthropic API key (ANTHROPIC_API_KEY). The samples call Anthropic directly.

Install Dependencies

Pin the installed Anthropic SDK version, since tracing is in beta.

Register a Tracer Provider

Register the tracer provider before the first call to trace. The exporter can be configured with environment variables or inline in code.
Set the standard OpenTelemetry exporter variables:
The exporter reads them, so it takes no arguments:

Configure the SDK

SDK options are set with ANTHROPIC_OPEN_TELEMETRY* environment variables or inline on the client with open_telemetry (TypeScript: openTelemetry). They are separate from the exporter’s OTEL_* settings. By default, spans exclude prompts and responses. Set the content mode to content to see inputs and outputs in Orq.ai. Content mode sends prompts and responses to Orq.ai. See Data privacy. The conversation ID is exported in both content modes, so keep personal data and secrets out of it.
Passing open_telemetry to a client overrides all ANTHROPIC_OPEN_TELEMETRY* environment variables as a whole. Any nested key left out uses its default, even when its environment variable is set.
With @opentelemetry/auto-instrumentations-node or @opentelemetry/instrumentation-undici, set openTelemetry: { propagation: false }. Otherwise an undici instrumentation bug adds a second traceparent header and invalidates the trace context. Set the content mode inline too, per the Warning above.

Tracing Streams

A stream’s span ends once the stream is fully read or closed, so close streams that stop early. In content mode, the span records the response only when the stream is read with messages.stream().

View Traces

Open the Traces view in Orq.ai to see each anthropic.messages.create span with its model, token usage, and latency, plus prompts and responses in content mode.

Troubleshooting