SDKs

SDKs#

The operator interacts with frontier models through vendor SDKs. The structure of the API has converged across vendors (messages, tools, streaming, structured outputs); you should learn the structure once and treat the vendor choice as configuration.

Vendor SDKs#

  • Anthropic, anthropic (Python, TypeScript, Go, Java, Ruby). Claude family. Native support for tool use, prompt caching, extended thinking, vision, files.

  • OpenAI, openai (Python, TS, .NET, Go, Java). GPT family, plus the Responses and Assistants APIs.

  • Google, google-genai (Gemini), Vertex AI SDK for cloud-billed deployments.

  • AWS Bedrock, boto3 bedrock-runtime. Multi-vendor catalog (Anthropic, Meta, Mistral, Amazon) under one IAM surface.

  • Azure OpenAI, the OpenAI SDK pointed at an Azure resource endpoint and key.

Structure of a request#

All current SDKs use a messages model. A minimal call looks the same across vendors.

from anthropic import Anthropic
client = Anthropic()
resp = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    system="You are a security analyst.",
    messages=[{"role": "user", "content": "Summarize this log."}],
)
print(resp.content[0].text)

The same call against OpenAI.

from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a security analyst."},
        {"role": "user", "content": "Summarize this log."},
    ],
)
print(resp.choices[0].message.content)

Cross-cutting features#

  • Streaming, token-by-token output via SSE. Use for interactive UX; do not use in batch pipelines (no benefit, more failure modes).

  • Tool use / function calling, the model returns a structured call (name, input); the application executes and returns the result; the model continues. The basis for agents (Agents).

  • Structured outputs, JSON-schema-constrained responses. Prefer over “ask the model for JSON” prompting.

  • Prompt caching, mark stable prefixes (system prompt, large context, tool definitions) for reuse. Cuts cost and latency for repeated calls. Vendor specifics differ; the pattern is the same.

  • Vision, images as input parts. Useful for screenshots, diagrams, scanned documents.

  • Files API, upload once, reference by ID across calls. Avoids re-sending large context on every request.

Operational practice#

  • Pin the model. Frontier models change. Pin a model ID in config, bump deliberately, version-control the bump.

  • Keep secrets out of the prompt. The prompt and response are logged by default at most vendors. If the operation requires zero exposure, run a local model (Local Models).

  • Budget every call. Set max_tokens; track input + output tokens; alert on cost regressions.

  • Retry on transient errors. 429 and 5xx are normal at scale; idempotent requests can retry with exponential backoff.

  • Log the full request/response for any call that ships to production. The structure is small and the artifact is the most useful debugging input.

References#

Contents