v0.4.2 Edge Engine • Private Beta Now Open

Unified AI Gateway & Deterministic Edge Routing

Route, cache, and observe LLM completions across 14+ inference providers with a single SDK drop-in. Built in Rust for sub-12ms proxy overhead.

npm i @oral12/edge-sdk • SOC2 Type I Compliant

router.config.ts — @oral12/edge-sdk

import { Oral12Gateway } from '@oral12/edge-sdk';

export const gateway = new Oral12Gateway({
  endpoint: 'https://edge.oral12.studio/v1',
  semanticCache: {
    enabled: true,
    similarityThreshold: 0.94,
    ttlSeconds: 86400
  },
  fallbackChain: [
    'anthropic/claude-3-7-sonnet',
    'openai/gpt-4o',
    'groq/llama-3.3-70b'
  ],
  piiRedaction: 'strict'
});

// Execute with automatic failover <11ms overhead
const res = await gateway.chat.create({
  messages: [{ role: 'user', content: prompt }]
});

Core Infrastructure

Engineered for Production Inference

Stop managing brittle retry loops and rate-limit cascades in application code. Offload resilience to the edge.

Vector Semantic Caching

Identifies semantically identical prompts at the edge using lightweight quantized embeddings, serving cached completions in under 15ms at zero token cost.

Automated Multi-Provider Failover

Detects HTTP 429s and 5xx gateway timeouts in real time, rerouting payloads across OpenAI, Anthropic, and Bedrock without dropping client streams.

Zero-Retention PII Firewall

Strips SSNs, API keys, and sensitive customer entities in-flight before payloads leave your regional compliance boundary.

Granular Cost Attribution

Tag requests by tenant ID, feature flag, or environment to track per-customer unit economics and enforce hard daily token budgets.

Prompt Version Control

Decouple system prompts from application deployments. Roll back templates, run A/B shadow evaluations, and validate JSON schemas effortlessly.

OpenTelemetry Native

Stream structured trace logs and token latency histograms directly into Datadog, Grafana, or Axiom with zero vendor lock-in.

Transparent Metering

Predictable Usage-Based Tiers

Zero markup on underlying model provider token costs. Pay only for gateway throughput.

Sandbox

$0 / month

For solo builders prototyping AI workflows and testing failover policies.

Up to 50,000 requests / mo
Standard semantic caching
7-day log retention
Community Discord support

Deploy Free

Frequently Asked Questions

How does oral12 Studio avoid adding latency to streaming responses?

Our edge proxy is written in Rust and compiled to WebAssembly across 280+ PoPs. We inspect headers and authenticate API keys in memory without buffering SSE (Server-Sent Events) chunks, adding less than 12ms at P99.

Do you store our prompt payloads or model completions?

By default, payloads are encrypted at rest for observability and semantic caching within your isolated tenant namespace. Enterprise and Growth users can enable strict Zero-Data-Retention (ZDR) mode via request headers.

Can we bring our own OpenAI, Anthropic, and AWS Bedrock keys?

Yes. oral12 Studio operates on a Bring-Your-Own-Key (BYOK) vault architecture. Your provider keys are encrypted using envelope encryption and injected only at the moment of upstream dispatch.