| Prompts and completions | Request and response content, tool definitions, reasoning output | Not persisted. In memory only, for the duration of the request | Serving the request |
| In-memory prefix cache | Token-level KV state of recent request prefixes, in GPU memory | Transient; continuously evicted, fully cleared on process restart | Latency reduction for repeated prefixes |
| Distributed traces | Request lifecycle timing and sizes (OpenTelemetry spans); no content | Up to 7 days, then automatically deleted | Reliability debugging |
| Application logs | Serving-engine and gateway logs: timestamps, status codes, request/queue state; no content | Up to 30 days, then automatically deleted | Incident investigation, abuse prevention |
| Edge / network logs | Connection metadata at the network perimeter (IPs, URLs, timing) | Up to 30 days | Security, DDoS and abuse mitigation |
| Aggregate metrics | Latency histograms, token-count totals, error and utilization counters; fully aggregated, no per-request payloads | Up to 13 months | Capacity planning, SLO reporting |
| Usage and billing records | Per-partner / per-customer token counts and request totals | 7 years | Invoicing, accounting, and tax obligations |
| Direct-customer account data | Business contact and billing details | Life of the agreement + 7 years | Contract and accounting obligations |