Inference Infrastructure

GoCreative AI operates self-hosted LLM inference on NVIDIA GPUs and serves it through an OpenAI-compatible API. This page documents the endpoint, its measured performance, pricing, and our data-handling policy.

GoCreative AI runs two lines of business: an AI creative product at gocreativeai.com, and the inference infrastructure described here. This page covers the latter.

Endpoint

Base URL	https://llm.gocreativeai.com/v1
Models	`GET /v1/models` — returns pricing, context length, datacenter location, throughput capacity, supported features
Completions	`POST /v1/chat/completions` — OpenAI-compatible, SSE streaming
Serving stack	vLLM, continuous batching, paged KV-cache, prefix caching
Region	us-central1 (additional regions on request)

curl https://llm.gocreativeai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen/qwen3-8b","messages":[{"role":"user","content":"Hello"}],"max_tokens":64}'

Measured performance

Benchmarked on our own hardware. These are measured figures, not vendor specifications or projections.

Metric	Value	Conditions
Sustained output throughput	6,924 tokens/sec	concurrency 200, NVIDIA RTX PRO 6000 Blackwell
Errors across the sweep	0	concurrency 32 to 200
End-to-end latency	283–534 ms	measured through the public URL, warm
Reported capacity	290,808 input tokens/min	quoted at 70% of measured peak, deliberately conservative

We report capacity below measured peak on purpose. Peak was achieved in a clean benchmark; production traffic is messier. Advertising a number we cannot hold in production would produce timeouts, which serves no one.

Pricing

Model	Input	Output	Context
qwen/qwen3-8b (GPTQ-Int4)	$0.150 / 1M tokens	$0.301 / 1M tokens	8,192

Live pricing is always served from /v1/models and is authoritative over this page.

Data handling and retention

Zero data retention. We do not store prompts or completions.

Request and response content is never written to disk. The inference engine runs with request logging disabled.
Our metering ledger records only: request ID, model name, input token count, output token count, latency, and price. No message content, at any layer.
No training on customer traffic. We do not fine-tune, evaluate, or otherwise use inference traffic to improve models.
No third-party sharing. Request content is not transmitted to any party beyond the GPU serving the request.
Retention period for metadata: token counts and timing are retained for billing reconciliation and are not linked to request content, because no content exists.

Because no request content is retained, inference traffic through this endpoint holds no personal data at rest. We are happy to execute a Data Processing Agreement.

Reliability

Availability is sampled automatically every 10 minutes against the public URL and logged.
The endpoint reports drain state, so upstream routers can reroute cleanly rather than receiving errors.
Every response — streaming and non-streaming — carries a unique Inference-Id header for independent billing reconciliation.
usage token counts are returned on both streaming and non-streaming responses.

Integration

The API is OpenAI-compatible, so any client that speaks the OpenAI Chat Completions format works without modification — including the OpenAI SDKs, LiteLLM, and standard gateway software. Point the base URL at https://llm.gocreativeai.com/v1

Contact

Partnership, routing, and capacity enquiries: contact@gocreativeai.com