Live answer, requirement 2

LLM applications in production

One gateway for every model call, guardrails on both sides, and a measured choice of model per task.

The requirementShip LLM features that are safe, measurable and cost-aware, across more than one model and provider.

Try it

Send one prompt. A safety classifier screens it first; if it passes, an 8B and a 70B model answer side by side with measured latency and token use, so the size and cost trade-off is visible.

How it works

One gateway for every callAll model traffic goes through the Luca gateway, which holds provider keys, rate limits and per-call logging in one place. It routes to more than one provider. Spend caps and a cost ledger are in early access.
Right-size the model per taskClassification and extraction run on a small fast model; reasoning and drafting run on a large one. Both are measured on the customer's own examples before the choice is fixed.
Guardrails on input and outputA safety classifier screens the prompt before any model sees it, and output is constrained to a schema the application validates.
Evaluate every changeAn LLM judge scores answers on fixed metrics, with a fairness check, and a prompt or model change ships only if the scores hold.

Tools

Tool	What it does here	Status
Luca gateway	Multi-provider model gateway with keys, rate limits and logging.	Available
Claude (Anthropic API)	Primary reasoning model for agentic and drafting work.	Available
Luca evaluation judge	Six-metric LLM judge with a fairness harness.	Available
Spend caps, approval gates	Budget limits and human approval on agent actions.	Early access
Cloudflare Workers AI	Runs this public demo: Llama Guard 3 plus Llama 3.1 8B and Llama 3.3 70B.	Demo only

The full write-up covers access control, evaluation and how this runs inside a customer environment.

Read it on gusit.de

LLM applications in production

Try it

How it works

Tools

Other answers