Forward Deployed Engineer, live answer, requirement 1 of 12

LLM applications in production

On Luca, model calls normally go through a gateway that holds provider keys, records each chat call and sets a model per task.

Built by Gus Assuncao of Gus IT LLC for a Forward Deployed Engineer application. Luca is the AI platform Gus IT builds and operates.

Try it

Send one prompt to a small model (Llama 3.2 3B) and a large one (Llama 3.3 70B) side by side. Your input and both replies are screened by Llama Guard 3, and each column shows latency, tokens and Workers AI neurons.

Public demo on Cloudflare Workers AI. Inputs are screened by Llama Guard 3 before any other model runs, requests are rate limited, and nothing here connects to a Luca or customer system.

The requirementShip LLM features that are safe, measurable and cost-aware across more than one model and more than one provider.

How it works

One gateway for model callsOn Luca, the gateway holds the keys for OpenAI, Anthropic, Gemini, Mistral and Azure and also reaches local models, so apps call one interface. The general chat route can fall back to another model or provider when one fails. Pinned routes, such as the judges and bulk generation, fail rather than switch providers, and each denial is logged. Some paths still call a provider directly.
Right-size the model per taskIn the gateway, named task routes set their own default model. The high-volume scoring judge is set to Claude Haiku, bulk catalog generation to a local model, and the mastery judge to Claude Opus. The demo measures that trade-off for a small and a large model: latency, tokens and Workers AI neurons.
Where guardrails run todayOn Luca's public Ask service, pattern filters check both the question and the reply. A Llama Guard classifier is built into those checks, but it passes all text when no endpoint is configured or when it fails. In Luca's agent dispatcher, a failed classifier call marks the request medium risk and flags it for human approval. This demo screens the prompt and both replies with Llama Guard 3.
Evaluate with a rubric judgeFor a tutoring product on Luca, I built a judge that scores up to four rubric metrics against versioned anchors. When a judge call fails, the score stays empty instead of being guessed. The harness computes agreement with human expert labels. The golden set it ships with is synthetic and still waits for those labels. At a customer, I would build that set from their own examples first.

Tools

Tool	What it does here	Status
Luca gatewayAvailable	Routes model calls to OpenAI, Anthropic, Gemini, Mistral and local models, with per-task routes and a record of each chat call.	Available
Claude (Anthropic API)Available	Set as the model for the gateway's judge, tutoring dialogue, document intake, email drafting and terminal routes.	Available
Luca rubric judgeEarly access	Built for a tutoring product: up to four scored metrics, a draft rubric and a synthetic golden set, not yet checked against human labels.	Early access
Spend caps, approval gatesEarly access	Per-user spend caps and a cost ledger, built but off by default, plus approval on build, infrastructure and data-migration jobs.	Early access
Cloudflare Workers AIDemo only	Runs this public demo: Llama Guard 3, Llama 3.2 3B and Llama 3.3 70B.	Demo only

Available means shipped and in use. Early access means built but not generally available. Roadmap means designed, not built. Demo only means it powers the public demo, not customer deployments.

The write-up has every step of how I deliver this at a customer, with the tools and their status.

Read the write-up on gusit.de

The other answers

Prompt engineeringI write the output shape first, measure the prompt against each model's context window, and check replies against that shape.Retrieval-augmented generationChat answers over a customer's own documents, returned with the matched document names, after a check that the caller may use that persona.AI agents and orchestrationI build agents that call your systems through registered tools, still in early access, and workflows that start on business events.Request classification and routingI build automation that classifies each inbound request, plans bounded tasks, and holds high-risk tasks for approval; the dispatcher is in early access.AI workflow orchestrationI map the steps, API contracts, and data flows between a customer's systems and the models before I build the workflow.API and integration designI publish the API contract first, gate it with keys and scopes, then build integrations against it.Hybrid enterprise integrationI connect AI services to your identity, applications and data, and run the platform on Kubernetes on Azure Local.Prototype to productionI take a prototype through generated tests and a two-approval merge gate to a production deploy that rolls back if the new pod keeps crashing or cannot start.Approvals and exception handlingIn chat, I hold each Gatekeeper tool marked destructive until the signed-in user approves it, with expiry and a record. This path is in early access.Evals and output validationI built a versioned rubric judge for a tutoring app, and I trace failed or slow model calls to their provider.Client-facing deliveryI turn engineering work into updates that client architects, security leads and business owners can each act on.

Built by Gus Assuncao of Gus IT LLC for a Forward Deployed Engineer application. Luca is the AI platform Gus IT builds and operates. Contact gus@gusit.de. Claude is a product of Anthropic PBC. Cloudflare and Workers AI are trademarks of Cloudflare, Inc. Llama is a trademark of Meta Platforms, Inc. Other names are trademarks of their respective owners. Gus IT LLC is an independent engineering firm and is not affiliated with or endorsed by these companies.