Say a customer has just finished a load test. Sales has a follow-up call the next morning and needs to know whether the endpoint will stay within its latency target at the concurrency the customer expects in production.
The answer is spread across several systems. Someone has to connect the customer account to the right endpoints and deployments, check which inference engine each deployment is running, and find the corresponding metrics. The warm-up period and an unusual traffic spike need to be excluded. Then someone has to calculate p50, p95, and p99 time to first token, along with per-user throughput, and explain what those numbers mean for the customer's workload.
Until recently, that request went to Engineering. An engineer would open a few dashboards, adjust the queries, run a script, and write up the result. Sales waited while the engineer switched context. Once the answer was sent, the method often disappeared into a one-off script or a Slack thread.
We wanted the person talking to the customer to be able to do that first pass themselves. That's what Vela is for. It's our internal Slack agent, and it can follow a question like this across the systems involved.
Why AI agents need shared operational context
We were already using Claude to help with operational work, but each new session needed to learn Parasail again.
It had to be told which dashboard to use, which query had produced the number we trusted last time, whether a deployment was running vLLM or SGLang, and whether an endpoint name in Grafana matched a deployment identifier in our control plane. Someone who knew our infrastructure could steer it through all of that, but the path had to be reconstructed in every session.
None of this knowledge fits neatly into an API schema. A single question can touch the control plane, Grafana and Prometheus, the warehouse, and the systems that hold billing and customer context. They update at different rates, and each answers a slightly different version of the question.
A model with access to all of them does not automatically know which one is authoritative for a given fact. It can write a reasonable query against the wrong metric, or join two identifiers that happen to look related. The result may be plausible enough that the mistake is hard to notice.
Building operating knowledge into an AI agent
There are plenty of ways to put an agent in Slack. What we needed to figure out was how to embed Parasail's operating knowledge in the agent.
We had to teach Vela which sources to trust. The live control plane is the right place to check how a deployment is configured now. A warehouse datastore can be better for historical analysis, but it may lag behind production. A dashboard can be a useful view without being the source that should drive a calculation.
We had to document how our identifiers fit together. A customer name has to lead to an account, an endpoint, one or more deployments, the serving engine, and finally the labels used in our metrics.
We also had to decide which calculations were safe to reuse. Time to first token and per-user token throughput are not just column names waiting to be selected. The query, time window, filters, units, and aggregation all matter. When those choices affect a customer answer or an operating decision, they should not be made up again on every run.
Turning recurring AI agent requests into reviewed workflows
Some questions really are new. For those, Vela reads the relevant guides, queries several sources, follows a lead, and revises its approach when the first query is wrong. Nothing on that path has been reviewed ahead of time, so someone still has to check the result.
When the same question keeps coming back, we stop asking Vela to rediscover the method. Once someone finds a reliable way to answer it, we move the important parts into reviewed code. The query, joins, unit conversions, calculations, and validation become a script with explicit inputs. The instructions for when to use it are reviewed alongside the code.
The next time someone asks, Vela routes the request into that workflow. The script gathers the data and does the math, so the arithmetic behind an answer never gets improvised in the middle of a report. The model's job is interpretation: work out what the person is asking, explain the result, compare it with other evidence, and flag anything that does not line up.
The load-test analysis could begin as an unfamiliar investigation. If it becomes common, the working method can graduate into a standard workflow that accepts an endpoint, a time window, and an expected concurrency. The workflow resolves the deployment, selects the known metrics, removes the periods that should not count, and computes the same set of results every time.
One person still has to figure out the workflow. Everyone after them gets to start with it instead of hunting for the engineer who solved it or searching for an old Slack thread. They can also see its assumptions and recognize when it no longer fits.
Trusted AI agent access, bounded in production
Giving more people access raised two questions: whether an answer could be trusted, and what Vela could do if it got one wrong.
For recurring workflows, the source choices and calculations are reviewed. With an exploratory answer, Vela shows the evidence it found and leaves uncertainty visible. When two sources disagree, it should say so instead of picking one.
For access, our threat model starts with an uncomfortable assumption: anything available to the model's process may be exposed by a successful prompt injection. Telling the model not to use a credential is not a security boundary. Sensitive capabilities have to be kept out of its reach or placed behind a separate confirmation path.
Vela's access to production infrastructure is read-only. It can inspect deployment configuration and operational data, but it cannot change how production is configured. It can help prepare a benchmark or evaluation, then show the proposed run in Slack. The test starts only after a person confirms it, and the credential that actually dispatches the run is held by a separate bot and never enters the model's context.
How teams use Vela for inference operations
Sales and GTM were the first teams to adopt Vela. Their customer questions regularly cut across accounts, deployments, performance, usage, and commercial context. Those are the questions that used to require an engineer to assemble the evidence.
Engineering still owns production changes and the problems nobody has seen before. However, when a customer asks how their load test looked, Sales can arrive at the next call with evidence instead of a promise to ask Engineering.
Parasail runs production open-source inference without making your team carry the MLOps and GPU-procurement load. If you’re working through capacity, reliability, or cost tradeoffs, talk to an engineer about your workload.