Can an open model cut your Claude Code and Codex costs? Here’s how to test it on your own repo

If your team runs Claude Code or Codex on every bug fix and test, the token bill grows with each session. Open models on Parasail cost far less per token, but a cheaper session saves money only when its output passes review without extra repair.

This guide shows you how to test that trade-off on your own repository. You’ll define an acceptable result before either agent runs, then compare the full cost of each workflow, including retries and developer review. Each step uses the regression-test tasks we ran on three open-source Python projects as a worked example.

GLM-5.3 Flash on Parasail passed the automated checks at approximately 89% less estimated API cost than the saved GPT-6.1 Sol sessions. In a second comparison, GPT-6 Luna was cheaper and faster than Flash on average. Your pilot should establish which workflow offers better value for the tasks you actually do.

Use the copyable prompt below to run a task in your coding agent, then record its cost, time, and outcome.

The five steps below start with what your coding agent costs today and end with a routing decision for your team.

1. Record your current costs and choose a model to compare

Write down the workflow your team currently uses, including its agent, model, provider, and billing arrangement. That is your baseline. Before running anything, decide what would make a switch worthwhile, including how much review effort your team will accept.

Separate these three types of costs:

1. Metered API usage grows with the requests you make. Include retries and fallback when comparing it. Save the applicable model pricing and provider usage report, including cached input, alongside each estimate.
2. Subscriptions and included credits work differently. Using fewer tokens inside an already-paid subscription may leave the bill unchanged. Note whether you want to avoid overages, reduce paid credits, or change subscription spending.
3. Developer time includes reviewing output, explaining failures, and repairing mistakes. Record these minutes separately so your team can judge whether the saving is worth the effort.

Compare the incumbent with one model on Parasail. Shortlist candidates with the Model Selection guide and public benchmarks, then let your own tasks decide. Before changing your default, test a second low-cost open or closed model as well. Beating an expensive model doesn’t prove you’ve found the best cheap option.

Record the agent version and effort setting for both workflows, and note any differences in tools or billing.

Start with one task and two workflows. Use the same task and checks for any additional model. Once you know what you’re comparing, choose a task that will make the comparison useful.

2. Choose a recurring task and define an acceptable result

Choose a recurring task from your development queue that your team knows how to check. As the pilot grows, add routine work alongside cases that often need correction. Note where each task came from and what you left out. That record tells you which kinds of work your results apply to.

The three starting points below ask progressively more of the agent:

This walkthrough uses regression-test writing. You supply the production fix, and the agent writes tests that protect it. That setup measures test writing only. For a bug-repair pilot, start from the unfixed code and keep the known solution outside the agent’s workspace.

How we defined a python-dotenv test task

A python-dotenv fix corrected a command-line tool that treated an explicitly empty variable as a missing value. We supplied the fix and asked each workflow to write tests that protected the intended behavior.

These cases should behave differently:

• KEY= and KEY="" should succeed, print one newline, and exit with code 0.
• A missing key or a bare key without an equals sign should exit with code 1 and produce no output.
• A nonempty value should continue to work.

Use this prompt for your own regression-test task, replacing the bracketed fields:

Write regression tests for [fix or commit]. The fix is already applied. Expected behavior: [what should happen, including important edge cases]. Preserve production code and existing tests.

Add tests in [test file], following the repository’s conventions and using its real API.

Run the new tests against the fixed code and the original code at [commit before the fix]. They must pass with the fix and fail on the original because of the bug. Run [existing test command] too. Correct your tests until the checks pass or [time limit] is reached. Finish with a short report of the changes, test results, and remaining concerns.

Report input, output, and cached tokens, session cost, and elapsed time when available in the coding tool’s records, including corrections. Label estimates and mark missing figures “not recorded.” If final totals are only available after the session ends, tell me what to copy from the tool.

Decide how you’ll verify the output before either workflow starts. In this experiment, the submitted tests also had to detect deliberately broken variants that dropped the expected output or incorrectly accepted a bare key. Import errors, skipped tests, or unrelated failures didn’t count as detecting the bug.

A passing test run answers only part of the acceptance question. Our review criteria also covered readable assertions, realistic use of the public API, and coverage of the requested behavior. Write your own review criteria down before the runs too.

Save the filled-in prompt with your review criteria and stopping rules. Once the repository, dependencies, and checks are ready, you have a shared assignment both workflows can run.

3. Run both workflows under the same conditions and log every attempt

Give each workflow a clean copy of the same repository state, assignment, dependencies, and checks. In our test-writing example, that meant applying the same production fix while withholding the upstream new tests and later history. Keep those conditions consistent in your own pilot.

Let the agent read files, edit its tests, run them, inspect failures, and correct mistakes. Decide beforehand how much external feedback it can receive and when an unsuccessful attempt stops.

These regression experiments ran on October 16, 2026, using OpenCode 1.19.3 and low effort. Each model kept its native instructions and tools enabled. The results reflect each model together with its workflow configuration.

Record the setup you actually use. Our OpenCode and Parasail walkthrough covers connecting the two. It uses a different OpenCode version from our runs.

Treat effort as part of the configuration you test. The same effort label can mean different amounts of computation on different models. Choose the model and effort level from your task results. Parasail’s GLM-5.3 and Flash comparison provides broader cost and capability context.

Record cost, time, and outcome

Run the same filled-in prompt in each workflow’s fresh session. Keep one short note per run using the fields below.

Include retries and fallback in the cost and time totals. Save the prompt, output, and test results with your note. Write “not recorded” for missing measurements, and check estimates against provider billing when available.

Compare costs with this prompt

After reviewing the work, paste this prompt and your run notes into your coding tool. Identify the baseline and candidate, add your acceptance decisions and review minutes, and leave the hourly rate blank if you only want API costs.

Compare [baseline workflow] with [candidate workflow] using these run reports: [paste reports]. Human review: [accepted, rejected, or pending for each trial; review/repair minutes]. Optional hourly rate: [amount or leave blank].

Use a calculator or code. Compare the same tasks and number of trials. Return one table with total model cost, accepted tasks, model cost per accepted task, total and average elapsed time per trial, and total review/repair minutes.

Include failed attempts, corrections, and fallback. Count each charge once; don’t add session totals to their component calls. Keep fallback with its original workflow; count the task once if accepted. Separate successes without fallback from fallback-assisted successes.

Model cost per accepted task = all model spending ÷ accepted tasks. Show the candidate’s savings per accepted task: baseline cost minus candidate cost, and that difference ÷ baseline cost × 100. Show the arithmetic and label estimates.

If I supplied an hourly rate, also compare (model spending + review/repair minutes ÷ 60 × hourly rate) ÷ accepted tasks. Keep elapsed time separate from human work time.

Use recorded costs. If only tokens are supplied, ask for the applicable prices and cache breakdown before estimating. Missing values aren’t zero; passing tests aren’t human acceptance. Show known subtotals, flag missing data, and withhold affected comparisons. Don’t divide by zero.

End with a brief recommendation based on these tasks and any information still needed.

4. Compare the cost of accepted work and inspect the output

Use your run notes to compare what each workflow cost, how long it took, and whether its output met your standard. Our results below show how to read those measurements. We tested dotenv’s empty values, Requests’ temporary-file uploads, and Packaging’s mixed path separators. Each task got three fresh sessions, which gave each model nine sessions per comparison.

How GLM-5.3 and GLM-5.3 Flash compared with GPT-6.1 Sol

We ran GLM-5.3 Flash after the GPT-6.1 Sol and GLM-5.3 sessions. This is a staged comparison, rather than a simultaneous three-model trial.

Full GLM cost about 27% less than Sol and took 46% less time. Flash cost about 89% less than Sol and took 30% less time.

These results justify a pilot on bounded test-writing work. They stop short of a cost for merge-ready output. We didn’t measure developer acceptance or review time, and code review found weaknesses in some submissions.

The absolute API spend is also small. Across nine sessions, Flash saved $0.62 against Sol, or about seven cents per session. At $100 an hour, an engineer costs that much in under three seconds of review. The savings add up only on high-volume work that needs little review per run. Before you route anything, check whether another low-cost model does better on the same work.

How GLM-5.3 Flash compared with GPT-6 Luna and GPT-5.6 Luna

A second comparison tested GPT-6 Luna, GPT-5.6 Luna, and Flash on the same three exercises. Each model again got nine sessions, with model and task order rotated.

Flash cost about 45% less than GPT-5.6 Luna and finished 18% faster on average. Against GPT-6 Luna, Flash cost 44% more and took 15% longer.

A code review by Codex flagged three Flash submissions that passed the automated checks but needed substantive improvements. One set the upload filename explicitly, so its tests never checked how Requests infers the filename. The same limited review found no material issues in the nine GPT-6 Luna submissions. No human reviewed either set.

For these tasks, GPT-6 Luna offered the stronger overall result. Flash’s earlier savings against Sol support testing a cheaper workflow, while the Luna comparison shows why you should include an inexpensive proprietary alternative.

These three public Python fixes are a narrow sample. Costs include recorded cache usage and are usage-derived estimates, not reconciled invoices. Elapsed time includes agent execution and verification, but excludes preparation and human review. Checks covered selected behavior and existing tests, not full upstream suites. To understand those totals, trace one task’s sessions through the worked calculation below.

How to calculate cost per accepted task

In the first comparison, Flash’s three dotenv sessions cost a total estimated $0.01379435. All three passed the automated checks:

$0.01379435 ÷ 3 automated passes = approximately $0.004598 per automated pass.

On dotenv alone, Flash cost 93% less than Sol. In the second comparison, GPT-6 Luna cost less than Flash on dotenv, and Flash finished faster. Across all three tasks, GPT-6 Luna was faster on average. Check the results for the specific tasks you plan to route.

Because no one reviewed these submissions, our example stops at cost per automated pass. Your pilot should calculate cost per accepted task instead.

Model cost per accepted task = total model cost of every attempt and fallback ÷ accepted tasks.

Count unsuccessful trials in the numerator. If no task is accepted, report the total spend and zero accepted tasks. Cost per accepted task can’t be calculated in that case.

Suppose a $0.02 open-model attempt fails and the incumbent fixes it for $0.08. That trial costs $0.10 for one accepted task. Mark it as fallback-assisted. It doesn’t count as an open-model success.

Count a task as accepted only after it passes the checks and a reviewer approves it.

5. Decide which work to route to an open model

Turn your results into a decision about a specific category of your team’s work. Use the costs, time, and reviewed output you recorded to decide what to route and when to escalate. The closed-to-open migration guide puts that task-by-task decision in a broader context.

If the Parasail workflow meets your review criteria at a useful saving, expand it to more tasks in that category. If a low-cost closed model does better, record that result and test it the same way. If review or repair erases the saving, keep your current workflow.

Write the decision down as a rule with a clear escalation point. A starting rule for tasks like our dotenv example looks like this.

Trial this workflow on regression tests for small, already-reviewed fixes. Require the test to catch the original bug, pass the fixed code, and satisfy normal review. Allow one feedback round, then escalate to the incumbent. Record all attempts and review minutes.

Adjust the scope and the feedback limit once your own results come in.

Choose further tasks from your team’s work. For CI classification, include ambiguous logs and infrastructure failures, compare a simple rules baseline, and validate the required output format. Evaluate it separately from repository editing. Report cache reuse when repeating identical logs.

A first pilot can be completed within a week. Spend day one choosing tasks and checks, then run the workflows over the next two days. Review output and accounting on day four, and make a limited decision on day five. Allow more time when the tasks or the review need it.

Commit the working configuration, agent instructions, and verification steps to the repository. Revisit the decision whenever the model, provider, agent, or workload changes materially.

Run your pilot on Parasail

Parasail’s serverless endpoints handle a pilot at up to 500 requests per minute. Elastic Endpoints and Dedicated deployments remove that limit once a route runs at volume. Going direct to Parasail also performs better than reaching the same model through a public gateway.

Start with one recurring task your team already knows how to check, then browse the models available on Parasail to choose your first candidate.