Your AI Workflow Failed. The Cost Didn't Disappear.

A customer asks an AI research product to generate one report.

The workflow starts normally.

Planning runs. A model generates an initial response. Search retrieves external information. A tool processes part of the result.

Then validation fails.

Research Report #123

Attempt 1

Technically, the attempt failed.

No report was delivered from it.

But the work performed before the failure did not disappear. The model was called, search ran, the tool executed, and resources were consumed.

In this simplified example, we can directly observe $0.16 of resource consumption before validation rejected the attempt.

So what does FAILED actually tell us?

It tells us something important about the technical result of the attempt. It does not, by itself, tell us that no economically relevant work occurred.

FAILED ≠ FREE.

That does not mean every failed AI request necessarily incurs provider charges.

A request can fail before meaningful execution begins. Different providers may handle different failure modes differently. A timeout observed by the caller may also leave uncertainty about how much work occurred upstream.

The point is narrower:

A failed technical state does not, by itself, tell us whether economically relevant resources were consumed.

If resources were already consumed, a later FAILED state does not reverse that consumption.

And that creates a more interesting problem than simply asking whether failures cost money.

Suppose the runtime retries Report #123.

The second attempt succeeds.

The customer eventually receives exactly the report they requested.

What happens economically to the work performed by the first attempt?

One successful outcome can contain unsuccessful work

After validation rejects Attempt 1, the runtime retries the work.

This time the path succeeds:

From the customer's perspective, the final result is simple:

1 Research Report

From the runtime's perspective, however, that successful delivery contains two attempts.

One failed. One succeeded. Both consumed resources.

If we look only at the successful attempt, we see $0.18.

If we look at the observed resources consumed across both attempts, we see $0.34.

Attempt 1     FAILED      $0.16\nAttempt 2     SUCCESS     $0.18\nAcross attempts           $0.34

Which number represents the economics of Report #123?

The tempting answer is to choose one.

But that is too early.

$0.18 tells us how much observed resource consumption belonged to the successful attempt.

$0.34 tells us how much observed resource consumption occurred across the two attempts associated with delivering the report.

And $0.16 tells us how much occurred in the unsuccessful attempt.

These numbers are not necessarily competing estimates of one supposedly "true" cost.

They can answer different questions.

So the deeper question is no longer simply whether the failed attempt consumed resources. In this example, we already know that it did.

The more interesting question is:

Where should the economics of that unsuccessful work appear in our analysis?

Same evidence. Different economic questions.

Suppose our runtime evidence is sufficient to establish:

Outcome #123\n\nAttempt 1     FAILED      $0.16\nAttempt 2     SUCCESS     $0.18

Those facts do not change when we change the question.

But the economic view we need can.

If we want to know how much the successful attempt consumed, the relevant number is $0.18.

If we want to know how much observed resource consumption occurred across the attempts associated with delivering Report #123, the answer in this simplified example is $0.34.

And if we want to isolate the observed resource consumption of unsuccessful attempts, the answer is $0.16.

Same execution. Same evidence. Different economic questions.

The mistake would be to take one of these numbers and declare it universally correct.

The successful-attempt view describes the execution path that actually completed, but excludes resources consumed by the earlier attempt.

The across-attempts view captures more of the work associated with reaching delivery, but that does not automatically make every dollar in that view part of a universal definition of "report cost."

The unsuccessful-attempt view isolates resources consumed by work that did not directly complete the outcome, but it does not tell us whether that work was necessary, avoidable, useful, or wasteful.

The same execution evidence can support different economic views because those views answer different questions.

Imagine an AI product that measures only the resource consumption of successful attempts.

Its numbers may be perfectly correct for those attempts while still excluding economically relevant work performed during retries, rejected outputs, fallbacks, or other unsuccessful paths.

The opposite approach can also mislead. Automatically assigning every failed operation to the eventual successful outcome assumes an economic relationship that the technical status alone does not establish.

Some work may belong to a different unit. Some may be shared. Some may have occurred after cancellation. Some relationships may remain uncertain.

An analysis may also intentionally preserve unsuccessful work separately because the question is not:

What did delivery consume?

but:

How much are unsuccessful executions costing us?

Economic attribution therefore cannot begin with a number alone.

It begins with the question the number is supposed to answer.

Economic attribution depends partly on what we are trying to understand.

That distinction becomes especially important when the additional work comes from retries.

Because a retry can restore correct application behavior while leaving a different economic history underneath.

The retry fixed the execution. It didn't erase the first attempt.

Retries create an important separation between technical correctness and economic behavior.

Imagine a slightly different failure.

The runtime sends a request to an upstream service. The caller waits.

Then it times out.

From the caller's perspective, the operation did not complete successfully.

But a timeout does not necessarily tell us what happened upstream.

The service may not have started the work. It may have started and failed. Or it may have completed the work while the response never reached the caller.

Suppose the application retries:

At the application level, we want the retry to leave the system in the intended state.

This is where mechanisms such as idempotency matter.

If the same logical operation is delivered more than once, an idempotent design can help prevent repeated delivery from creating duplicate effects.

But that answers a correctness question.

It does not, by itself, answer the economic one.

Those questions can have different answers.

A system might correctly produce one final business result, record one commercial consumption event, and avoid creating the same resource twice.

And yet more than one upstream operation may have occurred while the system reached that state.

That does not mean every retry duplicates cost.

Sometimes the first attempt never begins meaningful execution. Sometimes the retry is deduplicated before expensive work occurs. Sometimes only part of the first attempt executes.

And sometimes both attempts consume economically relevant resources.

Application correctness alone cannot tell us which case occurred.

Idempotency can protect state from duplicate effects without, by itself, proving that no additional upstream work occurred.

It is therefore possible to investigate a retry and conclude:

Application state       CORRECT\nCommercial state        CORRECT\nCustomer outcome        DELIVERED

while still asking:

Economic evidence\n\nAttempt 1    ?\nAttempt 2    $0.18

The ? matters.

If Attempt 1 performed no meaningful work, there may be little or nothing economically relevant to account for.

If it consumed model inference, tool execution, external API usage, or other resources before the timeout, those resources remain part of the execution history even though the retry restored the intended result.

And when we know that the first attempt did consume resources, the principle becomes simple:

The retry fixed the execution. It didn't erase the first attempt.

But knowing that unsuccessful work consumed resources still does not tell us whether that work was economically wasteful.

A failed attempt can be expensive without being useless.

Failed doesn't mean wasted

Once we start measuring unsuccessful work, there is another tempting shortcut:

That conclusion is too strong.

Consider a workflow that generates several candidates and validates them before choosing one.

Some candidates may be rejected. A search path may return nothing useful. A validation step may discover that an otherwise complete output cannot be delivered. A fallback may run because the preferred path could not produce an acceptable result.

In each case, work occurred without directly becoming the final successful output.

But that does not automatically make the work avoidable.

Sometimes unsuccessful work is part of how the system reaches an acceptable result. Sometimes it provides information that changes the next step. Sometimes redundancy is intentional because reliability is worth its additional cost.

And sometimes the extra work really is unnecessary rework caused by poor orchestration, duplicate execution, an application bug, or an overly aggressive retry policy.

The technical status alone does not distinguish between these cases.

This is not intended as a formal taxonomy.

It illustrates a more important distinction:

FAILED ≠ USELESS.

And economically:

FAILED COST ≠ AVOIDABLE COST.

Suppose two workflows each consume an additional $0.10 before producing a successful outcome.

In Workflow A, the additional work is a fallback that recovers from a transient provider failure.

In Workflow B, the additional work is a duplicate model call caused by an orchestration bug.

The additional resource consumption is $0.10 in both cases.

Its interpretation is not.

This separates two questions that are easy to collapse:

Did the unsuccessful work consume resources?

and:

Was that additional work necessary, informative, intentionally redundant, or avoidable?

The first is primarily a question of evidence.

The second requires explanation.

Finding failed work is therefore not enough. We need to understand why the work happened before deciding whether it should be eliminated.

Economic cost is not the customer's bill

Return to Research Report #123.

The customer asked for one report.

The first attempt failed after consuming $0.16.

The second succeeded after consuming $0.18.

Suppose the product's commercial model is simple:

1 completed report\n=\n10 credits

What should the customer consume?

Nothing about the economic evidence alone answers that question.

The product may intentionally charge only for the completed report. It may absorb retries, validation failures, provider variance, fallbacks, and other internal execution behavior behind the commercial abstraction.

So three views can coexist:

There is no inherent inconsistency here.

The technical view describes what happened during execution.

The economic view describes the resources consumed underneath it.

The commercial view describes what the product chooses to expose, allocate, or charge according to its commercial rules.

COST INCURRED ≠ CUSTOMER SHOULD BE CHARGED.

If a product promises one completed report for 10 credits, absorbing the economics of internal recovery may be exactly what that commercial abstraction is designed to do.

That does not make the unsuccessful work economically irrelevant.

It means the business, rather than the customer, may be absorbing its variability.

The customer does not need a line item for every retry, rejected candidate, or fallback.

But the business may still need to understand them.

Otherwise, stable commercial consumption can hide changing execution economics underneath.

Same successful outcomes. Different work underneath.

Now scale the problem beyond one report.

Imagine two customers using the same AI product during the same period.

Both generate 100 successful research reports, are on the same commercial plan, and consume the same number of credits per completed report.

But their runtime histories look different.

The successful-outcome count is identical.

The execution burden underneath it is not.

That does not tell us that Customer B is less profitable.

We would need more evidence to reach that conclusion: revenue, applicable rates, resource mix, execution paths, shared costs, and potentially other variables.

Nor does the attempt count tell us why Customer B required more work.

The difference might come from input complexity, model routing, tool reliability, provider behavior, validation, retries, application defects, or some combination of them.

But it exposes something that successful outcomes alone cannot show:

The same amount of delivered customer value can sit on top of different amounts of unsuccessful work.

The same pattern can appear over time.

From the customer's perspective, very little may have changed.

Commercial usage may also appear stable if customers consume the same credits per report.

But if those additional attempts consume economically relevant resources, they can contribute to an execution-driven change in the economics of delivery.

That still does not explain the change by itself.

More attempts are evidence of different execution behavior, not proof of its cause or full economic impact.

We would still need to understand what happened inside those attempts, which resources they consumed, why they occurred, and whether other economic variables changed at the same time.

Successful delivery alone would have hidden the difference.

This is why failure economics matters beyond reliability.

A system can continue producing the expected customer outcomes while the amount of work required to produce them changes underneath.

Success is the next boundary

Throughout this Guide, we have treated FAILED and SUCCESS as if their meaning were obvious.

That simplification was useful. It allowed us to ask what happens economically when unsuccessful work consumes resources before an eventual successful delivery.

But real AI workflows introduce another problem.

What exactly does SUCCESS mean?

A model call can complete successfully while producing an unusable answer.

A workflow can reach its final technical state and still fail validation.

An output can pass automated validation and later be rejected by a human.

A report can be generated correctly from the system's perspective while failing to produce the business result the customer actually wanted.

Technical completion, therefore, is not necessarily the same thing as a useful outcome.

For the reasoning in this Guide, we have assumed that the product has some criterion by which Report #123 is considered successfully delivered.

We have not defined that criterion.

And we do not need to yet.

What matters here is the distinction we can already establish:

Between those two states can sit retries, rejected candidates, fallbacks, validation failures, redundant execution, and recovery work.

Some of that work may be necessary.

Some may be avoidable.

Some may belong to the successful delivery for one analytical purpose and remain separate for another.

And none of those economic decisions can be derived from FAILED or SUCCESS alone.

Technical status tells us what happened to the execution. It does not, by itself, tell us how to interpret its economics.

There is one boundary left.

If we eventually want to compare economic consumption against successful outcomes, the denominator matters too.

Before asking how much unsuccessful work costs us per successful outcome, we need to answer a more fundamental question:

What exactly counts as a successful outcome?

References

The examples and dollar amounts in this Guide are illustrative. The following sources support the distributed-systems concepts discussed around timeouts, retries, and idempotency.

1. AWS Well-Architected Framework — Set client timeouts AWS explains that the exact cause of a timeout may not be known from the client side, and discusses how timeouts interact with remote execution, resource consumption, and retries.

2. AWS Well-Architected Framework — Control and limit retry calls AWS describes retries as a recovery mechanism that can consume additional server resources, and explains why retry behavior needs to be controlled to avoid compounding work and load.

3. AWS Well-Architected Framework — Make mutating operations idempotent AWS describes idempotency as a way to make repeated requests safe with respect to duplicate effects, including the use of idempotency tokens for retried operations.