One AI Outcome. Many Runtime Operations. What Actually Belongs to Its Cost?

A customer asks an AI research product to do something apparently simple:

Generate one research report.

From the customer's perspective, the unit of value is clear.

One request goes in.

One research report comes out.

Commercially, the product might deduct 10 credits for the completed report.

But neither of those views tells us everything that happened while producing it.

Underneath Research Report #123, the runtime might have looked more like this:

• Research Report #123
  • planning
  • model_call_01
  • retrieval
  • search_01
  • tool_call_01
    • FAILED
  • tool_call_02
    • RETRY
  • model_call_02
  • validation_01
    • FAILED
  • model_call_03
    • FALLBACK
  • validation_02
    • SUCCESS

The customer still receives one report.

The commercial system can still correctly deduct 10 credits.

But economically, the product performed much more than one operation.

Some of that work succeeded. Some failed. Some existed only because earlier work failed. Some may have consumed model tokens; other steps may have invoked paid search, external APIs, retrieval infrastructure, or other resources.

Now ask what sounds like a straightforward question:

How much did Research Report #123 cost to deliver?

It is tempting to start with arithmetic.

Add the model cost. Add the search calls. Add the tools. Add the retrieval cost. Add whatever other variable resources were consumed.

But before we can add the numbers, we need to know which numbers belong in the calculation.

Does the failed tool call belong to the cost of Report #123?

What about the retry?

Does the failed validation belong to the report even though it produced no customer-visible value?

If the fallback model ran only because an earlier attempt failed, does its cost belong to the same unit of work?

What if some work continued asynchronously?

What if some underlying work was shared with another report?

The difficult part is no longer arithmetic.

It is deciding what work belongs inside the boundary of the thing we are trying to measure.

Cost is arithmetic. Attribution starts with a boundary.

An API call is evidence of work, not the unit of economics

Suppose we have excellent provider data.

For every model request we know which customer triggered it, which model handled it, how many tokens it consumed, when it occurred, and how much it cost.

That is valuable economic evidence.

But a provider call is still one technical operation.

It does not automatically tell us what economically meaningful unit that operation belonged to.

One customer action can trigger many runtime operations:

• Customer action
  • Generate Research Report #123
  • Runtime work
    • model call
    • retrieval
    • search
    • tool call
    • retry
    • fallback model
    • validation

And not every piece of runtime work has to map cleanly to exactly one customer action.

A background operation may continue after the original request has returned. A child workflow may perform work on behalf of a larger workflow. A shared computation may contribute to more than one result.

So a technical event and an economically meaningful unit of work are not interchangeable concepts.

Aggregating provider calls can still answer useful questions:

How much did Customer A consume this month?

or:

How much did we spend on Model X?

But those answers do not necessarily tell us what work produced a specific customer outcome.

Suppose the unit we care about economically is:

one delivered Research Report.

We now have a different question:

Which runtime work belonged to the delivery of Research Report #123?

That is the boundary we need to reason about.

Draw the boundary around the work

For a simple synchronous operation, the answer may be easy.

One request starts. One operation runs. One result returns.

The technical boundary and the economic boundary may line up closely enough that there is little ambiguity.

But consider Report #123 again:

• Research Report #123
  • attempt_01
    • model call
    • tool call
      • FAILED
  • attempt_02
    • model call
    • retrieval
    • validation
      • FAILED
  • attempt_03
    • fallback model
    • validation
      • SUCCESS

If the question is:

How much work did the product perform to successfully deliver Report #123?

then the successful final attempt is not necessarily the whole story.

The first failed attempt consumed resources.

The second failed attempt consumed resources.

The fallback existed because the earlier path did not succeed.

A reasonable analytical boundary for this particular question might therefore include all three attempts:

• ECONOMIC BOUNDARY
  • attempt_01
    • FAILED
  • attempt_02
    • FAILED
  • attempt_03
    • SUCCESS
  • Research Report #123

But the important word is might.

A retry might belong to the same unit in one product. In another system, what looks technically like a retry could represent separately meaningful work.

A child operation might exist exclusively to produce Report #123. Another might create an artifact reused by several reports.

The goal is not to discover a universal rule saying:

Every event downstream of a customer request belongs to that customer's outcome.

The goal is to make the boundary defensible.

We should be able to explain why particular pieces of runtime work are being considered part of the economics of the unit we are analyzing.

Retries show why this distinction matters so quickly.

Because failed technical work does not necessarily disappear when the final outcome succeeds.

Failed work does not disappear economically

Suppose Report #123 required three attempts.

For simplicity, imagine that we can directly observe the variable resource cost associated with each attempt:

• Research Report #123
  • attempt_01
    • FAILED · $0.03
  • attempt_02
    • FAILED · $0.04
  • attempt_03
    • SUCCESS · $0.08

If we looked only at the successful attempt, we might describe the report as having $0.08 of illustrated execution cost.

But all three attempts consumed $0.15 of resources:

$0.03 + $0.04 + $0.08 = $0.15

The failed attempts did not produce additional customer-visible value.

They still consumed economically relevant resources.

Technical failure does not automatically erase the resources consumed by the failed attempt.

Whether the full $0.15 should be attributed to Report #123 depends on the economic boundary we have chosen.

If all three attempts were performed solely in the process of delivering that report, including them may be a reasonable representation of the direct execution resources consumed to produce it.

But this does not imply that the customer should be charged for three attempts.

The product might still deduct exactly 10 credits:

• Economic view
  • attempt_01
    • $0.03
  • attempt_02
    • $0.04
  • attempt_03
    • $0.08
  • Illustrated attributed execution cost
    • $0.15
• Commercial view
  • Research Report #123
    • 10 credits

Both views can be correct.

They answer different questions.

Economic attribution asks what resource consumption or cost belongs to the unit being analyzed.

Commercial charging asks what the customer should pay or consume under the product's commercial rules.

A product may deliberately absorb the economics of retries instead of exposing them commercially. The customer may be buying a completed report, not every internal attempt required to produce it.

The important point is not which charging policy is correct.

It is that the charging policy does not determine whether the failed work economically occurred.

Economic work can exist without becoming a separate commercial charge.

This becomes particularly relevant in AI workflows because one customer-visible task can contain model generations, retrieval, tools, validations, retries, handoffs, and fallbacks underneath it.

A retry can trigger another model call. A fallback can move execution to a more expensive model. A failed tool invocation may still incur an external API cost. A validation failure can trigger additional generation.

If we erase failed work simply because it failed technically, we may understate the resources consumed to produce the successful result.

But automatically assigning every failed or downstream operation to the final outcome can create the opposite mistake.

The boundary still has to be justified.

One request is not always the boundary

The examples so far have been convenient: one customer action eventually converges on one report.

Real workflows can be less cooperative.

A request might return immediately while the economically relevant work continues asynchronously:

• Customer request
  • 202 Accepted
    • background workflow
      • retrieval
      • model
      • tool
      • retry
      • final report

If request lifetime were our boundary, much of the work that produced the report would sit outside it.

The reverse problem also exists: some work may contribute to more than one outcome.

• shared work
  • retrieval / index
    • Report A
    • Report B
    • Report C

Which report owns the cost?

There may be no single objectively correct answer.

We could assign it to one report, divide it equally, allocate it according to measured usage, or leave it classified as shared.

Shared technology costs create the same fundamental allocation problem elsewhere in infrastructure economics: FinOps practices explicitly distinguish directly assignable costs from shared costs that require an apportionment strategy. [FinOps Foundation, Allocation]

This is where simply "following the calls" stops being sufficient.

Technical relationships can tell us a great deal about what happened.

They do not automatically decide how shared economic work should be attributed.

Some economic relationships can be observed directly. Others require allocation rules or declared assumptions.

A useful boundary therefore does not have to pretend uncertainty has disappeared.

It needs to make clear what was observed, what was related directly, and where an attribution decision was required.

Aggregation is not reconstruction

Imagine we have done a good job collecting provider usage and cost data.

At the end of the month, we can say:

• Customer ACME
  • Model usage
    • $41
  • Search
    • $18
  • External APIs
    • $14
  • Total
    • $73

That is useful.

It tells us how much direct provider cost accumulated under ACME during the period, within the cost boundary represented by those sources.

We can compare customers, observe trends, see which providers account for most of the spend, and detect that ACME became more expensive.

Aggregation answers an important question:

How much accumulated here?

But suppose the $73 covers twenty Research Reports, several failed workflows, background work, retries, and one shared retrieval operation.

Now ask:

Why was Report #123 more expensive than Report #122?

The monthly aggregate cannot answer that by itself.

We need to move from totals back toward the work that produced them.

For Report #123, we may want to establish something closer to:

• Report #123
  • attempt_01
    • model
    • tool
      • FAILED
  • attempt_02
    • model
    • search
    • validation
      • FAILED
  • attempt_03
    • fallback model
    • validation
      • SUCCESS

We are now trying to reconstruct which pieces of runtime work belonged together.

That gives us three related but different questions:

• AGGREGATION
  • How much accumulated?
• RECONSTRUCTION
  • What work belonged together?
• ATTRIBUTION
  • What economic evidence belongs to the unit we are analyzing?

These are not competing approaches, nor do they have to form a rigid implementation pipeline.

They separate different reasoning problems.

Aggregation can tell us how much accumulated.

Reconstruction can recover the structure of the work we want to investigate.

Attribution can associate relevant economic evidence with the unit we are analyzing, according to the boundary and assumptions we have chosen.

And reconstruction does not magically solve attribution.

Knowing that a shared retrieval operation contributed to Reports A, B, and C gives us important structural evidence. It still does not tell us whether its cost should be divided equally, allocated according to usage, retained as shared, or handled under another policy.

The structure informs the decision.

It does not make every economic decision for us.

Reconstruction requires relationships, not just events

Suppose our logs contain every operation from Report #123.

Nothing is missing.

But imagine the records look like this:

• 09:14:02 · model call
• 09:14:04 · tool call
  • FAILED
• 09:14:07 · model call
• 09:14:09 · search
• 09:14:12 · validation
  • FAILED
• 09:14:15 · model call
• 09:14:19 · validation
  • SUCCESS

We know what happened during those seventeen seconds.

Do we know that all seven events belong to Report #123?

Perhaps.

If Report #123 was the only workflow running for that customer, temporal proximity may give us a strong clue.

Now imagine thousands of workflows executing concurrently.

Timestamps alone become much less informative.

Customer identity helps, but it may still be too broad. A request ID may help, until the original request launches asynchronous work that continues under different technical operations.

What reconstruction needs is not simply more events.

It needs enough identity and relationship information to establish how those events fit together.

• IDENTITY
  • What piece of work is this?
• RELATIONSHIP
  • What larger piece of work does it belong to?
• EVIDENCE
  • What happened and what resources did it consume?

The implementation can vary.

One system might use workflow and attempt IDs. Another might use trace and span relationships. Another might preserve parent/child execution identifiers or attach business-level outcome identifiers to runtime events.

The engineering principle is more general:

If independent runtime events cannot be related back to the unit of work being analyzed, later reconstruction becomes harder.

That does not make reconstruction impossible.

Provider logs, timestamps, customer identity, workflow metadata, or other signals may allow relationships to be recovered later.

But instead of following an explicitly preserved relationship, we may now be inferring one.

• EXPLICIT RELATIONSHIP
  • Report #123
    • attempt_01
    • attempt_02
    • attempt_03
• INFERRED RELATIONSHIP
  • Maybe Report #123?
  • 09:14:02 · model call
  • 09:14:04 · tool failure
  • 09:14:07 · model call
  • 09:14:09 · search
  • 09:14:12 · validation

Both can contribute to analysis.

They do not provide the same support for the claim we eventually want to make.

When execution relationships are not preserved explicitly, later economic reconstruction may have to rely more heavily on inference.

This is why economic evidence is more than a collection of cost numbers.

The relationships between those numbers can matter too.

A trace can help reconstruct the work without defining its economics

Distributed tracing gives us a useful example.

Context propagation allows related operations to remain connected across service or process boundaries. Traces and spans can then represent technical relationships between those operations. [OpenTelemetry, Context Propagation]

In AI systems, tracing can expose structures such as model generations, tool calls, handoffs, guardrails, and other runtime operations. The OpenAI Agents SDK, for example, represents an end-to-end workflow as a trace composed of spans, with trace IDs and parent relationships; it also supports grouping multiple traces under a shared logical identifier. [OpenAI Agents SDK, Tracing]

Conceptually:

• trace
  • model generation
  • tool call
    • external API
  • model generation
  • validation

That structure can be extremely valuable for economic reconstruction.

If the trace corresponds closely to Report #123, much of the question what work belonged together? may already have a strong technical answer.

Observability systems can bring the two views even closer. LangSmith, for example, can associate token usage and costs with individual runs, aggregate costs through a trace tree, and attach custom costs to non-LLM operations such as tools or retrieval. [LangSmith, Cost tracking]

So the point is not that tracing is somehow separate from economic analysis.

Technical lineage may provide some of the strongest evidence available for it.

But there is still a distinction worth preserving.

A trace describes technical execution according to the boundaries and relationships represented by the tracing system.

The economic analysis asks which work belongs to the unit we care about and how its economic evidence should be treated.

Sometimes those boundaries line up almost perfectly.

Sometimes they do not.

A long-lived customer workflow may span several traces. Several technical workflows may contribute to one customer outcome. Shared work may contribute to multiple units. Even when the technical relationship is perfectly known, an allocation policy may still be required.

That leads to a more precise principle:

Technical lineage can provide evidence for economic attribution. It does not automatically define the attribution policy.

Tracing can help tell us what happened and how operations were related technically.

Economic attribution can use those relationships while still asking a different question:

Given the unit we are analyzing, what economic evidence should be associated with it?

Reconstruction does not eliminate uncertainty

It would be convenient if every workflow produced a complete tree of runtime work, every operation carried perfect identity, every provider cost arrived immediately, and every shared resource had one obvious owner.

Real systems do not always give us that.

Some evidence arrives late. Some provider data may be corrected. Some operations may lack correlation context. Some infrastructure costs may be shared across thousands of executions. Some relationships may have to be reconstructed after the fact.

And some attribution decisions depend on policy rather than an observable technical fact.

For a shared retrieval operation used by Reports A, B, and C, we may know with high confidence that all three reports used it.

That is evidence about execution structure.

It still does not create a uniquely correct answer to:

How much of this cost belongs to Report B?

The answer may depend on the purpose and boundary of the analysis.

A team might allocate the cost evenly. Another might allocate according to measured usage. Another might retain it as shared infrastructure when calculating direct execution cost and include it only in a broader cost model.

Those approaches produce different numbers because they encode different assumptions.

The important requirement is not to make the assumption disappear.

It is to avoid presenting an assumption as if it were an observed fact.

Conceptually, an economic reconstruction may therefore contain different kinds of support:

• OBSERVED
  • This model call occurred.
  • This tool call failed.
  • This retry consumed these resources.
• RELATED
  • These operations were explicitly linked to the same workflow.
• ALLOCATED
  • This shared cost was assigned according to a declared rule.
• INFERRED
  • This event was associated after the fact using incomplete evidence.

The labels themselves are not the point.

The distinction is.

Not every economically useful number has the same evidentiary status.

That does not make estimates or allocations invalid. It means they should remain identifiable as estimates or allocations.

A defensible economic reconstruction does not require pretending that uncertainty has disappeared.

It requires enough evidence, relationships, and declared assumptions to understand how the result was produced.

Why reconstruction matters when economics change

Now return to a different question:

Why did this workflow become more expensive?

Suppose ACME's Research Reports show:

• August
  • Average illustrated execution cost
    • per Research Report: $0.20
• September
  • Average illustrated execution cost
    • per Research Report: $0.31

The change tells us something happened economically.

It does not tell us why.

Perhaps provider rates changed.

Perhaps customers requested different work.

Perhaps commercial conditions changed.

Or perhaps execution behavior changed.

Assume the investigation has ruled out the relevant rate and commercial changes and we want to test an execution-driven explanation.

A representative August report might show:

• Report #081
• model
• tool
• validation
• SUCCESS

While comparable September reports might show:

• Report #123
• model
• tool
• FAILED
• retry
• larger-model fallback
• validation
• SUCCESS

That difference is evidence consistent with an execution-driven explanation.

But it does not become causal proof merely because the September workflow contains more steps.

We still need to establish that the compared units are meaningful, that the additional work belongs to them, that the relevant economic evidence supports the cost difference, and that competing explanations have not been mistaken for execution effects.

The reasoning looks something like this:

• ECONOMICS CHANGED
• EXECUTION-DRIVEN?
• WHICH UNIT OF WORK?
• WHAT WORK BELONGED TO IT?
• WHAT CHANGED INSIDE THAT WORK?
• WHAT EVIDENCE SUPPORTS THE EXPLANATION?

This is why reconstruction matters for economic explanation.

Seeing that Report #123 cost more tells us what happened economically.

Reconstructing its runtime work helps establish what happened underneath.

Comparing that structure with other relevant executions can then provide evidence for why the economics changed.

Without that discipline, explanation can collapse into correlation.

We might see more provider spend and assume more customer usage.

We might see a larger-model call and assume routing caused the increase.

We might see several events close together in time and assume they belonged to the same workflow.

Each explanation can sound plausible.

The question is how much evidence actually supports it.

One outcome can hide more work than the customer ever sees

Return to Research Report #123 one final time.

The customer saw:

• 1 Research Report

The commercial system saw:

• 10 credits

A provider-level view might see:

• 3 model calls
• 1 search
• 2 tool calls
• 2 validations

And the runtime might know that those operations formed:

• Report #123
  • failed attempt
  • retry
  • failed validation
  • fallback
  • successful delivery

None of these views is inherently wrong.

They describe different aspects of the same product behavior.

The economic question appears when we ask which runtime facts belong to the unit whose economics we want to understand.

That boundary may be simple.

It may follow an existing workflow or trace almost exactly.

Or it may require relationships across asynchronous work, allocation of shared resources, or declared assumptions where the evidence is incomplete.

The goal is not perfect attribution.

The goal is to know what supports the number we are using.

A useful reasoning progression is:

• AGGREGATION
  • How much accumulated?
• RECONSTRUCTION
  • What work belonged together?
• ATTRIBUTION
  • What economic evidence belongs to the unit being analyzed?
• EXPLANATION
  • What changed underneath, and what evidence supports it?

This is not a mandatory implementation pipeline.

It separates questions that are easy to collapse into one another.

Aggregation can be correct without reconstruction.

Reconstruction can be accurate without resolving every attribution decision.

Attribution can contain declared assumptions.

And explanation should not claim more than the evidence underneath it can support.

The deeper engineering principle is therefore not that every AI workflow needs one universal definition of an execution.

It is almost the opposite.

Different products can have different economically meaningful boundaries.

What matters is whether the system preserves enough structure to reason about the boundary it actually cares about.

Because before asking:

Why did this AI workflow become more expensive?

there is a more fundamental question:

Can you reconstruct what work actually belonged to it?

References

• OpenTelemetry — Context Propagation
• OpenAI Agents SDK — Tracing
• LangSmith — Cost tracking
• FinOps Foundation — Allocation