Skip to content

fix: preserve unknown response usage details in responses converters - #2583

Merged
ananthsub merged 1 commit into
NVIDIA-NeMo:mainfrom
ananthsub:ananthsub/fix-usage-fabrication
Aug 18, 2026
Merged

ananthsub merged 1 commit into
NVIDIA-NeMo:mainfrom
ananthsub:ananthsub/fix-usage-fabrication

Conversation

@ananthsub

@ananthsub ananthsub commented Aug 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Preserve absent cache and reasoning token details as null instead of converting them to measured zeroes.
  • Propagate provider-reported usage details through Chat Completions conversion, Anthropic conversion, and LiteLLM normalization.
  • Keep aggregated detail counts unknown when any contributing count is unknown, preventing observability from reporting fabricated cache misses.

@ananthsub
ananthsub requested review from Glorf and ffrujeri August 17, 2026 23:38
@ananthsub ananthsub changed the title fix: preserve unknown response usage details fix: preserve unknown response usage details in converter Aug 17, 2026
@ananthsub ananthsub changed the title fix: preserve unknown response usage details in converter fix: preserve unknown response usage details in responses converters Aug 17, 2026
Keep absent cache and reasoning token measurements as unknown so observability does not report fabricated cache misses or token counts.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@ananthsub
ananthsub force-pushed the ananthsub/fix-usage-fabrication branch from dc28489 to 3dc01fb Compare August 18, 2026 18:23
@copy-pr-bot

copy-pr-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ananthsub
ananthsub enabled auto-merge (squash) August 18, 2026 18:25
@github-actions

Copy link
Copy Markdown
Contributor

@ananthsub
ananthsub merged commit b1e0f67 into NVIDIA-NeMo:main Aug 18, 2026
19 of 22 checks passed
@ananthsub
ananthsub deleted the ananthsub/fix-usage-fabrication branch August 18, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants