Repository navigation
[Stirrup] Lift LLM kwargs from config and clear stale deliverables - #1187
Merged
bxyu-nvidia merged 2 commits intoApr 30, 2026
Merged
Conversation
Signed-off-by: Serge Panev <spanev@nvidia.com>
bxyu-nvidia
previously approved these changes
Apr 30, 2026
Signed-off-by: Serge Panev <spanev@nvidia.com>
bxyu-nvidia
approved these changes
Apr 30, 2026
cmunley1
pushed a commit
that referenced
this pull request
May 1, 2026
β¦1187) ## Summary Two issues surfaced while validating the agent against an RL training loop: ### 1. Stale deliverables leak across rollouts `responses_api_agents/stirrup_agent/app.py:347` does `task_dir.mkdir(parents=True, exist_ok=True)` and then copies the current run's output files into `task_dir`. It never clears the directory first, and only same-named files are overwritten by `shutil.copy2`. When an agent revisits the same `(task_id, rollout_index)` (e.g. across RL training steps that reuse `persist_deliverables_dir`), files from the prior visit linger in `repeat_<n>/`. The judge in `resources_servers/gdpval/app.py::_verify_rubric` calls `read_deliverable_files(body.deliverables_dir)` and `convert_deliverables_to_content_blocks(...)` against the whole directory, so stale artifacts from a prior step contaminate the reward signal. Fix: `shutil.rmtree(task_dir, ignore_errors=True)` immediately before the `mkdir`. Each visit gets a clean directory. ### 2. Hardcoded LLM sampling kwargs and per-call completion cap `nemo_client.DynamicMaxTokensChatCompletionsClient.generate` hardcoded `temperature=1.0`, `top_p=0.95`, `extra_body.chat_template_kwargs.enable_thinking=True`, and `_MAX_COMPLETION_TOKENS = 64000`. Although `temperature` is plumbed all the way through `_run_stirrup_agent`, the client never consumed it β every call ignored the configured value and ran at 1.0. The 64K cap on per-call `max_completion_tokens` similarly couldn't be tuned to match a training-side response-length budget. Fix: expose the four values as constructor params, with the historical hardcoded values as defaults, and surface them on `StirrupAgentWrapperConfig`: - `temperature` β already a config field; now actually reaches the client. - `top_p: float = 0.95` (new field) - `enable_thinking: bool = True` (new field) - `max_completion_tokens_cap: int = 64000` (new field) `top_p` also accepts a per-request override via `body.top_p`, mirroring how `temperature` and `max_output_tokens` already work. ## Test plan - [x] New unit tests in `responses_api_agents/stirrup_agent/tests/test_nemo_client.py` cover the kwargs landing on the wire request, the `max_completion_tokens_cap` ceiling, and the unchanged defaults. - [ ] CI green. --------- Signed-off-by: Serge Panev <spanev@nvidia.com> Signed-off-by: cmunley1 <cmunley@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two issues surfaced while validating the agent against an RL training loop:
1. Stale deliverables leak across rollouts
responses_api_agents/stirrup_agent/app.py:347doestask_dir.mkdir(parents=True, exist_ok=True)and then copies the current run's output files intotask_dir. It never clears the directory first, and only same-named files are overwritten byshutil.copy2. When an agent revisits the same(task_id, rollout_index)(e.g. across RL training steps that reusepersist_deliverables_dir), files from the prior visit linger inrepeat_<n>/. The judge inresources_servers/gdpval/app.py::_verify_rubriccallsread_deliverable_files(body.deliverables_dir)andconvert_deliverables_to_content_blocks(...)against the whole directory, so stale artifacts from a prior step contaminate the reward signal.Fix:
shutil.rmtree(task_dir, ignore_errors=True)immediately before themkdir. Each visit gets a clean directory.2. Hardcoded LLM sampling kwargs and per-call completion cap
nemo_client.DynamicMaxTokensChatCompletionsClient.generatehardcodedtemperature=1.0,top_p=0.95,extra_body.chat_template_kwargs.enable_thinking=True, and_MAX_COMPLETION_TOKENS = 64000. Althoughtemperatureis plumbed all the way through_run_stirrup_agent, the client never consumed it β every call ignored the configured value and ran at 1.0. The 64K cap on per-callmax_completion_tokenssimilarly couldn't be tuned to match a training-side response-length budget.Fix: expose the four values as constructor params, with the historical hardcoded values as defaults, and surface them on
StirrupAgentWrapperConfig:temperatureβ already a config field; now actually reaches the client.top_p: float = 0.95(new field)enable_thinking: bool = True(new field)max_completion_tokens_cap: int = 64000(new field)top_palso accepts a per-request override viabody.top_p, mirroring howtemperatureandmax_output_tokensalready work.Test plan
responses_api_agents/stirrup_agent/tests/test_nemo_client.pycover the kwargs landing on the wire request, themax_completion_tokens_capceiling, and the unchanged defaults.