[ananthsub]

What does this PR do?

Implements the built-in resources-backed single-agent-turn Environment Server protocol. The Environment Server assigns participant session IDs, registers cleanup before each seed, runs one Agent activation, verifies the response through Resources, and closes both participants.

The episode result is the Resources verify response itself, plus the Agent's session observations as ng_agent_observations. SingleAgentTurnResult subclasses BaseVerifyResponse and allows extra fields, so each Resources Server's benchmark-specific fields are preserved. A stored rollout record therefore has the same shape as an Agent's /run result, and rollout collection can store it without knowing its fields. This makes ResourcesVerifyResponse redundant, so it is removed.

Tool-grant session requests are serialized as JSON before dispatch so Pydantic URLs are strings and the kind/transport discriminators remain present. The Resources session closes only in final cleanup, which the episode deadline does not bound, so a slow or failed close is logged and cannot replace a verified result. Every downstream call in an episode, including cleanup, carries the attempt's rollout ID for observability. Connection failures, including a response body cut off mid-read, are retryable. Verification needs the Agent close response, which carries the Agent's observations and final Resources cookies, so a transient close failure makes the episode fail as retryable instead of retrying the close.

Single-agent-turn interaction flow

This diagram describes the built-in single-agent-turn use case. Other Environment Servers may coordinate different participants and use different request, result, and lifecycle flows.

sequenceDiagram
    participant C as RolloutCollector
    participant E as EnvironmentServer
    participant R as ResourcesServer
    participant A as AgentServer
    participant M as ModelServer

    C->>E: POST /run
    E->>R: Seed resources session
    R-->>E: Tool access and optional SandboxAccess
    E->>A: Seed agent session
    E->>A: One /v1/responses activation
    A->>M: Model requests
    A-->>E: Agent response
    E->>A: Close agent session
    A-->>E: Agent observations
    E->>R: Verify response
    Note over E,R: Final cleanup, outside the episode deadline
    E->>R: Close resources session
    E-->>C: Verify response plus agent observations

Loading

Stack position

Part 1 of 6. Next: #3724.

No issue is linked because this is one layer of the Environment Server stack.

Validation

• uv run pytest environment_servers/single_agent_turn/tests/test_app.py -q
• uv run pytest tests/unit_tests/test_resources_sessions.py tests/unit_tests/test_agent_sessions.py -q
• Production-shaped fake requests are encoded with orjson and validated with AgentSeedSessionRequest.model_validate_json.
• The tests check that the episode result has no nested verification and keeps benchmark-specific fields.
• The tests check that a slow Resources close keeps the verified result, that every downstream call carries the rollout ID, and that an interrupted response body is retryable.

Rollouts

See #3722 for real-model rollouts through this Environment Server.

Compatibility and benchmark impact

No rollout is redirected by default. Resources tool transports remain explicit opt-ins. Cleanup failures after a verified reward no longer turn that rollout into a terminal failure.

ResourcesVerifyResponse, added to nemo_gym.base_resources_server in #3544, is removed. Nothing outside this stack used it. A caller that parses a verify response without knowing its concrete type can declare extra="allow" on its own model, as SingleAgentTurnResult does.

[adil-a]

mentioning single_agent_turn feels a bit clunky because it's not clear whether it should be the top level item or under environment_servers. any way we can fix this confusion?

[ananthsub]

The top-level key is the instance name, which routes, _inherit_from, and server references use. The key under environment_servers is the implementation directory. The two usually match, which is where the confusion comes from. This follows the repo's config convention, so I added a comment to the template explaining both keys in 895d0af instead of renaming.

[yaoyu-33]

Could we leave this teardown to the registered final cleanup? It still runs under the episode deadline: if that expires here, an already verified reward becomes an Episode timed out failure, even when final cleanup succeeds.

[ananthsub]

Good catch. I removed the explicit close after verify, so the Resources session now closes only in final cleanup, outside the episode deadline. A new test checks that a slow close keeps the verified result. Fixed in 895d0af.

[yaoyu-33]

Could we bind rollout_context(request.episode_id.capture_key) for downstream calls? This route has no rollout prefix, and ServerClient's fallback does not recognize native episode_id, so verifier/judge model calls lose rollout correlation when observability is enabled.

[ananthsub]

Agreed. run_request now binds rollout_context(request.episode_id.capture_key) around the whole episode, including final cleanup, so the base class covers every Environment Server. A new test checks that each downstream call, including /verify and /close_session, carries the attempt's rollout ID. Fixed in 895d0af.

[yaoyu-33]

Could we also classify interrupted response-body reads as retryable? A mid-body disconnect raises ClientPayloadError, not ClientConnectionError, so this currently produces terminal=True and prevents another rollout attempt.

[ananthsub]

Yes, ClientPayloadError is now retryable, and the retry test covers it. Fixed in 895d0af.