Skip to content

feat: register environment server type - #3482

Merged
bxyu-nvidia merged 1 commit into
mainfrom
ananthsub/episode-orchestration-01-server-type
Sep 19, 2026
Merged

feat: register environment server type#3482
bxyu-nvidia merged 1 commit into
mainfrom
ananthsub/episode-orchestration-01-server-type

Conversation

@ananthsub

@ananthsub ananthsub commented Sep 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What does this PR do?

Registers environment_servers as a first-class Gym server type for configuration, discovery, startup, addressing, status, and telemetry.

Interaction flow

sequenceDiagram
    participant C as Rollout collector
    participant E as Environment Server
    participant R as Task / Resources Server
    participant A as Agent Server
    participant M as Model Server
    participant S as Task sandbox

    C->>E: POST /run
    E->>R: Seed task session
    R-->>E: Tool access and optional SandboxAccess
    E->>A: Seed agent session
    A->>S: Connect as borrower when supplied
    E->>A: Invoke /v1/responses
    A->>M: Model and capture requests
    A-->>E: Agent response
    E->>A: Close agent session
    E->>R: Verify response
    E->>R: Close task session
    E-->>C: Episode result
Loading

The Environment Server coordinates ordering and the final result. Agent and Task/Resources Servers retain ownership of the state and resources they create.

This is part 1 of 13 in the Environment Server stack. Previous: main. Next: #3483.

Validation

  • uv run pytest tests/unit_tests/test_global_config.py -q

Rollouts

See the final PR for end-to-end rollout evidence.

Compatibility and benchmark impact

Existing Agent Server routing remains available during migration.

@copy-pr-bot

copy-pr-bot Bot commented Sep 17, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@ananthsub
ananthsub added this pull request to stack #3493 September 17, 2026 16:31
@ananthsub ananthsub changed the title ananthsub/episode orchestration 01 server type feat: register episode processor server type Sep 17, 2026
@ananthsub ananthsub mentioned this pull request Sep 17, 2026
Merged
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-01-server-type branch 2 times, most recently from 650d8eb to d662430 Compare September 18, 2026 00:55
@ananthsub ananthsub changed the title feat: register episode processor server type feat: register environment server type Sep 18, 2026
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-01-server-type branch from d662430 to 7b74c39 Compare September 18, 2026 01:06
@ananthsub
ananthsub marked this pull request as ready for review September 18, 2026 19:45
@ananthsub
ananthsub requested a review from a team as a code owner September 18, 2026 19:45
@nemo-automation-bot
nemo-automation-bot Bot requested review from a team September 18, 2026 19:46
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-01-server-type branch from 7b74c39 to 7c30274 Compare September 18, 2026 20:44
@ananthsub
ananthsub removed this pull request from stack #3493 September 18, 2026 22:09
@ananthsub
ananthsub marked this pull request as draft September 18, 2026 22:09
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-01-server-type branch from e4d9429 to 7032f04 Compare September 18, 2026 22:42
@yaoyu-33 yaoyu-33 added area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation complexity:medium Single-domain change with interacting parts or a moderate review surface feature New capabilities, enhancements, or enablement work needs-review PR is ready for code review and waiting on a reviewer labels Sep 18, 2026
yaoyu-33
yaoyu-33 approved these changes Sep 19, 2026
View reviewed changes

@yaoyu-33 yaoyu-33 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for this server-type registration layer. Config/discovery tests and focused contract checks passed locally (184 tests).

@yaoyu-33 yaoyu-33 added ready-to-merge PR is approved, current, and waiting for final CI or merge and removed needs-review PR is ready for code review and waiting on a reviewer labels Sep 19, 2026
@ananthsub
feat: register environment server type
27cd6b1 
Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@ananthsub
ananthsub force-pushed the ananthsub/episode-orchestration-01-server-type branch from 7032f04 to 27cd6b1 Compare September 19, 2026 01:02
@nemo-automation-bot
nemo-automation-bot Bot requested a review from a team September 19, 2026 01:02
@bxyu-nvidia
bxyu-nvidia merged commit 664f595 into main Sep 19, 2026
42 checks passed
@bxyu-nvidia
bxyu-nvidia deleted the ananthsub/episode-orchestration-01-server-type branch September 19, 2026 04:07
ananthsub added a commit that referenced this pull request Sep 22, 2026
@ananthsub
feat: define core episode types (#3483)
bfbbd63 
## What does this PR do?

Adds only the shared episode identity and envelope in
`nemo_gym/episode_types.py`: `EpisodeId`, minimal `TaskId`,
`MaterializedTask`, base request/response, and handled failure.

`TaskId` contains only `taskset` and `task_id`. Taskset names use the
`<environment>:<taskset>` convention when a run blends tasksets.

## Single-agent interaction flow

This diagram shows the built-in single-agent use case. Other Environment
Server protocols may coordinate different participants and use different
request, result, and lifecycle flows.
```mermaid
sequenceDiagram
    participant C as Rollout collector
    participant E as Environment Server
    participant R as Task / Resources Server
    participant A as Agent Server
    participant M as Model Server
    participant S as Task sandbox

    C->>E: POST /run
    E->>R: Seed task session
    R-->>E: Tool access and optional SandboxAccess
    E->>A: Seed agent session
    A->>S: Connect as borrower when supplied
    E->>A: Invoke /v1/responses
    A->>M: Model and capture requests
    A-->>E: Agent response
    E->>A: Close agent session
    E->>R: Verify response
    E->>R: Close task session
    E-->>C: Episode result
```

The Environment Server coordinates ordering and the final result. Agent
and Task/Resources Servers retain ownership of the state and resources
they create.

This is part 2 of 14 in the Environment Server stack. Previous:
[#3482](#3482). Next:
[#3484](#3484).

## Validation

- `uv run pytest tests/unit_tests/test_episode_types.py -q`

## Rollouts

See the final PR for end-to-end rollout evidence.

## Compatibility and benchmark impact

No agent, resources, tool, or sandbox lifecycle contract is included.

Signed-off-by: Ananth Subramaniam <ansubramania@nvidia.com>
@alexandery-nvidia alexandery-nvidia mentioned this pull request Oct 10, 2026
Open
alexandery-nvidia added a commit to NVIDIA-NeMo/RL that referenced this pull request Oct 12, 2026
@alexandery-nvidia @claude
fix(nemo_gym): list environment_servers among the Gym server-type keys
7e583be 
NeMo Gym registers environment_servers as a fourth server group next to
responses_api_agents, responses_api_models, and resources_servers
(NVIDIA-NeMo/Gym#3482; nemo_gym/discovery.py:71 at the Gym commit this
repository pins). The mirrored constant GYM_SERVER_TYPE_KEYS in
nemo_rl/environments/nemo_gym.py still listed only the original three.

NemoGym.list_entries() walks that constant to decide which top-level
entries of Gym's resolved config start a server, and drops an entry that
holds a server under none of the listed keys. A config entry whose only
server group is environment_servers was therefore left out of the report
even though Gym starts a process for it, and an entry that pairs
environment_servers with another group was reported without that type.
_discover_route_shard_map() feeds the report to build_route_shard_map()
in nemo_rl/environments/nemo_gym_shards.py, whose per-entry duplicate
check rejects a config entry hosted by two shards unless it is
allowlisted. A dropped entry never reaches that check, so an environment
server that two shards both start was launched twice without the
ShardSetupError every other server type gets.

This change adds environment_servers to GYM_SERVER_TYPE_KEYS, extends the
list_entries() unit test with an entry whose only group is
environment_servers, and adds a test that compares the constant against
Gym's own nemo_gym.discovery._SERVER_GROUP_KEYS so the two lists cannot
drift apart again. The new test carries the nemo_gym marker because it
needs an installed Gym.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: alexandery <alexandery@nvidia.com>
reinainblood added a commit to reinainblood/Gym that referenced this pull request Oct 27, 2026
@reinainblood @claude
fix(asb): declare environment server for benchmark dispatch
4615d98 
Upstream rollout dispatch now resolves benchmarks through environment_servers
(NVIDIA-NeMo#3482); without a declaration gym eval run --benchmark asb cannot route rows
to the ASB agent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Kirsten Ruge <kirstenruge@gmail.com>
reinainblood added a commit to reinainblood/Gym that referenced this pull request Oct 28, 2026
@reinainblood @claude
fix(exploitbench): declare environment server for benchmark dispatch
ece152b 
Upstream rollout dispatch now resolves benchmarks through environment_servers
(NVIDIA-NeMo#3482); the ExploitBench agent owns the episode, so it is wired through the
legacy-agent bridge like other agent-owned benchmarks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Kirsten Ruge <kirstenruge@gmail.com>

This branch was previously deployed

1 inactive deployment
public — 27cd6b14 Deployed Sep 19, 2026 by copy-pr-bot[bot] via release / finalize / notify #3256
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ffrujeri ffrujeri ffrujeri approved these changes
@yaoyu-33 yaoyu-33 yaoyu-33 approved these changes
@bxyu-nvidia bxyu-nvidia bxyu-nvidia approved these changes
@marta-sd marta-sd Awaiting requested review from marta-sd
@cmunley1 cmunley1 Awaiting requested review from cmunley1
@adil-a adil-a Awaiting requested review from adil-a

Assignees

No one assigned

Labels

area:env-infra Shared environment framework, lifecycle, registry, scaffolding, and validation complexity:medium Single-domain change with interacting parts or a moderate review surface feature New capabilities, enhancements, or enablement work ready-to-merge PR is approved, current, and waiting for final CI or merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ananthsub @ffrujeri @yaoyu-33 @bxyu-nvidia