[oyilmaz-nvidia]

Summary

• Adds a Ray Serve–based gateway (nemo_gym/orchestration/ray_serve_gateway.py) that manages both instance creation and request routing for multi-instance vLLM deployments in gym eval submit, replacing vLLM's own --data-parallel-size/multi-node DP mechanism for topologies it can't express.
• Lifts the previous restriction that forbade an individual vLLM instance's own tensor/pipeline-parallel footprint from spanning multiple Slurm nodes when number_of_instances > 1 — each instance now gets its own vllm serve --distributed-executor-backend ray process, explicitly pinned to a distinct slice of nodes via Ray actor node-affinity, so instances can themselves span nodes (e.g. TP8×PP2 across 2 nodes/instance, 2 instances, 4 nodes total).
• Selection is automatic: the gateway kicks in whenever the topology requires it (an instance's footprint exceeds one node's GPU count), or opt-in via a new use_ray_serve: true field on VllmServiceConfig for topologies that would otherwise work fine with vLLM's own DP.
• Fetches the gateway script from driver.gym_install's repo/ref directly into the vLLM service's own container (git clone + run as a standalone script — no nemo_gym package install there, to avoid clobbering vLLM's pinned torch/ray), auto-installing git if the image doesn't bundle it.
• Verified end-to-end on the real cluster: both a single-node-per-instance topology and the target TP8×PP2/2-instance/4-node topology start cleanly and reach full vLLM readiness with zero errors.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[oyilmaz-nvidia]

/ok to test 11e412d

[oyilmaz-nvidia]

/ok to test 3db55fc

[prokotg]

Thank you! For the record, we had detailed discussion when reviewing #3168