[yunzhoul-nv]

Summary

Cherry-pick #16401 onto release/1.6.0 to explicitly shut down the vLLM EngineCore when a prefill worker exits.

prefill worker exit or startup failure
  close state agent
  clean up handler
  shut down engine with configured timeout

Shares _WorkerLifecycle across prefill, decode, and aggregated workers. Includes the upstream regression test and FPM test port-binding adjustment. Based on release commit d94072d69df850d183ce0a98001b0029893f5bb2, with cherry-pick provenance and DCO sign-off.

Conflict Resolution

Resolved one import-only conflict in components/src/dynamo/vllm/tests/test_vllm_worker_factory.py, caused by #15110 (commit 9dc820205913655f0b3642d52395b00ab411e361).

#15110 added _benchmark_engine_identity and _benchmark_worker_probe_path imports adjacent to _DecodeWorkerLifecycle on main. Those helpers are absent from release/1.6.0; #16401 removes the adjacent old lifecycle import and adds _WorkerLifecycle.

This backport retains the release branch's imports, removes _DecodeWorkerLifecycle, and adds _WorkerLifecycle, without bringing in the main-only benchmark helpers. Compared with #16401, only the surrounding import context differs; implementation and test-body changes applied without conflicts. Git range-diff confirms this scope.

Evidence

• Before: fix(vllm): shut down EngineCore when a prefill worker exits #16401 reports that prefill exit could hang in the multiprocessing join of a live EngineCore, eventually requiring a forced container kill.
• After: Upstream reports the fixed worker exited in 7–8 seconds under both tested exit orders on a one-A100 1P/1D setup. The regression test covers normal serving completion and registration failure.

Validation

• Source PR CI passed: Pre Merge and PR.
• No local tests or pre-commit hooks run. These CI results apply to the source PR; release-branch CI is pending.
• Upstream did not test TP2 or the Kubernetes kill -SIGINT 1 case. The cause of the changed exit order in the rc3 image remains unknown.

Merge Danger

Door: two-way

Revert the single cherry-pick commit.

Blast Radius: vLLM

Changes worker cleanup and explicitly shuts down EngineCore after prefill exit or startup failure.

Related Issues

Refs DYN-4917 — https://linear.app/nvidia/issue/DYN-4917

Source PR: #16401

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.