Skip to content

bug: ng_run with no config spawns Ray and hangs instead of failing fastΒ #1489

Open

Description

@cwing-nvidia

Summary

Found by our virtual developer review agent.

When +config_paths is omitted (or resolves to an empty list), ng_run proceeds to spin up Ray and the head server with zero configured servers. The process never converges and is eventually killed with a native SIGTERM dump, leaving the user with low-level signal noise instead of a clear message that a config is required.

Location

nemo_gym/global_config.py β€” config parsing (~line 411)

config_paths = merged_config_for_config_paths.get(CONFIG_PATHS_KEY_NAME) or []
config_paths = ta.validate_python(config_paths)
config_paths, extra_configs = self.load_extra_config_paths(config_paths)

There is no validation that at least one config path is provided before downstream startup.

Current behavior

ng_run

Ray cluster and head server start with zero servers. The run hangs until killed externally, producing output like:

logging.cc: *** SIGTERM received ... AbslFailureSignalHandler() ... kevent

No message indicates that +config_paths is required.

Expected behavior

Fail fast before any Ray or head-server startup with a clear message, similar to the guard already used in ng_prepare_benchmark:

assert benchmarks_dict, (
    'No benchmark config found in config_paths. Pass a benchmark config, e.g.: "+config_paths=[benchmarks/aime24/config.yaml]"'
)

Suggested fix

In parse() or early in run() (before RunHelper.start()), check whether resolved config_paths is empty after parsing. If so, raise a clean SystemExit:

No config provided. Pass +config_paths=[...]. See ng_help.

This check must happen before Ray or head-server startup.

Repro

ng_run
# or explicitly:
ng_run "+config_paths=[]"

Activity

  1. added theissue type on Jun 1, 2026
  2. changed the title [-]Bug: ng_run with no config spawns Ray and hangs instead of failing fast[/-] [+]bug: ng_run with no config spawns Ray and hangs instead of failing fast[/+] on Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions