[sven-rankov]

What does this PR do?

Fixes #2688 ([VDR][v0.6.0] CLI-8c0bb15d).

gym eval submit --config X was the only --config consumer that let user mistakes escape as Python tracebacks:

• a path that does not exist (or is a directory) died inside Hydra compose with a ~60-line MissingConfigException traceback whose search-path dump never names the offending path;
• a file that exists but has the wrong shape leaked a raw pydantic_core.ValidationError traceback, because _handle_pydantic_validation_error only recognises BaseNeMoGymCLIConfig subclasses and SubmitConfig is an orchestration model.

Note: the issue quotes OmegaConf.load at main.py:443 raising FileNotFoundError. Since #3142 the loader is Hydra compose, so the exception type and line moved (MissingConfigException, ~main.py:600), but the symptom is identical and reproduces on current main.

Changes

• _eval_submit is wrapped in exit_cleanly_on_config_error (the decorator every other gym env/gym eval command already uses) and checks Path(args.config).is_file() before Hydra sees the path, raising a ConfigError that names it.
• SubmitConfig validation errors are rendered as a ConfigError naming the file, with the same missing required configuration: ... / invalid configuration: path (reason) wording as the generic handler. The field formatting is extracted from _handle_pydantic_validation_error into a shared _describe_validation_errors (pure move, no behaviour change) so the two paths cannot drift.

I went with rendering the error inside _eval_submit rather than the issue's alternative of making SubmitConfig a BaseNeMoGymCLIConfig: that base class carries a sys.argv-sniffing --help validator that does not belong on an orchestration model, and the generic handler's +key=<value> hint would be misleading when the fields live in a YAML file.

Before / after

Same inputs, --dry-run, on main (787ac2b) vs this branch. Exit code is 1 in every error case both before and after; only the output changes.

1. --config points at a file that does not exist (the issue's first case)

$ gym eval submit -c missing.yaml --dry-run

Before — 80 lines:

Traceback (most recent call last):
  File ".../bin/gym", line 10, in <module>
    sys.exit(main())
  ...
hydra.errors.MissingConfigException: Cannot find primary config 'missing'. Check that it's in your config search path.

Config search path:
	provider=hydra, path=pkg://hydra.conf
	provider=main, path=file:///abs/path
	provider=schema, path=structured://

After — 1 line:

Error: Submit config '/abs/path/missing.yaml' was not found. Check the path is spelled correctly and is relative to your working directory.

(A directory passed as --config gets the same treatment: ... is a directory, not a file.)

2. --config exists but is not a submit config (the issue's second case)

$ printf 'foo: 1\nbar: hello\n' > wrong_shape.yaml
$ gym eval submit -c wrong_shape.yaml --dry-run

Before — 41 lines:

Traceback (most recent call last):
  File ".../bin/gym", line 10, in <module>
    sys.exit(main())
  ...
pydantic_core._pydantic_core.ValidationError: 6 validation errors for SubmitConfig
services
  Field required [type=missing, input_value={'foo': 1, 'bar': 'hello'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/missing
  ...
bar
  Extra inputs are not permitted [type=extra_forbidden, input_value='hello', input_type=str]
    For further information visit https://errors.pydantic.dev/2.13/v/extra_forbidden

After — 1 line:

Error: Submit config '/abs/path/wrong_shape.yaml' is invalid: missing required configuration: services, compute, driver, job. invalid configuration: foo (Extra inputs are not permitted); bar (Extra inputs are not permitted).

Errors deeper in the schema keep their location, e.g. invalid configuration: job.output_path (Input should be a valid string), and SubmitConfig's own cross-field @model_validator failures render the same way (<config> (Value error, driver.policy_model 'nope' does not match any service (svc).)) instead of a traceback that echoes the whole config. A valid config produces byte-identical --dry-run output before and after.

Testing

• tests/unit_tests/test_cli_eval_submit.py: new TestEvalSubmitConfigFileErrors (missing file, directory, wrong shape, nested dotted path, cross-field @model_validator) plus two main()-level tests asserting exit 1, the Error: line, and no traceback. test_typo_in_top_level_key_is_rejected now asserts the ConfigError (and names the rejected key) instead of the raw ValidationError that was the bug.
• All 8 new/changed tests fail on main without the source change and pass with it.
• pytest tests/unit_tests/ -m "not sandbox": 4138 passed.
• pre-commit run on the changed files: clean.
• Manual CLI check of both error paths and the happy path (--dry-run against a valid config) as above.

Out of scope / follow-up

While reproducing I found that compose() bypasses the compact error rendering that @hydra.main's run_and_report gives every other command, so eval submit also tracebacks on a .yml extension, malformed YAML, a missing defaults: group file, a non-mapping file, a bad override, and an unresolvable interpolation. Those are one root cause and deserve their own issue rather than widening this PR; I can open it with reproductions if that is useful.

Checklist

• I have read the contributing guidelines.
• The change is focused; unrelated "drive-by" edits are tracked as separate issues/PRs.
• Tests added or updated and pass locally, or N/A for docs-only / non-code changes (so CI unit/server checks pass when applicable).
• Pre-commit checks pass locally (pre-commit run --all-files) (so CI lint/format/copyright pass).
• All commits have DCO sign-off (git commit -s) (so the DCO check passes).

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

LGTM. The single-use validation wrapper is now inlined, preserving the shared formatter and CLI error behavior. 291 CLI/orchestration tests and pre-commit run --all-files passed; the 31 submit tests were also rerun after formatting. Model rollouts are N/A for this CLI-only change; full CI remains pending.

[yaoyu-33]

/ok to test a407938

[github-actions]

🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA-NeMo/Gym/actions/runs/35248985180