Repository navigation
tests: codify the testing contract - #914
Merged
Merged
Conversation
compose-go's tests are the executable counterpart of the Compose Specification: the document humans read to know what behavior is locked, and the surface coding agents work against. TESTING.md elevates the two patterns that emerged in this codebase — attribute conformance files in loader/tests/, YAML-in/YAML-out expectations in override/ — to official status, and codifies the rules that so far were implicit: intent stated on every test, spec section linked per attribute file, t.Run for named table cases, error assertions on config path + message, and the stages the load helpers silently skip. AGENTS.md points agents at it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
loadsAs(t, input, canonical) loads the input compose YAML, marshals the project back to YAML and compares both documents as YAML trees. The expected value then reads as compose documentation — diffable in review, writable without any knowledge of types.Project — instead of Go struct literals. Demonstrated on entrypoint (string form parsed into exec form) and extra_hosts (host:ip short syntax canonicalized to host=ip, repeated and multi-address entries expanded), where the previous typed assertions required the reader to know ShellCommand and HostsList internals to understand the locked behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Each conformance file now opens with the URL of the Compose Specification section it locks and a quote of the normative sentence being exercised, making the suite a navigable conformance matrix: from a test to the spec clause, and from a spec section back to its tests (the URLs are greppable). template_driver is flagged explicitly as a Docker Swarm extension with no spec section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Every service attribute of the JSON schema must either declare the test file locking its behavior (coveredBy) or appear in an explicit, reviewed known-gaps list. Adding a schema property without deciding where it is tested now fails the suite; stale entries (attribute removed from the schema, declared file deleted) fail too. Current gaps, tracked rather than silent: mem_limit, mem_reservation, mem_swappiness, memswap_limit, pull_refresh_after, read_only, shm_size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Several tables carried a doc/name label per case but ran the rows in a bare loop: the label never appeared in failure output, so a failing row could not be identified without bisecting the table. Named cases now run under t.Run (using the interpolation expression itself as the case name where no label existed), making failures self-identifying and individual cases runnable with -run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
ndeloof
force-pushed
the
testing-contract
branch
from
August 17, 2026 06:08
6b71d2b to
1abfb9c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
compose-go's tests are the executable counterpart of the Compose Specification — and, increasingly, the surface maintainers and their coding agents communicate through. Five dedicated commits codify that contract:
loader/tests/, YAML-in/YAML-out expectations inoverride/) to official status, and states the implicit rules: intent on every test, spec section linked per attribute file,t.Runfor named table cases, and what the load helpers silently skip. AGENTS.md points coding agents at it.loadsAs(t, input, canonical)asserts an input compose YAML loads into the canonical model expressed as YAML: expectations read as compose documentation instead of Go struct literals. Convertedentrypointandextra_hoststests as demonstrators.loader/tests/file now opens with the URL of the spec section it locks and a quote of the normative sentence, making the suite a navigable conformance matrix in both directions.mem_*,pull_refresh_after,read_only,shm_size.t.Runsubtests, so a failing row identifies itself.No behavior change; test-only (plus docs).
🤖 Generated with Claude Code