From 708f42a455cdfb39e3362f064ef7cde492913ce5 Mon Sep 17 00:00:00 2001 From: Nikos Mavrogiannopoulos Date: Sun, 31 May 2026 22:33:44 +0200 Subject: [PATCH] CONTRIBUTING.md/AGENTS.md: require tests to be self-diagnosing This is to enable agents bring good tests that can be debugged easily. Signed-off-by: Nikos Mavrogiannopoulos --- AGENTS.md | 5 +++++ CONTRIBUTING.md | 7 +++++++ 2 files changed, 12 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 901da0bf..c5961268 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -325,6 +325,11 @@ Every new feature or bug fix is incomplete without tests: more important of the two — write it first. - For **bug fixes**, write the test that reproduces the bug and confirm it fails *before* applying the fix. A test written after the fix cannot prove it is meaningful. +- **Tests must be self-diagnosing.** A failure must be explainable from the test output + alone — no silent exit-code-only failures that require local reproduction to understand. + Shell tests must print what they were testing and why it failed (e.g. `echo "FAIL: expected + X, got Y"`). C unit tests must print the failing condition and relevant values before + returning non-zero. Reviewers will reject tests whose failures are opaque. - Register all new tests in `tests/meson.build`. ### Adding Configuration Options diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8c2fbe7b..978ab2b2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -57,6 +57,12 @@ every new merge request prior to merging. There are no particular rules for the test targets, except for them being reliable and running in a reasonable time frame (~1 hour). +Tests must be **self-diagnosing**: a failure must be understandable from CI +log output alone, without requiring local reproduction. Shell tests must print +what they were testing and why it failed (e.g. `echo "FAIL: expected X, got Y"`). +C unit tests must print the failing condition and relevant values before +returning non-zero. Silent tests that fail with only an exit code will not be accepted. + ## Reviewing code @@ -80,6 +86,7 @@ No review will begin before CI passes. - [ ] `ninja -C build` succeeds after each commit, not just at the final set - [ ] Every commit has `Signed-off-by: Your Name ` - [ ] Both a positive test (correct behavior) and a negative test (bad input rejected) +- [ ] Tests are self-diagnosing: a CI failure is explainable from the log output alone - [ ] No new Linux-specific syscalls without `#ifdef __linux__` guard