hack/cozytest.sh runs every test body in a subshell under set -eu -x and defines no skip command; the only match for the word in the file is the unrelated skip_next variable. A test that calls skip therefore hits a missing command, exits 127, and set -e fails the test. The thing written to tolerate a condition is what breaks on it.
Three call sites today:
hack/e2e-install-cozystack.bats:593, the file CI actually runs. The call guards a ValidatingAdmissionPolicy assertion for clusters below Kubernetes 1.30, so the branch is out of reach in practice, since the platform requires a much newer management cluster. Unreachable is not the same as harmless: the guard is dead weight that would fail closed the day it fired.hack/e2e-apps/monitoring-oidc-system.bats:132 and hack/e2e-apps/monitoring-oidc-customconfig.bats:112, which nothing has executed since the suite moved to Chainsaw. Latent rather than live.
In a cozytest suite the working form is echo "…" >&2; return 0, which reads the same and also behaves under real bats.
There is a second defect in the same spot, at the prose layer. The comment above hack/e2e-install-cozystack.bats:592 reads: "Detect by attempting to fetch the policy by name; if the API is present, the resource will be retrievable, otherwise kubectl exits non-zero on an unknown resource type." The code below it fetches nothing by name and tests output instead (api-resources … | grep -qw), which is the correct shape. But the comment teaches precisely the exit-code model that caused #3516, where kubectl api-resources on an absent API group exits 0 and the guard never fires. A comment that explains the right code with the wrong mechanism is worse than no comment: the next person copies the reasoning, not the line.
hack/cozytest.shruns every test body in a subshell underset -eu -xand defines noskipcommand; the only match for the word in the file is the unrelatedskip_nextvariable. A test that callsskiptherefore hits a missing command, exits 127, andset -efails the test. The thing written to tolerate a condition is what breaks on it.Three call sites today:
hack/e2e-install-cozystack.bats:593, the file CI actually runs. The call guards a ValidatingAdmissionPolicy assertion for clusters below Kubernetes 1.30, so the branch is out of reach in practice, since the platform requires a much newer management cluster. Unreachable is not the same as harmless: the guard is dead weight that would fail closed the day it fired.hack/e2e-apps/monitoring-oidc-system.bats:132andhack/e2e-apps/monitoring-oidc-customconfig.bats:112, which nothing has executed since the suite moved to Chainsaw. Latent rather than live.In a cozytest suite the working form is
echo "…" >&2; return 0, which reads the same and also behaves under real bats.There is a second defect in the same spot, at the prose layer. The comment above
hack/e2e-install-cozystack.bats:592reads: "Detect by attempting to fetch the policy by name; if the API is present, the resource will be retrievable, otherwise kubectl exits non-zero on an unknown resource type." The code below it fetches nothing by name and tests output instead (api-resources … | grep -qw), which is the correct shape. But the comment teaches precisely the exit-code model that caused #3516, wherekubectl api-resourceson an absent API group exits 0 and the guard never fires. A comment that explains the right code with the wrong mechanism is worse than no comment: the next person copies the reasoning, not the line.