My golden tests locked in two bugs
Around noon on 2026-06-15 I locked initree’s generated output with golden tests. Two full trees, every file compared byte for byte. If anything in the output moved, the test failed.
About four hours later I opened two issues against the same output. Both bugs were in the goldens.
What was frozen
initree generates a project’s CI, container and deploy config from a recipe like go+gin+docker+gitlab-ci+k8s+slack. The golden for that recipe had this in the GitLab pipeline:
- docker build -t ghcr.io/myapp:$CI_COMMIT_SHA .
- docker push ghcr.io/myapp:$CI_COMMIT_SHA
GitHub Container Registry wants ghcr.io/<owner>/<image>. Every generated pipeline that actually ran its push step would have been rejected. And the Slack notify step, with no ingress host set, ended like this:
"text":"✅ myapp deployed — kubernetes · ns/default · 2 replicas https://"
A dangling https://, because deploy.url was always "https://${deploy.k8s.host}" and the host defaults to empty.
Both were in the golden files, exactly as generated, and the tests passed.
I didn’t find them myself. A review session did.
The fixes
The owner can’t be guessed from the project name, so it became an input, registry.docker.owner, folded into the image base. deploy.url moved to a compute hook: a host gives https://<host>, no host gives an empty string.
Then the goldens were regenerated to match:
- - docker build -t ghcr.io/myapp:$CI_COMMIT_SHA .
+ - docker build -t ghcr.io/your-org/myapp:$CI_COMMIT_SHA .
What the tests were checking
The goldens could only say the output hadn’t moved since noon. The behavioural tests didn’t catch it either. They asserted the wrong value by name:
assert "docker build -t ghcr.io/myapp:${{ github.sha }} ." in workflow
A session wrote those tests from the generated output, and I didn’t check the expected values. So they could only agree with it.