Deep Dives

My golden tests locked in two bugs

Around noon on 2026-06-15 I locked initree’s generated output with golden tests. Two full trees, every file compared byte for byte. If anything in the output moved, the test failed.

About four hours later I opened two issues against the same output. Both bugs were in the goldens.


What was frozen

initree generates a project’s CI, container and deploy config from a recipe like go+gin+docker+gitlab-ci+k8s+slack. The golden for that recipe had this in the GitLab pipeline:

- docker build -t ghcr.io/myapp:$CI_COMMIT_SHA .
- docker push ghcr.io/myapp:$CI_COMMIT_SHA

GitHub Container Registry wants ghcr.io/<owner>/<image>. Every generated pipeline that actually ran its push step would have been rejected. And the Slack notify step, with no ingress host set, ended like this:

"text":"✅ myapp deployed — kubernetes · ns/default · 2 replicas https://"

A dangling https://, because deploy.url was always "https://${deploy.k8s.host}" and the host defaults to empty.

Both were in the golden files, exactly as generated, and the tests passed.

I didn’t find them myself. A review session did.


The fixes

The owner can’t be guessed from the project name, so it became an input, registry.docker.owner, folded into the image base. deploy.url moved to a compute hook: a host gives https://<host>, no host gives an empty string.

Then the goldens were regenerated to match:

-    - docker build -t ghcr.io/myapp:$CI_COMMIT_SHA .
+    - docker build -t ghcr.io/your-org/myapp:$CI_COMMIT_SHA .

What the tests were checking

The goldens could only say the output hadn’t moved since noon. The behavioural tests didn’t catch it either. They asserted the wrong value by name:

assert "docker build -t ghcr.io/myapp:${{ github.sha }} ." in workflow

A session wrote those tests from the generated output, and I didn’t check the expected values. So they could only agree with it.