Image pushes fail with 502 through the public ingress; push to the in-cluster registry instead #26

New issue

Closed

opened 2026-09-10 11:39:18 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-09-10 11:39:18 +00:00

Member

Problem

The coder-xfce-vnc CI job builds successfully, then fails pushing the image:

error pushing image: failed to push to destination git.van-hemmen.com/actions/sindri:coder-xfce-vnc-…:
PATCH https://git.van-hemmen.com/v2/actions/sindri/blobs/uploads/…: unexpected status code 502 Bad Gateway

This has happened on #22 and #23 across several attempts on 2026-09-10. On Forgejo's side, those uploads ended with UploadBlob() [E] unexpected EOF after 60.0–60.7 s.

Cause

The workflows push to git.van-hemmen.com, which resolves to the public ingress (Traefik, on talos-ovh-gateway-one). The runner pods and Forgejo both run on talos-ovh-worker-two, so every layer leaves the node, goes through Traefik, and comes back. Traefik's per-request timeout cuts any single layer upload that takes longer than about a minute, and the desktop image's layers are large enough to hit it.

Fix

Push to Forgejo's in-cluster Service, app-http-service.forgejo.svc.cluster.local:3000, over plain HTTP. This uses the REGISTRY_INSECURE setting added to the kaniko wrapper in actions/kaniko#8. Forgejo stores images by owner and name, so pulls stay git.van-hemmen.com/actions/sindri:<tag>.

🤖 Generated with Claude Code

## Problem The `coder-xfce-vnc` CI job builds successfully, then fails pushing the image: ``` error pushing image: failed to push to destination git.van-hemmen.com/actions/sindri:coder-xfce-vnc-…: PATCH https://git.van-hemmen.com/v2/actions/sindri/blobs/uploads/…: unexpected status code 502 Bad Gateway ``` This has happened on #22 and #23 across several attempts on 2026-09-10. On Forgejo's side, those uploads ended with `UploadBlob() [E] unexpected EOF` after 60.0–60.7 s. ## Cause The workflows push to `git.van-hemmen.com`, which resolves to the public ingress (Traefik, on `talos-ovh-gateway-one`). The runner pods and Forgejo both run on `talos-ovh-worker-two`, so every layer leaves the node, goes through Traefik, and comes back. Traefik's per-request timeout cuts any single layer upload that takes longer than about a minute, and the desktop image's layers are large enough to hit it. ## Fix Push to Forgejo's in-cluster Service, `app-http-service.forgejo.svc.cluster.local:3000`, over plain HTTP. This uses the `REGISTRY_INSECURE` setting added to the kaniko wrapper in actions/kaniko#8. Forgejo stores images by owner and name, so pulls stay `git.van-hemmen.com/actions/sindri:<tag>`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot referenced this issue from a commit

2026-09-10 11:40:03 +00:00

#26 - Push images to the in-cluster Forgejo registry instead of the public ingress

claude-bot referenced this issue from a pull request that will close it,

2026-09-10 11:40:21 +00:00

#26 - Push images to the in-cluster Forgejo registry instead of the public ingress #27

claude-bot referenced this issue from a commit

2026-09-10 11:56:22 +00:00

#26 - Pin the kaniko builder to the 26.37.0 release