registrypush: push core for exporting cached images to remote registries - #348
registrypush: push core for exporting cached images to remote registries#348chruffins wants to merge 8 commits into
Conversation
083dc8c to
a9d9feb
Compare
|
reviewed — clean extraction of the OCI cache reader into Structural / Maintainability (open)
status: all review findings addressed except |
101c783 to
56e884f
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 56e884f. Configure here.
sjmiller609
left a comment
There was a problem hiding this comment.
lib/ocicache/image.go:216-230 — actual DockerForeignLayer media types are left unchanged inside the converted OCI manifest. map them to OCIRestrictedLayer and update the test to use DockerForeignLayer.
lib/registrypush/provider.go:29-34 — KeychainProvider drops the caller’s context. use authn.Resolve(ctx, kc, ref.Context()) so context-aware keychains receive cancellation and deadlines.
Move the blob-store-backed v1.Image reader out of lib/registry into a new ocicache package so cached images can be reconstructed outside the inbound push path. Export the registry error classifier as ClassifyRegistryError for reuse.
Push core for exporting images from the local OCI cache to external registries. Credentials are resolved through a Provider interface with keychain, static, and per-host implementations; pushed blobs are identical to the source so manifest digests are preserved. Registry errors are classified into the typed hypeman errors.
The classifier substring-matches its input, so embedding the destination reference in the error before classification let benign target text (hosts, ports, tags containing 404 or 'not found') turn unrelated transport failures into images.ErrNotFound.
…er contracts Classify registry errors by typed transport fields (status/code) instead of substring-matching the full error text, which embedded the request URL and misclassified failures when host/port/tag/digest contained "404". Add a push-side classifier so a 401/403 from the destination surfaces as ErrUnauthorized instead of ErrNotFound (pull semantics unchanged). Validate cache digests as 64 lowercase hex chars before using them as path components, blocking traversal via crafted digests. Make cacheLayer Uncompressed() actually decompress gzip layers and DiffID() return an error instead of a lying zero hash. Fix the dead isOCIMediaType condition and Size() duplication, wrap LayerByDigest errors in ErrNotFound, and share the duplicated cache-writing test helper via lib/ocicache/testutil.
…shed manifests for GC cacheImage now parses manifestData once (sync.Once) into gcr's v1.Manifest, deleting the duplicate internalManifest struct and every v1.NewHash string round-trip. OCI manifests are returned as-is; Docker v2 manifests are copied and converted so annotations, subject, and foreign-layer descriptor urls survive the rewrite instead of being dropped. v1.Hash unmarshal validates hex/length, so manifest-supplied digests remain safe as cache path components. The embedded registry now roots every pushed manifest digest (not just BuildKit cache tags) in LiveCacheManifestDigests: the Docker manifest blob a push stores is served by the registry but unreachable from the OCI layout (which holds the converted manifest under a different digest), so without this root the OCI cache GC swept it and broke tag/digest pulls and PushFromCache after MinBlobAge. Push/PushFromCache docs now state the Docker-v2 conversion digest behavior and the source-vs-target ErrNotFound phases.
The typed classifier returns a transport error with empty diagnostics (common for HEAD or non-JSON bodies) unchanged, regressing the old substring match that turned a 401's "unauthorized" message into ErrNotFound for pull privacy. Status 401 now maps to ErrNotFound again; the push classifier handles 401/403 as ErrUnauthorized before reaching this code, so push semantics are unaffected.
blobHex accepted a bare 64-hex digest, but the cache image stored the caller string unchanged, so Digest() (v1.NewHash) failed on bare-hex inputs after a successful push. Store the canonical sha256: form so both inputs work.
fbc8142 to
a80989c
Compare
Map Docker foreign layers to the OCI restricted-layer media type during Docker v2 conversion instead of leaking the Docker media type into the converted manifest; the foreign-layer test now exercises the real DockerForeignLayer type and asserts the mapping. Forward the caller's context through KeychainProvider via authn.Resolve so context-aware keychains receive cancellation and deadlines.

Layer 1 of remote registry push support (first layer of a stack).
What
Hypeman can pull from remote registries and accept pushes into its embedded registry, but has no outbound push path. This adds the registry-agnostic push core that later layers (manager integration, API endpoint) build on.
lib/ocicache (new)
ImageFromCachereconstructs av1.Imagefrom the OCI blob cache, extracted fromlib/registry(previously the privateimageFromBlobStore). Preserves the transparent Docker v2 → OCI conversion and adds a typedErrNotFoundfor missing digests.lib/registrynow uses this package; the inbound push path is otherwise unchanged.lib/registrypush (new)
Push/PushFromCache: write a cached image to any registry viaremote.Write. Blobs are pushed as stored, so manifest digests are preserved end to end.Providerinterface for credential resolution — the modularity point for registry-specific auth (e.g. ECR token exchange in a later phase). Ships with:KeychainProvider— docker config.json incl. credential helpersStaticProvider— fixed credentialsMulti— per-host routing with default fallbackimages.ClassifyRegistryError(exported fromlib/images, renamed fromwrapRegistryError) so rate limits and missing repos map to the existing typed errors.Tests
Verification
go build -tags containers_image_openpgp ./...— full tree buildsgo test -tags containers_image_openpgp ./...— all packages pass, including the fulllib/imagessuite (withmkfs.erofsinstalled) andlib/builds, except 42 VM-instance/integration tests that require Docker Hub pulls (anonymous rate limit) and instance networking. Verified these 42 fail identically on main (same test names, side-by-side run), so they are environmental, not caused by this change.Note
Medium Risk
Touches registry error mapping, embedded-registry GC roots, and a large new cache read path; behavior changes are mostly additive but misclassification or GC regressions could break pulls/pushes.
Overview
Adds the outbound push foundation: read images from hypeman's OCI blob cache and write them to any remote registry, with pluggable credentials and shared error typing for API layers.
lib/ocicacheextracts cache-backedv1.Imageconstruction (formerly private registry code). It validates digest paths, converts Docker v2 manifests to OCI on read, and serves layers with proper gzip handling.lib/registrydelegates layout append to this package and broadens in-memory tag tracking from BuildKit-onlycacheTagstopushedTagsfor every accepted manifest so OCI GC keeps blobs still served by tag/digest pulls.lib/registrypushimplementsPush/PushFromCacheviaremote.Write, plusProviderimplementations (Docker keychain, static creds, per-hostMulti). Push auth failures map toErrUnauthorized; other registry errors use exportedimages.ClassifyRegistryError, which now prefers typedtransport.Errorstatus/codes so URLs or tag names (e.g.v404) cannot be misclassified as not-found.ClassifyRegistryErrorreplaces the privatewrapRegistryErrorat pull/mirror/oci call sites, with expanded tests for transport-shaped errors.Reviewed by Cursor Bugbot for commit 7471a0c. Bugbot is set up for automated code reviews on this repo. Configure here.