I've deployed an registry:2 behind an traefik.
I'm facing with traying to push an image to it:
$ docker push registry.dind.localhost:32785/feedly:v1
The push refers to repository [registry.dind.localhost:32785/feedly]
aa0f3a996547: Preparing
43c5bd749b73: Preparing
d7fc70cd5810: Preparing
27cd2023d60a: Preparing
4b52dfd1f9d9: Preparing
error parsing HTTP 404 response body: invalid character 'p' after top-level value: "404 page not found\n"
I've took a look on dockerd logs:
level=debug msg="Calling POST /v1.40/images/registry.dind.localhost:32785/feedly/push?tag=v1"
level=debug msg="Trying to push registry.dind.localhost:32785/feedly to https://registry.dind.localhost:32785 v2"
level=debug msg="Pushing repository: registry.dind.localhost:32785/feedly:v1"
level=debug msg="Checking for presence of layer sha256:aa0f3a9965479ba5f42cdbdddb14d00fa321d56c267bc0b944c6af8ffbfa7beb (sha256:c602ede608540175f12957c5b9fb7ebeb0e17a9625e0f8e8711306556cb251a8) in registry.dind.localhost:32785/feedly"
level=debug msg="Checking for presence of layer sha256:d7fc70cd581085df5bc3d2ed1249e507b290f9e9873024736a4861870fd00baf (sha256:cf2f38b98af20b81ce03efaa616cda3ff098f68275c55342eceee05f204a326c) in registry.dind.localhost:32785/feedly"
level=debug msg="Checking for presence of layer sha256:4b52dfd1f9d986c4319979f31a32f5ff7acef454f207d2b7ac72bb9f336c6ce8 (sha256:7fe5fcc0340ea9baed3fa3a50c78e997ed51112a739a64f093dd8938656f53c4) in registry.dind.localhost:32785/feedly"
level=debug msg="Checking for presence of layer sha256:43c5bd749b737695c814bb528d170b117e415440a2bd54a27e58d66cb2ecba28 (sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021) in registry.dind.localhost:32785/feedly"
level=debug msg="Checking for presence of layer sha256:27cd2023d60ab891655435a9102be0ce246410f5a54db39edf070a97819b5d38 (sha256:086296bbdfc786dec2bd9f9637f937fcdfaa4ac89de84b99cea50228059aed2c) in registry.dind.localhost:32785/feedly"
level=debug msg="Pushing layer: sha256:aa0f3a9965479ba5f42cdbdddb14d00fa321d56c267bc0b944c6af8ffbfa7beb"
level=debug msg="Pushing layer: sha256:4b52dfd1f9d986c4319979f31a32f5ff7acef454f207d2b7ac72bb9f336c6ce8"
level=debug msg="Pushing layer: sha256:27cd2023d60ab891655435a9102be0ce246410f5a54db39edf070a97819b5d38"
level=debug msg="Pushing layer: sha256:d7fc70cd581085df5bc3d2ed1249e507b290f9e9873024736a4861870fd00baf"
level=debug msg="Pushing layer: sha256:43c5bd749b737695c814bb528d170b117e415440a2bd54a27e58d66cb2ecba28"
level=error msg="Upload failed: error parsing HTTP 404 response body: invalid character 'p' after top-level value: \"404 page not found\\n\""
level=error msg="Upload failed: error parsing HTTP 404 response body: invalid character 'p' after top-level value: \"404 page not found\\n\""
level=info msg="Attempting next endpoint for push after error: error parsing HTTP 404 response body: invalid character 'p' after top-level value: \"404 page not found\\n\""
level=debug msg="Skipping non-TLS endpoint http://registry.dind.localhost:32785 for host/port that appears to use TLS"
I've also took a look on traefik logs:
"GET /v2/ HTTP/1.1" - - "-" "-" 32 "-" "-" 0ms
"HEAD /v2/feedly/blobs/sha256:c602ede608540175f12957c5b9fb7ebeb0e17a9625e0f8e8711306556cb251a8 HTTP/1.1" - - "-" "-" 33 "-" "-" 0ms
"HEAD /v2/feedly/blobs/sha256:7fe5fcc0340ea9baed3fa3a50c78e997ed51112a739a64f093dd8938656f53c4 HTTP/1.1" - - "-" "-" 34 "-" "-" 0ms
"HEAD /v2/feedly/blobs/sha256:086296bbdfc786dec2bd9f9637f937fcdfaa4ac89de84b99cea50228059aed2c HTTP/1.1" - - "-" "-" 35 "-" "-" 0ms
"HEAD /v2/feedly/blobs/sha256:cf2f38b98af20b81ce03efaa616cda3ff098f68275c55342eceee05f204a326c HTTP/1.1" - - "-" "-" 36 "-" "-" 0ms
"HEAD /v2/feedly/blobs/sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021 HTTP/1.1" - - "-" "-" 37 "-" "-" 0ms
"POST /v2/feedly/blobs/uploads/ HTTP/1.1" - - "-" "-" 38 "-" "-" 0ms
"POST /v2/feedly/blobs/uploads/ HTTP/1.1" - - "-" "-" 39 "-" "-" 0ms
"POST /v2/feedly/blobs/uploads/ HTTP/1.1" - - "-" "-" 40 "-" "-" 0ms
"POST /v2/feedly/blobs/uploads/ HTTP/1.1" - - "-" "-" 41 "-" "-" 0ms
"POST /v2/feedly/blobs/uploads/ HTTP/1.1" - - "-" "-" 42 "-" "-" 0ms
I've also took a look on registry logs and it doesn't generate logs!
Next step: I've tried to reach at registry using curl. I've realized it's reached:
$ curl -s registry.dind.localhost:32785/v2/_catalog | jq
{
"repositories": []
}
I've also tried to perform previously tried requests by docker push like HEAD _/v2/feedly/blobs/sha256:c602ede608540175f12957c5b9fb7ebeb0e17a9625e0f8e8711306556cb251a8:
$ curl -s -X HEAD -I registry.dind.localhost:32785/v2/feedly/blobs/sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021
HTTP/1.1 404 Not Found
It's also reached and it's telling me layer doesn't exist.
This time, requests has reached to registry. Here registry logs:
level=info msg="response completed" go.version=go1.11.2 http.request.host="registry.dind.localhost:32785" http.request.id=f9698ee1-0ee3-4021-82ab-8b659ffc0549 http.request.method=GET http.request.remoteaddr=10.0.0.2 http.request.uri="/v2/_catalog" http.request.useragent="curl/7.58.0" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.094913ms http.response.status=200 http.response.written=20
"GET /v2/_catalog HTTP/1.1" 200 20 "" "curl/7.58.0"
"HEAD /v2/feedly/blobs/sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021 HTTP/1.1" 404 157 "" "curl/7.58.0"
level=error msg="response completed with error" err.code="blob unknown" err.detail=sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021 err.message="blob unknown to registry" go.version=go1.11.2 http.request.host="registry.dind.localhost:32785" http.request.id=e42c4aa6-ee13-4eda-ab08-ee15ba3d00a4 http.request.method=HEAD http.request.remoteaddr=10.0.0.2 http.request.uri="/v2/feedly/blobs/sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021" http.request.useragent="curl/7.58.0" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.692543ms http.response.status=404 http.response.written=157 vars.digest="sha256:0df3b2d147c7a5fefc06f031771af3c9e0905a3e2459f475a3166796b56d1021" vars.name=feedly
Here, you have docker stack with traefik and registry:
version: "3.7"
services:
traefik:
image: traefik
command:
--api=true
--api.debug=true
--api.dashboard=true
--providers.docker.endpoint=tcp://172.18.0.4:2376
--providers.docker.tls.ca=/certs/ca.pem
--providers.docker.tls.cert=/certs/cert.pem
--providers.docker.tls.key=/certs/key.pem
--providers.docker.swarmMode=true
--providers.docker.exposedByDefault=false
--accesslog=true
networks:
- traefik-net
ports:
- "80:80"
- "8080:8080"
volumes:
- /certs/client:/certs
deploy:
placement:
constraints: [node.role==manager]
labels:
- traefik.enable=true
- traefik.http.routers.api.rule=Host(`traefik.dind.localhost`)
- traefik.http.routers.api.service=api@internal
- traefik.http.routers.api.middlewares=auth
- traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$7h4Zhn7O$$aTpZnWbkA44AWTMgHCwrw1
# Dummy service for Swarm port detection. The port can be any valid integer value.
- traefik.http.services.dummy-svc.loadbalancer.server.port=9999
registry:
image: registry:2
deploy:
labels:
- traefik.enable=true
- traefik.http.routers.registry.rule=Host(`registry.dind.localhost`) && PathPrefix(`/v2`)
- traefik.http.services.registry.loadbalancer.server.port=5000
networks:
- traefik-net
Any ideas?