Getting "x509: certificate signed by unknown authority" when solving DNS-01 with Route53

When attempting to solve a DNS-01 challenge via Route53, I'm receiving the following:

traefik-7b454496d4-9kv4h traefik {"level":"info","msg":"legolog: [WARN] [www.format-staging.com] acme: error cleaning up: failed to determine Route 53 hosted zone ID: RequestError: send request failed","time"
:"2019-07-17T02:06:06Z"}
traefik-7b454496d4-9kv4h traefik {"level":"info","msg":"caused by: Get https://route53.amazonaws.com/2013-04-01/hostedzonesbyname?dnsname=format-staging.com: x509: certificate signed by unknown authority ","t
ime":"2019-07-17T02:06:06Z"}

I'm using the Traefik library image, v1.7.12 on Alpine, as a Kubernetes Ingress controller with Consul as the HA keystore. When I try to run curl https://route53.amazonaws.com from the container, the certificate is valid.

Any thoughts?

Found a similar problem in traefik 1.7.19 when trying to use route53 to generate wildcard certificates.

By setting in the consul section the

insecureSkipVerify = true

the problem was solved.

# certificate store
[consul]
endpoint = "traefik-store.endpoint.TLD:8443"
prefix = "traefik-wildcard"
watch = true
  [consul.tls]
  ca = "/etc/tls/ca.pem"
  cert = "/etc/tls/consul.pem"
  key = "/etc/tls/consul-key.pem"
  insecureSkipVerify = true

Maybe the consul section definition overrides or replaces system CAs or something like that. Something is fishy with the insecureSkipVerify if you define the certs.

Won't debug more into this because traefik 2.0 changed too much + removed the support for HA in the community edition (consul and any other distributed store integration was removed) without any signs to add it back, which is a deal breaker, so we'll migrate to another solution.