Can't create letsencrypt wildcard certificates with traefik 2.0.0-beta1

Hello,

what's the right approach for acme wildcard certificates on traefik 2.0.0-beta1? In my docker-stack.yml example below I have two docker containers with ...tls.domains = domain.tld, *.domain.tld labels on my http routes. ldez mentioned in this thread Multiple Sites / Domains that domains are optional and that certificates are created based on the host rule.

In my example it seems that certifcates are only created based on containers' host rule and tls.domains label seems to be completely ignored....

It'll be grateful if someone could tell how to fix configuration to get wildcard certificates generation working. Thanks a lot in advance!

docker-stack.yml
version: '3.7'

services:

  traefik:
    image: traefik:2.0-alpine
    environment:
      - AWS_REGION=eu-central-1
      - AWS_ACCESS_KEY_ID=xxx
      - AWS_SECRET_ACCESS_KEY=xxx
      - AWS_HOSTED_ZONE_ID=xxx
    deploy:
      replicas: 1
      restart_policy:
        delay: 5s
        max_attempts: 3
      placement:
        constraints:
          - node.role == manager
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.traefik_http.rule=Host(`traefik.domain.tld`)"
        - "traefik.http.routers.traefik_http.entrypoints=http"
        - "traefik.http.routers.traefik_http.middlewares=http-chain@file"
        - "traefik.http.routers.traefik_https.rule=Host(`traefik.domain.tld`)"
        - "traefik.http.routers.traefik_https.entrypoints=https"
        - "traefik.http.routers.traefik_https.middlewares=https-basicauth-chain@file"
        - "traefik.http.routers.traefik_https.tls=true"
        - "traefik.http.routers.traefik_https.tls.certresolver=letsencrypt"
        - "traefik.http.routers.traefik_https.tls.domains=domain.tld, *.domain.tld"
        - "traefik.http.routers.traefik_https.service=traefik_https"
        - "traefik.http.services.traefik_https.loadbalancer.server.port=8080"
        - "traefik.http.services.traefik_https.loadbalancer.server.scheme=http"
    ports:
      - 80:80
      - 443:443
      - 25:25
      - 465:465
      - 587:587
      - 110:110
      - 995:995
      - 143:143
      - 993:993
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /opt/docker.swarm/traefik/traefik.toml:/traefik.toml:ro
      - /opt/docker.swarm/traefik/etc/:/etc/traefik/:ro
      - /opt/docker.swarm/traefik/acme.json:/acme.json
      - /var/log/:/var/log/

  whoami:
    image: containous/whoami
    deploy:
      replicas: 1
      restart_policy:
        delay: 5s
        max_attempts: 3
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.whoami_http.rule=Host(`whoami.domain.tld`)"
        - "traefik.http.routers.whoami_http.entrypoints=http"
        - "traefik.http.routers.whoami_http.middlewares=http-chain@file"
        - "traefik.http.routers.whoami_https.rule=Host(`whoami.domain.tld`)"
        - "traefik.http.routers.whoami_https.entrypoints=https"
        - "traefik.http.routers.whoami_https.middlewares=https-chain@file"
        - "traefik.http.routers.whoami_https.tls=true"
        - "traefik.http.routers.whoami_https.tls.certresolver=letsencrypt"
        - "traefik.http.routers.whoami_https.tls.domains=domain.tld, *.domain.tld"
        - "traefik.http.routers.whoami_https.service=whoami_https"
        - "traefik.http.services.whoami_https.loadbalancer.server.port=80"
        - "traefik.http.services.whoami_https.loadbalancer.server.scheme=http"

networks:
  default:
    external: true
    name: traefik
traefik.toml
[global]
  checkNewVersion = false
  sendAnonymousUsage = false

[entrypoints]
  [entrypoints.http]
    address = ":80"

  [entrypoints.https]
    address = ":443"

  [entrypoints.smtp-relay]
    address = ":25"

  [entrypoints.smtp-ssl]
    address = ":465"

  [entrypoints.smtp]
    address = ":587"

  [entrypoints.pop3]
    address = ":110"

  [entrypoints.pop3-ssl]
    Address = ":995"

  [entrypoints.imap]
    address = ":143"

  [entrypoints.imap-ssl]
    address = ":993"

[log]
  level = "DEBUG"
# filePath = "/var/log/traefik.log"

#[accessLog]
#  filePath = "/var/log/traefik.access.log"
#  format = "common"

[accessLog.filters]
  statusCodes = ["200", "300-302"]
  retryAttempts = true
  minDuration = "10ms"

[accessLog.fields]
  defaultmode = "keep"
  [accessLog.fields.names]
    "clientUsername" = "drop"

  [accessLog.fields.headers]
    defaultMode = "keep"
    [accessLog.fields.headers.names]
      "User-Agent" = "redact"
      "Authorization" = "drop"
      "Content-Type" = "keep"

[api]

[ping]

[providers]
  [providers.file]
    directory = "/etc/traefik/conf.d"

  [providers.docker]
    network = "traefik"
    defaultRule = "Host(`{{ normalize .Name }}.domain.tld`)"
    exposedByDefault = false
    swarmMode = true

[certificatesResolvers.letsencrypt.acme]
  email = "admin@domain.tld"
  storage = "acme.json"

  [certificatesResolvers.letsencrypt.acme.dnsChallenge]
    provider = "route53"
middlewares.toml
    [http.middlewares.http-chain.chain]
      middlewares = ["redirect-https"]

    [http.middlewares.https-chain.chain]
      middlewares = ["headers-sts", "compress"]

    [http.middlewares.https-basicauth-chain.chain]
      middlewares = ["headers-sts", "compress", "test-auth"]

    [http.middlewares.redirect-https.redirectScheme]
      scheme = "https"
      permanent = true

    [http.middlewares.headers-sts.headers]
      STSSeconds = 315360000
      STSIncludeSubdomains = true
      STSPreload = true
      forceSTSHeader = true

    [http.middlewares.compress.compress]

    [http.middlewares.test-auth.basicauth]
      users = [
        "admin:$apr1$JVZBrHHX$xbqhKQPVmR1O6b7v3aG4g0"
      ]
tls.toml
[tls]
  [tls.options]
    [tls.options.default]
      sniStrict = true
      cipherSuites = [
        "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
        "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
        "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", # TLS 1.2
        "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
        "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", # TLS 1.2
        "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
        "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
        "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305",
        "TLS_AES_128_GCM_SHA256",
        "TLS_AES_256_GCM_SHA384",
        "TLS_CHACHA20_POLY1305_SHA256",
        "TLS_FALLBACK_SCSV"
      ]
1 Like

domains are optional except for wildcard because a wildcard domain is not valid Host rule.

This is not a valid labels.

The valid syntax:

- "traefik.http.routers.traefik_https.tls.domains[0].main=domain.tld,"
- "traefik.http.routers.traefik_https.tls.domains[0].sans=*.domain.tld"

A simple example for Cloudflare:

version: "3.3"

services:

  traefik:
    image: "traefik:v2.0.0-beta1"
    command:
      - --entrypoints.web.address=:80
      - --entrypoints.websecure.address=:443
      - --api.dashboard=true
      - --providers.docker=true
      - --global.sendAnonymousUsage=true
      # - --log.level=DEBUG
      - --certificatesresolvers.basic.acme.email=postmaster@mydomain.com
      - --certificatesresolvers.basic.acme.storage=/le/acme.json
      - --certificatesresolvers.basic.acme.dnschallenge.provider=cloudflare
      # - --certificatesresolvers.basic.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    labels:
      - "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https@docker"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
    volumes:
      - "./acme.json:/le/acme.json"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
    environment:
      - CLOUDFLARE_EMAIL=foo@bar.com
      - CLOUDFLARE_API_KEY=mykey

  whoami:
    image: containous/whoami
    labels:
      - "traefik.http.routers.whoami.rule=Host(`mydomain.com`)"
      - "traefik.http.routers.whoami.entrypoints=websecure"
      - "traefik.http.routers.whoami.tls.certresolver=basic"
      - "traefik.http.routers.whoami.tls.domains[0].main=mydomain.com"
      - "traefik.http.routers.whoami.tls.domains[0].sans=*.mydomain.com"
4 Likes

I've also been banging my head and struggling with this. I've pored over the release notes, the latest documentation, this thread, and other threads

I attempted to use the exact same compose file provided by @ldez here. However, when I run it I get the following error message.

level=error msg="the router whoami uses a non-existent resolver: basic"

In the interim I've reverted back to traefik:v2.0.0-alpha8

Hopefully later this week I'll have time to debug further and can provide my docker file and traefik.toml

FYI the logs of my example:

Summary
Recreating tem_traefik_1 ... done
Recreating tem_whoami_1  ... done
Attaching to tem_whoami_1, tem_traefik_1
whoami_1   | Starting up on port 80
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="Configuration loaded from flags."
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="Traefik version 2.0.0-beta1 built on 2019-07-19T16:04:34Z"
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true,\"sendAnonymousUsage\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}},\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}},\"websecure\":{\"address\":\":443\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":15000000000}},\"api\":{\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"},\"certificatesResolvers\":{\"basic\":{\"acme\":{\"email\":\"postmaster@mydomain.com\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/le/acme.json\",\"keyType\":\"RSA4096\",\"dnsChallenge\":{\"provider\":\"cloudflare\"}}}}}"
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="\nStats collection is enabled.\nMany thanks for contributing to Traefik's improvement by allowing us to receive anonymous information from your configuration.\nHelp us improve Traefik by leaving this feature on :)\nMore details on: https://docs.traefik.io/basics/#collected-data\n"
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="Starting provider aggregator.ProviderAggregator {}"
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Start TCP Server" entryPointName=websecure
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Start TCP Server" entryPointName=web
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="Starting provider *acme.Provider {\"email\":\"postmaster@mydomain.com\",\"caServer\":\"https://acme-staging-v02.api.letsencrypt.org/directory\",\"storage\":\"/le/acme.json\",\"keyType\":\"RSA4096\",\"dnsChallenge\":{\"provider\":\"cloudflare\"},\"ResolverName\":\"basic\",\"store\":{},\"ChallengeStore\":{}}"
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="Testing certificate renew..." providerName=acme.basic
traefik_1  | time="2019-07-21T22:40:08Z" level=info msg="Starting provider *docker.Provider {\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":15000000000}"
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Start TCP Server" entryPointName=traefik
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Provider connection established with docker 18.09.6-ce (API 1.39)" providerName=docker
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Configuration received from provider acme.basic: {\"http\":{},\"tls\":{}}" providerName=acme.basic
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Configuration received from provider docker: {\"http\":{\"routers\":{\"http-catchall\":{\"entryPoints\":[\"web\"],\"middlewares\":[\"redirect-to-https@docker\"],\"service\":\"traefik_tem\",\"rule\":\"hostregexp(`{host:.+}`)\"},\"whoami\":{\"entryPoints\":[\"websecure\"],\"service\":\"whoami_tem\",\"rule\":\"Host(`mydomain.com`)\",\"tls\":{\"certResolver\":\"basic\",\"domains\":[{\"main\":\"mydomain.com\",\"sans\":[\"*.mydomain.com\"]}]}}},\"middlewares\":{\"redirect-to-https\":{\"redirectScheme\":{\"scheme\":\"https\"}}},\"services\":{\"traefik_tem\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.21.0.3:80\"}],\"passHostHeader\":true}},\"whoami_tem\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://172.21.0.2:80\"}],\"passHostHeader\":true}}}},\"tcp\":{}}" providerName=docker
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating middleware" middlewareName=pipelining middlewareType=Pipelining entryPointName=web routerName=http-catchall@docker serviceName=traefik_tem
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating load-balancer" entryPointName=web routerName=http-catchall@docker serviceName=traefik_tem
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating server 0 http://172.21.0.3:80" serverName=0 entryPointName=web routerName=http-catchall@docker serviceName=traefik_tem
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Added outgoing tracing middleware traefik_tem" entryPointName=web routerName=http-catchall@docker middlewareName=tracing middlewareType=TracingForwarder
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating middleware" entryPointName=web routerName=http-catchall@docker middlewareName=redirect-to-https@docker middlewareType=RedirectScheme
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Setting up redirection to https " middlewareType=RedirectScheme entryPointName=web routerName=http-catchall@docker middlewareName=redirect-to-https@docker
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Adding tracing to middleware" entryPointName=web routerName=http-catchall@docker middlewareName=redirect-to-https@docker
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=web
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating middleware" serviceName=whoami_tem middlewareName=pipelining middlewareType=Pipelining entryPointName=websecure routerName=whoami@docker
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating load-balancer" entryPointName=websecure routerName=whoami@docker serviceName=whoami_tem
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating server 0 http://172.21.0.2:80" entryPointName=websecure routerName=whoami@docker serviceName=whoami_tem serverName=0
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Added outgoing tracing middleware whoami_tem" middlewareName=tracing middlewareType=TracingForwarder entryPointName=websecure routerName=whoami@docker
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="Creating middleware" middlewareName=traefik-internal-recovery middlewareType=Recovery entryPointName=websecure
traefik_1  | time="2019-07-21T22:40:08Z" level=debug msg="No default certificate, generating one"
traefik_1  | time="2019-07-21T22:40:09Z" level=debug msg="Looking for provided certificate(s) to validate [\"mydomain.com\" \"*.mydomain.com\"]..." providerName=acme.basic
traefik_1  | time="2019-07-21T22:40:09Z" level=debug msg="Domains [\"mydomain.com\" \"*.mydomain.com\"] need ACME certificates generation for domains \"mydomain.com,*.mydomain.com\"." providerName=acme.basic
traefik_1  | time="2019-07-21T22:40:09Z" level=debug msg="Loading ACME certificates [mydomain.com *.mydomain.com]..." providerName=acme.basic
traefik_1  | time="2019-07-21T22:40:09Z" level=debug msg="Building ACME client..." providerName=acme
traefik_1  | time="2019-07-21T22:40:09Z" level=debug msg="https://acme-staging-v02.api.letsencrypt.org/directory" providerName=acme
traefik_1  | time="2019-07-21T22:40:10Z" level=info msg=Register... providerName=acme
traefik_1  | time="2019-07-21T22:40:10Z" level=debug msg="legolog: [INFO] acme: Registering account for postmaster@mydomain.com"
traefik_1  | time="2019-07-21T22:40:11Z" level=debug msg="Using DNS Challenge provider: cloudflare" providerName=acme
traefik_1  | time="2019-07-21T22:40:11Z" level=debug msg="legolog: [INFO] [mydomain.com, *.mydomain.com] acme: Obtaining bundled SAN certificate"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [*.mydomain.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/9czmDXCmkQSKzGANxse2EBUzwSzgvd57qI7mjizE62I"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [mydomain.com] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/eZ-7mTcilDwQkzvrFkHHT6HO_tUheLgOTolwIQU1NGk"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [*.mydomain.com] acme: use dns-01 solver"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [mydomain.com] acme: Could not find solver for: tls-alpn-01"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [mydomain.com] acme: Could not find solver for: http-01"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [mydomain.com] acme: use dns-01 solver"
traefik_1  | time="2019-07-21T22:40:12Z" level=debug msg="legolog: [INFO] [*.mydomain.com] acme: Preparing to solve DNS-01"
traefik_1  | time="2019-07-21T22:40:13Z" level=debug msg="legolog: [INFO] [mydomain.com] acme: Preparing to solve DNS-01"
traefik_1  | time="2019-07-21T22:40:13Z" level=debug msg="legolog: [INFO] [*.mydomain.com] acme: Cleaning DNS-01 challenge"
1 Like

Thanks @ldez for bringing light into this topic. Now wildcard certifcate is created as expected.

Not to resurrect a dead thread, but I wanted to close the loop.

I was finally able to spend time following @ldez instructions to get letsencrypt wildcard certificates working. It turned out that I had failed to delete the old acme.json file because the storage format had changed. Once I did that, I now have it working well in traefik:v2.1

Thanks for your help!

1 Like

How would I create a second service or more to use the wildcard?

Would it be as follows?

 whoami2:
    image: containous/whoami
    labels:
      - "traefik.http.routers.whoami2.rule=Host(`whoami2.mydomain.com`)"
      - "traefik.http.routers.whoami2.entrypoints=websecure"
      - "traefik.http.routers.whoami2.tls.certresolver=basic"
      - "traefik.http.routers.whoami2.tls.domains[0].main=mydomain.com"
      - "traefik.http.routers.whoami2.tls.domains[0].sans=*.mydomain.com"