Unable to get Kubernetes Ingress served over HTTPS

Using Traefik version 2.0.0-beta1, my Traefik servers have two entrypoints: "web" for HTTP on port 80, and "web-secure" for HTTPS on port 443. When I create a Kubernetes Ingress object and attempt get Traefik to serve it only over HTTPS, Traefik always responds with a 404 response code, regardless of whether the Ingress uses a default backend, a host-based rule, a prefix-based rule, or both. I've tried including the "spec.tls" field nominating a Kubernetes Secret; doing so doesn't change the outcome.

I've tried forcing Traefik to use the HTTPS entrypoint with both of these annotations:

  • traefik.frontend.entryPoints
  • traefik.ingress.kubernetes.io/frontend-entry-points

Neither had any effect.

I tried removing the "web" entrypoint, so that Traefik would only serve HTTPS. That didn't help either.

I notice that Traefik is serving this route over HTTP. It talks to my backend pods over HTTPS as intended. However, I can't get Traefik to serve this route over HTTPS.

Here's a snapshot of my Ingress object. I've commented and uncommented just about every combination of fields that Kubernetes will accept as being valid.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: somesvc
  namespace: myns
  annotations:
    # Is this still honored?
    traefik.frontend.rule.type: PathPrefixStrip
    # What about this one?
    traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
    # Is this still honored?
    traefik.frontend.entryPoints: web-secure
    # What about this one?
    traefik.ingress.kubernetes.io/frontend-entry-points: web-secure
spec:
  tls:
  - secretName: somesvc-tls
  # backend:
  #   serviceName: somesvc
  #   servicePort: 443
  rules:
  - # host: traefik.local
    http:
      paths:
      - path: /somesvc
        backend:
          serviceName: somesvc
          servicePort: 443

What does it take to get this route served over HTTPS?

Hey @seh,

did you activate the Ingress Provider? Otherwise, Traefik wont load the Dynamic configuration. Can you share your toml maybe? :slight_smile:

Also, please have a look here.

It explains how to get that running in general with v2.

Yes, the "kubernetesIngress" provider is active, per my earlier inquiry about activating it here.

As far as I can tell, the provider is working fine, insofar as it watches Ingress objects and builds dynamic configuration from them. We can get it to serve Ingress-configured routes over HTTP. We just can't get it to work over HTTPS.

Testing this again this morning, I find that even over HTTP I can't get any of the path-based routing to work correctly. If my Ingress rule matches on the host name, the route works over HTTP. If I try to use a path-based rule, the route yields a 404 response code, both over HTTP and HTTPS.

Looking at the output of the /api/rawdata endpoint on the Traefix server, I can see that it's configuring a router and discovering servers for my Kubernetes Service as expected.

As far as I can tell, the problem with the path-based routing is that we can't get the PathPrefixStrip middleware to kick in. I've seen a terrifying history of issues with that middleware, so if there's an up-to-date document that explains what should actually work to remove the URL path before issuing the HTTP request to the backend/upstream servers, I'd appreciate a pointer.

Hello @seh, The KubernetesIngress provider in v2 has been implemented only minimally. It does not support the full annotation set that v1 does.

For example, the rule used in the Ingress v2 provider is fixed:
(https://github.com/containous/traefik/blob/v2.0/pkg/provider/kubernetes/ingress/kubernetes.go#L327)

If you want to have full customization, you need to use the KubernetesCRD provider, as annotations are not able to encompass the feature set that required us to implement the CRD in the first place.

Thank you, @daniel.tomcej. That explains why my path-based routes don't work as expected!

How about the HTTPS versus HTTP problem described above? Can you think of what's causing that mismatch? Reading the "v2.0" branch in the code, I don't see any references to the dynamic.Router.EntryPoints field from the "kubernetesIngress"-related code tree.

@seh,

In v2, the default is to bind all routers to all entrypoints.

Can you provide your entrypoint configuration (toml or CLI)?

Here's my current configuration.

api: {}
entryPoints: 
  traefik: 
    address: ":8080"
  web: 
    address: ":80"
  web-secure: 
    address: ":443"
global: 
  sendAnonymousUsage: false
log: 
  level: "DEBUG"
metrics: 
  prometheus: {}
providers: 
  file: 
    filename: "/etc/traefik/dynamic.yaml"
  kubernetesCRD: {}
  kubernetesIngress: 
    ingressEndpoint: 
      hostname: "some.local"
serversTransport: 
  insecureSkipVerify: true

It's the "web-secure" entrypoint on port 443 that I want Traefik to use here.

My main reason for trying to get Kubernetes Ingress objects to work for my immediate purposes is for their existing integration with the External DNS service, which makes host-based routing feasible for us.

Issue 4655 tracks the hope of improving the integration for IngressRoute and IngressRouteTCP, but for now there is only hope.

Hello @seh,

I have opened https://github.com/containous/traefik/issues/5161 to track this issue.

1 Like

I patched PR 5162 into a home-built Traefik container image—along with a few other changes—to try this. It looks like Traefik created two routers: one for HTTP, which it did before, and now one for HTTPS.

Is there a way to bind this route to only the HTTPS entrypoint? I tried using the following two annotations on my Ingress object, but neither had any effect:

  • traefik.frontend.entryPoints: web-secure
  • traefik.ingress.kubernetes.io/frontend-entry-points: web-secure

Coming back to this in Traefik version 2.0.1, is there any way to get a Kubernetes Ingress object that involves TLS served over just the HTTPS entrypoint, and not the HTTP entrypoint?

Our server running as a container in Kubernetes serves only over HTTPS. That provides secrecy between Traefik and the upstream server. However, we don't want any clients connecting to Traefik over HTTP to talk to this upstream server; we want to ensure that clients are using only HTTPS to connect to Traefik.

I see that Traefik still creates two routers for the service created on behalf of the Ingress object: one router for each of my two entrypoints (HTTP and HTTPS). Is there some way to restrict these routers to use only the latter entrypoint? As far as I know, none of the previous annotations work with Traefik version 2 and later.

Hey Steven! :slight_smile:

I'm not really sure if we talked about that already or not, but you can restrict a router to only listen on specific entrypoints.

entryPoints = ["websecure", "other"]

Is that what you're looking for? Or why is that not sufficent anymore?

In the v2, with Kubernetes Ingress you cannot managed entry points for an ingress.

But with the IngressRoute you can manage that.

1 Like

Ahh! Thanks Ldez. I overread that :slight_smile:

Yes, I'm aware of the fields in IngressRoute and IngressRouteTCP, but, as noted, this is about Ingress.

What's the reason for serving these routes across all entrypoints? It takes an upstream server declared to require a TLS connection and exposes it without any protection. Doesn't that strike you as a defect?

There were annotations to deal with this danger. Is this project's stance now that these annotations are never coming back? Note that we can't switch to using IngressRoute without support in External DNS.

@ldez and @SantoDE, the lack of response here suggests that the Traefik developers are intending to leave the Ingress TLS treatment in this state. Is that correct?

Again, my servers configured to require TLS are exposed over bare HTTP because I can't narrow an Ingress to be served by a subset of Traefik's entrypoints. Is the prescribed solution to patch Traefik myself?