Basic example for an instance running with admin interface enabled

Hi,
I am having troubles (maybe it is because I don't fully understand k8s/traefik) getting the migration done. I can access the traefik admin interface if I go straight to: http://10.11.6.73:8080/dashboard/#/http/services
but I cannot go anymore through my physical load balancer.

So traffic is going over SSL 443 from the LB to the traefik (443) instance and there based on "Host" it is supposed to redirect the traffic to 8080.
Here are the config files:

[root@ traefik-v2]# cat traefik-configmap.yaml && cat  traefik-deployment.yaml && cat traefik-web-ui-ingress.yaml && cat traefik-web-ui-service.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: traefik-conf
  namespace: traefik
data:
  traefik.toml: |
    [global]
      checkNewVersion = true
      sendAnonymousUsage = true
    
    [serversTransport]
      insecureSkipVerify = true
      maxIdleConnsPerHost = 0
    
    [entryPoints]
      [entryPoints.http]
        address = ":80"
      [entryPoints.https]
        address = ":443"
    
    [providers]
      providersThrottleDuration = "2s"
      [providers.kubernetesIngress]
        throttleDuration = "0s"
    
    [metrics]
      [metrics.prometheus]
        buckets = [0.1, 0.3, 1.2, 5.0]
        entryPoint = "metrics"
    
    [ping]
      entryPoint = "http"
    
    [log]
      level = "INFO"
    
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: traefik
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: traefik
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 3
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v2.0.0
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: admin
          containerPort: 8080
        args:
         - --accesslog=true
         - --api.insecure
         - --providers.kubernetescrd
         - --serverstransport.insecureskipverify=true
         - --entrypoints.http.address=:80
         - --entrypoints.https.address=:443
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-ingress
                operator: In
                values:
                - "yes"

---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: traefik
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - name: web
      port: 80
      protocol: TCP
      targetPort: 80
    - name: webs
      port: 443
      protocol: TCP
      targetPort: 443
    - name: admin
      port: 8080
      protocol: TCP
      targetPort: 8080
  externalIPs:
    - 10.11.6.73 
    - 10.11.6.79
    - 10.11.6.83
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  annotations:
    kubernetes.io/ingress.class: traefik
  creationTimestamp: null
  name: traefik-web-ui
  namespace: traefik
spec:
  entryPoints: []
  routes:
  - kind: Rule
    match: Host(`k1vip1.domain.com`)
    #middlewares:
    #- name: traefik-auth
    #  namespace: traefik
    priority: 0
    services:
    - name: traefik-web-ui
      port: 80
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-web-ui
  namespace: traefik
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - port: 80
    targetPort: 8080 # 8080

1 Like

What's the point of seeing a random instance of traefik dashboards? I cannot think of a scenario when this could be useful... Could you explain?

Hi,
I basically need traefik to listen 443 for all the traffic and apply IngressRoute that sends traffic to 8080 if matches the Host=k1vip1.domain.com. Meaning that if I type app.domain.com it sends the traffic to traefik for it to route to the appropriate POD, but if I type k1vip1.domain.com it means I want to access the traefik admin interface.
Is this a wrong approach?
Thanks.


It's not a wrong approach per se, It's just when you have a several traefik nodes, each one has it's won dashboard. And what you see on each dashboard could be different depending on your cluster configuration. If LBed you hit a random one, so I was wondering if that's useful.

On the second thought, if your traefik nodes in kubernetes cluster are configured all the same they should pick up the same configuration from ingresses / ingressroutes, so they should be the same, and since they are read-only it should not really matter which one you hit, so that makes sense.

As for you question, here is my take on it.

With exposing dashboard you have two options: api and api.insecure, you are using the latter. I would suggest using the former and then routing incoming requests to api@internal a docker example is here, but you can adapt it to your use case. You will probably need to use file provider in conjunction with kubernetes-ingress one for this, since kubernetes one cannot refer to traefik services.

Alternatively (and I do not recommend this), you can set up routing to 8080 as you are trying to.

So what does not work? You need to determine, where it break downs because there are quite a few steps here (browser > LB > Traefik Ingress > Traefik > Ingress Route >K8S Service > Dashboard) - this is not entirely scientific, since some of these above are just configuration and not run-time, but that would do.

I'm seeing entryPoints: [] on your IngressRoute, and I think this is one of possible problems.

You are also not telling what you are observing, obviously something "not working" but what are you seeing?

Hi @zespri , sorry for the late reply.
I am fine following whatever configuration seems better (I just need traefik working as ingress and the dashboard available) , but I don't understand using two providers, kubernetes like I am using now, and file provider. I am lost at this point.
Thanks.

Hi @titansmc! Thanks your interest and the report.

  • First, there is an issue in the configuration of the Traefik Deployment: you are providing both CLI arguments to Traefik AND a traefik.toml file through the ConfigMap.

    In Traefik v2, these configurations (CLI and file) are mutually exclusive as stated in https://docs.traefik.io/v2.0/getting-started/configuration-overview/#the-static-configuration:

    It means that you have to choose between CLI or file. This assumption is not true for the dynamic configuration.

    => If you check the log of any of the Traefik pods, you'll see a line that tells you which configuration is used when loading. So you'll be able to deduce which one is not used :slight_smile:

  • Then, can you start with only 1 replica of the Traefik pod? The goal is to have a first viable and working configuration before going on the road of distributed.

  • Remove the port "admin | 8080" from the Service named traefik-ingress-lb to avoid exposing the port 8080 of Traefik on the outside. Exposing the port 8080 of the Traefik pod(s) is done with the other service named traefik-web-ui.

  • To simplify your setup, edit the Service named traefik-web-ui so that both port: and targetPort: directive have the same value 8080 (easier setup: LB (443) -> Traefik (443) -> Traefik itself (8080)). The goal is to remove any other complex setup to reach a working configuration. Don't forget to adapt the port in the section services: of the IngressRoute as well :slight_smile:

Let us know the result of this setup

I think I am missing this part https://docs.traefik.io/v2.0/operations/api/#configuration , I followed your instructions and when trying to access the dashboard I get:

evel=debug msg="'502 Bad Gateway' caused by: dial tcp 10.133.105.36:8080: connect: connection refused"

From another container I don't see the port 8080 listening, :

[root@moscardo traefik-v2]# kubectl describe  pods  -n traefik traefik-ingress-controller-54f5598c4b-2g55h
Name:               traefik-ingress-controller-54f5598c4b-2g55h
Namespace:          traefik
Priority:           0
PriorityClassName:  <none>
Node:               ops-k1n01.domain.com/10.11.6.73
Start Time:         Mon, 14 Oct 2019 15:03:59 +0200
Labels:             k8s-app=traefik-ingress-lb
                    name=traefik-ingress-lb
                    pod-template-hash=54f5598c4b
Annotations:        <none>
Status:             Running
IP:                 10.133.105.36
Controlled By:      ReplicaSet/traefik-ingress-controller-54f5598c4b
Containers:
  traefik-ingress-lb:
    Container ID:  docker://51f769a399da67d88edc8f7c8cafd98fecde37afcbd589f5ea6c0dd931d361ba
    Image:         traefik:v2.0.2
    Image ID:      docker-pullable://traefik@sha256:ac9ac7c0509725ddb74b6c1b08a76cac540670f656770818a5119f6ac0ca391a
    Ports:         80/TCP, 443/TCP, 8080/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      --accesslog=true
      --api.dashboard=true
      --log.level=DEBUG
      --providers.kubernetescrd
      --serverstransport.insecureskipverify=true
      --entrypoints.web.address=:80
      --entrypoints.websecure.address=:443
      --entrypoints.websecure.forwardedheaders.insecure=true
    State:          Running
      Started:      Mon, 14 Oct 2019 15:04:00 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from traefik-ingress-controller-token-9hz45 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  traefik-ingress-controller-token-9hz45:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  traefik-ingress-controller-token-9hz45
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

So what I don't know is the traefik service that you guys talk about, do I need to reference it in the ingressroute ?

Thanks.

Hi @titansmc , the "service" we are mentioning is a Kubernetes Service, not a Traefik Service: https://kubernetes.io/docs/concepts/services-networking/service/.

The goal of Kubernetes Services is to expose the Pods through the network, because Pod's IP can change, or Pods can move. If you use the command line kubectl get svc --namespace= traefik, you can see the Kubernetes Services created within the namespace named traefik.
The definition of these services are found on the YAML manifest files where the attribute kind: has the value Service.

On the initial message of this post, you provided the content of these YAML manifest files you used to install Traefik on your Kubernetes: there are 2 defined Services:

  • One named traefik-ingress-service, which exposes/loadbalance the private TCP ports 80, 443 and 8080, of every pod having the selector k8s-app set to the value traefik-ingress-lb, to the public ports (respectively) 80, 443 and 8080 of the external IP of this service itself.
    • This "Kubernetes Service" allows the Ingress Controller to be reachable from the outside. So the port 8080 should not be defined here, as it exposes the port 8080 publically, while you want the dashboard to be available on HTTP/HTTPS on 80/443 with HTTP routing (host, path, auth, etc.).
    • Please note that, even though you specified 3 "external static IPs" to this service, the default type of Service in Kubernetes is ClusterIP, which means only reachable inside Kubernetes networks. I'm not sure about the behavior, but I would set the type of this service to NodePort and ensure that the published ports are configured correctly on the external Load Balancer. Please check Kubernetes documentation to understand what is going on here.
  • One named traefik-web-ui which exposes/loadbalance the private TCP port 8080, of every pod having the selector k8s-app set to the value traefik-ingress-lb, to the public port 80. This service is only internal to Kubernetes networks as the default type is ClusterIP.
    • It is fine to keep this service private, as you want it served by the Ingress.
    • This "Kubernetes Service" is already mapped to the "Traefik Service" of the IngressRoute. My recommendation is to stay with the port 8080 for the exposed port of the service (because it is currently 80 for you right now).
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
# ...
services:
    - name: traefik-web-ui
      port: 80

You light be interested by this presentation if you want a clearer understanding of the Kubernetes Services: https://containous.github.io/slides/webinar-cncf-sept-2019/#/