Using TCP router to route to service outside of Docker

[zegerius]

I have been trying to host my Unifi Controller on Docker - after many problems I have come to terms that I will just run it natively on my Ubuntu server. No problem. Ideally, I want to have reverse proxy to this local service through Traefik.

Quick sketch:

Running Traefik v2.1.9 with docker-compose with a variety of service (e.g., pihole, portainer) that all work flawlessly.

Unifi is running natively as a service on the docker host, at https://10.0.0.2:8443:

$ systemctl status unifi
● unifi.service - unifi
   Loaded: loaded (/lib/systemd/system/unifi.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2020-03-29 23:14:16 CEST; 11h ago
  Process: 19278 ExecStop=/usr/lib/unifi/bin/unifi.init stop (code=exited, status=0/SUCCES
  Process: 12416 ExecStart=/usr/lib/unifi/bin/unifi.init start (code=exited, status=0/SUCC
 Main PID: 12490 (jsvc)
    Tasks: 137 (limit: 4915)
   CGroup: /system.slice/unifi.service
           ├─12490 unifi -cwd /usr/lib/unifi -home /usr/lib/jvm/java-8-openjdk-amd64 -cp /
           ├─12491 unifi -cwd /usr/lib/unifi -home /usr/lib/jvm/java-8-openjdk-amd64 -cp /
           ├─12492 unifi -cwd /usr/lib/unifi -home /usr/lib/jvm/java-8-openjdk-amd64 -cp /
           ├─12515 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Dfile.encoding=UTF-8 -D
           └─12588 bin/mongod --dbpath /usr/lib/unifi/data/db --port 27117 --unixSocketPre

mrt 29 23:14:06 titan systemd[1]: Starting unifi...
mrt 29 23:14:06 titan unifi.init[12416]:  * Starting Ubiquiti UniFi Controller unifi
mrt 29 23:14:16 titan unifi.init[12416]:    ...done.
mrt 29 23:14:16 titan systemd[1]: Started unifi.

Because Unifi supplies a hard-coded, unchangeable self-signed certificate, I figured I have to use TCP with TLS Passthrough. Fine! This is how I defined my service:

[tcp]
  [tcp.services]
    [tcp.services.unifi.loadBalancer]
      [[tcp.services.unifi.loadBalancer.servers]]
        address="10.0.0.2:8443"

  [tcp.routers]
    [tcp.routers.unifi]
      entryPoints = ["https"]
      rule = "HostSNI(`unifi.hostname.com`)"
      service = "unifi"
    [tcp.routers.unifi.tls]
      passthrough=true

From the Traefik container, I can ping 10.0.0.2 and do a wget on https://10.0.0.2:8443/. The service is accessible.

Accessing http://10.0.0.2:8443 results in:

Bad Request
This combination of host and port requires TLS.

Because of this, I have also tried the following.

      [[tcp.services.unifi.loadBalancer.servers]]
        address="https://10.0.0.2:8443/"
        scheme="https"
and
      [[tcp.services.unifi.loadBalancer.servers]]
        address="https://10.0.0.2:8443/"
and
      [[tcp.services.unifi.loadBalancer.servers]]
        address="10.0.0.2:8443"
        scheme="https"

This does not work, unfortunately.

The server is correctly added, no errors in the logs and it shows up fine in the dashboard. I do see an exclamation mark next to the TCP service. As I understand it, this is because the health check fails. Accessing unifi.hostname.com results in a 404 Not Found.

I went over the docs and reference several times, but I feel I am at a lost. Is it even possible what I am trying to achieve?

Thanks!

[chon]

I'm having the exact same problem! If you find a solution please post it - it would really help others like us.

How did you deal with multiple ports having to be opened? In the docker version there are several ports that are opened but I haven't figured out quite how to do that in the @file version.

Thanks!

[cakiwi]

I haven't really touched TCP routers yet.
At first look the original confirg looks okay to me.

@zegerius

Accessing http://10.0.0.2:8443 results in:

Bad Request
This combination of host and port requires TLS.

Well yes you are doing http to an https service. Additionally you are not using SNI if you are using the IP so it wouldn't match your TCP router rule.

[zegerius]

I realize, port 8443 is internally expose (of course), so it was more to point out that intricacy. I figured I could maybe redirect the TCP to https (not really a network buff myself, as you can tell).

I do however have a HostSNI setup:

@zegerius

[tcp.routers] 
    [tcp.routers.unifi]
        entryPoints = ["https"]
        rule = "HostSNI(`unifi.hostname.com`)"

This is what the dashboard looks like now:

With the following config

[tcp]
  [tcp.services]
    [tcp.services.unifi.loadBalancer]
      [[tcp.services.unifi.loadBalancer.servers]]
        address="https://10.0.0.2:8443/"
        scheme="https"

  [tcp.routers]
    [tcp.routers.unifi]
      entryPoints = ["https"]
      rule = "HostSNI(`unifi.hostname.com`)"
      service = "unifi"
    [tcp.routers.unifi.tls]
      passthrough=true

To be fair, I am not sure tcp.services.unifi.loadBalancer.servers.scheme is even a supported key.

With address="10.0.0.2:8443" there's still an ! next to the server.

As I said, I can access the service with wget from within the Traefik container. The Traefik container lives within the following simple Docker network: docker network create --gateway 192.168.90.1 --subnet 192.168.90.0/24 proxy.

I tested with and without passthrough. Without TLS gives me a Traefik error.

I am just really confused how this is supposed to work...

Edit: I do see "TCP routers can only target TCP services (not HTTP services)" in the documentation. Am I making a mistake here?

[zegerius]

After fooling around a little bit more I setup the Unifi Controller successfully as follows:

[http.routers]
  [http.routers.unifi-rtr]
    entryPoints = ["https"]
    rule = "Host(`unifi.hostname.com`)"
    service = "unifi-svc"
    middlewares = ["secure-chain@file"]  # oauth
    [http.routers.unifi-rtr.tls]
      certresolver="dns-cloudflare"

[http.services]
  [http.services.unifi-svc.loadBalancer]
    [[http.services.unifi-svc.loadBalancer.servers]]
      url = "https://10.0.0.2:8443/"
      scheme = "https"

and adding the following CLI argument:

--serversTransport.insecureSkipVerify=true

Because I only care about accessing my controller and don't care about unifi:8080/inform working, this works for me. I figure it would be quite straight-forward adding new entrypoints for the ports used by the Unifi Controller.

And it turns out I was wrong about TCP

[chon]

Thanks for this zegerius! I had something similar and when I changed my url to https from http things fell into place. Now, I'm able to login perfectly from my internal IP but externally I get to the login page and when I login using my unifi credientials I get a "400: Bad Request" error.

This is what my url looks like after I enter my credentials:
https://unifi.mydomain.com/manage/fatal

Is this happening to you or to anyone else and if so, is there any easy fix to this?

Thanks in advance!

[zegerius]

Not had that problem specifically. Maybe seeing my current setup helps you out. Maybe it has something to do with which headers are passed to your controller.

Unifi Controller
I followed the native, Ubuntu instructions from the Ubiquiti website. It launched fine and I adopted all my USG and APs. I didn't change any settings.

Docker-Compose
Running as setup in docker-compose, scraped together from several sources. I have removed some details specific to my setup, but this should work for you.

docker-compose.yaml

version: "3.7"

networks:
  proxy:
    external:
      name: proxy
  default:
    driver: bridge

services:
# Traefik - Reverse Proxy
# docker network create --gateway 192.168.90.1 --subnet 192.168.90.0/24 proxy
# Subnet range 192.168.0.0/16 covers 192.168.0.0 to 192.168.255.255
# touch $USERDIR/infra/traefik/acme/acme.json
# chmod 600 $USERDIR/infra/traefik/acme/acme.json
# touch $USERDIR/infra/traefik/traefik.log
  traefik:
    container_name: traefik
    image: traefik:cantal  # Haven't tried v2.2 yet
    restart: always
    command: # CLI arguments
      - --global.checkNewVersion=true
      - --global.sendAnonymousUsage=false
      - --entryPoints.http.address=:80
      - --entryPoints.https.address=:443
      - --entryPoints.traefik.address=:8010  # Because Unifi Controller needs :8080 on host
      - --api=true
      - --api.dashboard=true
      - --log=true
      - --log.level=INFO
      - --serversTransport.insecureSkipVerify=true
      - --accessLog=true
      - --accessLog.filePath=/var/log/docker/traefik.log
      - --accessLog.bufferingSize=100
      - --accessLog.filters.statusCodes=400-499
      - --providers.docker=true
      - --providers.docker.defaultrule=Host(`{{ index .Labels "com.docker.compose.service" }}.$DOMAINNAME`)
      - --providers.docker.exposedByDefault=false
      - --providers.docker.network=proxy
      - --providers.docker.swarmMode=false
      - --providers.file.directory=/rules # Load dynamic configuration from one or more .toml or .yml files in a directory.
      - --providers.file.watch=true # Only works on top level files in the rules folder
      - --certificatesResolvers.dns-cloudflare.acme.email=$CLOUDFLARE_EMAIL
      - --certificatesResolvers.dns-cloudflare.acme.storage=/acme.json
      - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.provider=cloudflare
    networks:
      proxy:
        ipv4_address: 192.168.90.254
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8010
        published: 8010
        protocol: tcp
        mode: host
    volumes:
      - $USERDIR/infra/traefik/rules:/rules # file provider directory
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - $USERDIR/infra/traefik/acme/acme.json:/acme.json # cert location
      - $USERDIR/infra/traefik/traefik.log:/var/log/docker/traefik.log # for fail2ban
      - ${USERDIR}/docker/shared:/shared
    environment:
      - CF_API_EMAIL=$CLOUDFLARE_EMAIL
      - CF_API_KEY=$CLOUDFLARE_API_KEY
    labels:
      - "traefik.enable=true"
      # HTTP-to-HTTPS Redirect
      - "traefik.http.routers.http-catchall.entrypoints=http"
      - "traefik.http.routers.http-catchall.rule=HostRegexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      # HTTP Routers
      - "traefik.http.routers.traefik-rtr.entrypoints=https"
      - "traefik.http.routers.traefik-rtr.rule=Host(`traefik.$DOMAINNAME`)"
      - "traefik.http.routers.traefik-rtr.tls=true"
      - "traefik.http.routers.traefik-rtr.tls.certresolver=dns-cloudflare"
      - "traefik.http.routers.traefik-rtr.tls.domains[0].main=$DOMAINNAME"
      - "traefik.http.routers.traefik-rtr.tls.domains[0].sans=*.$DOMAINNAME"
      ## Middlewares
      - "traefik.http.routers.traefik-rtr.middlewares=rate-limit@file,oauth@file,traefik-headers"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolallowmethods=GET, OPTIONS, PUT"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolalloworigin=https://$DOMAINNAME"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolmaxage=100"
      - "traefik.http.middlewares.traefik-headers.headers.addvaryheader=true"
      - "traefik.http.middlewares.traefik-headers.headers.allowedhosts=traefik.$DOMAINNAME"
      - "traefik.http.middlewares.traefik-headers.headers.hostsproxyheaders=X-Forwarded-Host"
      - "traefik.http.middlewares.traefik-headers.headers.sslredirect=true"
      - "traefik.http.middlewares.traefik-headers.headers.sslhost=traefik.$DOMAINNAME"
      - "traefik.http.middlewares.traefik-headers.headers.sslforcehost=true"
      - "traefik.http.middlewares.traefik-headers.headers.sslproxyheaders.X-Forwarded-Proto=https"
      - "traefik.http.middlewares.traefik-headers.headers.stsseconds=63072000"
      - "traefik.http.middlewares.traefik-headers.headers.stsincludesubdomains=true"
      - "traefik.http.middlewares.traefik-headers.headers.stspreload=true"
      - "traefik.http.middlewares.traefik-headers.headers.forcestsheader=true"
      - "traefik.http.middlewares.traefik-headers.headers.framedeny=true"
      - "traefik.http.middlewares.traefik-headers.headers.customFrameOptionsValue='allow-from https:$DOMAINNAME'"
      - "traefik.http.middlewares.traefik-headers.headers.contenttypenosniff=true"
      - "traefik.http.middlewares.traefik-headers.headers.browserxssfilter=true"
      - "traefik.http.middlewares.traefik-headers.headers.referrerpolicy=same-origin"
      - "traefik.http.middlewares.traefik-headers.headers.featurepolicy=camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
      - "traefik.http.middlewares.traefik-headers.headers.customresponseheaders.X-Robots-Tag=none,noarchive,nosnippet,notranslate,noimageindex,"
      ## Services - API
      - "traefik.http.services.traefik-rtr.loadbalancer.server.port=8010"

The dashboard will live at traefik.$DOMAINNAME.

Please note middleware is specific to my setup (as specified in files in the /rules/ dir): -- traefik.http.routers.traefik-rtr.middlewares=rate-limit@file,oauth@file,traefik-headers

.env

##### SYSTEM

PUID=1000
PGID=1000
TZ=
USERDIR=/home/user

##### DOMAIN

DOMAINNAME=
CLOUDFLARE_EMAIL=
CLOUDFLARE_API_KEY=

rules/unifi.toml

[http.routers]
  [http.routers.unifi-rtr]
    entryPoints = ["https"]
    rule = "Host(`unifi.hostname.com`)"
    service = "unifi-svc"
    middlewares = ["secure-chain@file"]  # oauth
    [http.routers.unifi-rtr.tls]
      certresolver="dns-cloudflare"

[http.services]
  [http.services.unifi-svc.loadBalancer]
    [[http.services.unifi-svc.loadBalancer.servers]]
      url = "https://10.0.0.2:8443/"
      scheme = "https"

The Unifi Controller will live at unifi.$DOMAINNAME.

FYI the folder structure:

$USERDIR/infra/
├── docker
│   └── shared
├── docker-compose.yaml
└── traefik
    ├── acme
    │   └── acme.json
    ├── rules
    │   ├── basic-chain.toml
    │   ├── noauth-chain.toml
    │   ├── oauth.toml
    │   ├── rate-limit.toml
    │   ├── secure-chain.toml
    │   ├── secure-headers.toml
    │   └── unifi.toml
    ├── traefik.log
    └── traefik.toml  # Currently empty, because I define static config in CLI.

Does this help?

[chon]

Hi zegerius.

Sorry for the delayed reply but haven't had much time to spend on this until recently. I finally, after fiddling around with it, got it working. In my unifi.yml file, under services, I had to change passHostHeader: false to true and then it finally worked. And, by the way, I've been using v2.2 as well.....

Thanks for your help!
chon

[imnobaka]

I would also be curious if anyone has made it farther. I've followed a few guides comparing it to nextcloud which also has strict https. My docker compose code is below. I get a login, 2FA and then 400 Bad Request.

UniFi Controller

unifi:
container_name: unifi
image: linuxserver/unifi-controller:latest
restart: unless-stopped
volumes:
- $DOCKERDIR/unifi:/config
- $DOCKERDIR/shared:/shared
ports:
- 3478:3478/udp
- 10001:10001/udp
- 8080:8080
- 8443:8443
# # - 1900:1900/udp #optional
- 8843:8843 #optional
- 8880:8880 #optional
- 6789:6789 #optional
- 5514:5514 #optional
networks:
- t2_proxy
environment:
- PUID=$PUID
- PGID=$PGID
- TZ=$TZ
- MEM_LIMIT=1024M #optional
labels:
- "traefik.enable=true"
## TCP Routers
- "traefik.tcp.routers.unifi-tcp.entrypoints=https"
- "traefik.tcp.routers.unifi-tcp.rule=HostSNI(unifi.$DOMAINNAME)"
- "traefik.tcp.routers.unifi-tcp.tls=true"
## - "traefik.tcp.routers.unifi-tcp.tls.certresolver=dns-cloudflare"
- "traefik.tcp.routers.unifi-tcp.tls.passthrough=true"
## TCP Services
- "traefik.tcp.routers.unifi-tcp.service=unifi-tcp-svc"
- "traefik.tcp.services.unifi-tcp-svc.loadbalancer.server.port=8443"