Facing issue when trying to send h2 grpc request to the backend

Hi
I am facing an issue when trying to connect to grpc backend using the ingress. Below is my ingress yaml and I have set the params insecureskipverify to false
my ingress annotation:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/protocol: h2c
kubernetes.io/ingress.class: traefik-private
spec:
rules:

  • host: thanosprometheus-cxp.us-west.aws.cxps
    http:
    paths:
    • backend:
      serviceName: thanos-prometheus-server-grpc
      servicePort: grpc
      path: /

traefik pod debug logs:
{"level":"debug","msg":"Wiring frontend thanosprometheus-cxp.us-west.aws.cxps/ to entryPoint http","time":"2020-08-27T06:08:06Z"}
{"level":"debug","msg":"Creating backend thanosprometheus-cxp.us-west.aws.cxps/","time":"2020-08-27T06:08:06Z"}
{"level":"debug","msg":"Creating load-balancer wrr","time":"2020-08-27T06:08:06Z"}
{"level":"debug","msg":"Creating server thanos-prometheus-server-84cdc97dbf-m5cdv at h2c://172.16.0.156:10901 with weight 1","time":"2020-08-27T06:08:06Z"}

172.16.0.224 - - [26/Aug/2020:23:39:22 +0000] "POST /thanos.Store/Info HTTP/2.0" 404 19 "-" "grpc-go/1.27.1" 1675 "backend not found" "/thanos.Store/Info" 0ms
172.16.0.117 - - [26/Aug/2020:23:39:22 +0000] "POST /thanos.Store/Info HTTP/2.0" 404 19 "-" "grpc-go/1.27.1" 1676 "backend not found" "/thanos.Store/Info" 0ms

traefik Version: traefik:1.7.26
Please let me