I have a web application that is deployed on K8’s and it is working fine. I want to add google authentication to my application and I found that I had to make a few changes to my Ingress. I added the following block of code to my Ingress metadata – annotations

kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: 'false'
nginx.ingress.kubernetes.io/auth-response-headers: Authorization
nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth

After adding this, my web application did not work anymore. I believe I was getting Error 503.

After searching through google once again, I found that I need to deploy an oauth2-proxy server to make this work. So I referred to these helm charts and deployed oauth2-proxy server in my namespace. I obtained the necessary credentials from GCP and entered in the client ID, client secret and cookie secret. I just entered in the client ID,secret and cookie secret and entered my respective email domain here. I have literally made no other change in the helm charts and my application is still not working. Im getting Error 503 again when I try to access it.

Here is the Deployment, Service and Ingress of my web application

apiVersion: apps/v1
kind: Deployment
metadata:
  name: name
  labels:
    helm.sh/chart: name-0.1.0
    app.kubernetes.io/name: name
    app.kubernetes.io/instance: name
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: name
      app.kubernetes.io/instance: name
  template:
    metadata:
      labels:
        app.kubernetes.io/name: name
        app.kubernetes.io/instance: name
    spec:
      serviceAccountName: name
      securityContext:
        {}
      containers:
        - name: name
          securityContext:
            privileged: true
          image: image
          imagePullPolicy: Always
          args:
            - --api
          ports:
            - name: http
              containerPort: 8000
              protocol: TCP
          # livenessProbe:
          #   httpGet:
          #     path: /
          #     port: http
          # readinessProbe:
          #   httpGet:
          #     path: /
          #     port: http
          resources:
            {}

---

apiVersion: v1
kind: Service
metadata:
  name: name
  labels:
    helm.sh/chart: name-0.1.0
    app.kubernetes.io/name: name
    app.kubernetes.io/instance: name
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - port: 8000
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: name
    app.kubernetes.io/instance: name

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: name
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-response-headers: Authorization
    nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
    nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  defaultBackend:
    service:
      name: name
      port:
        number: 80

  labels:
    helm.sh/chart: name-0.1.0
    app.kubernetes.io/name: utility-deploy
    app.kubernetes.io/instance: name
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-response-headers: Authorization
    nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
    nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
    - host: "site.example.com"
      http:
        paths:
          - path: /app
            pathType: ImplementationSpecific
            backend:
              service:
                name: name
                port:
                  number: 8000

And here is the Deployment, Service and Ingress of my oauth2-proxy

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: oauth2-proxy
    chart: oauth2-proxy-4.3.0
    heritage: Helm
    release: auth
  name: auth-oauth2-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: oauth2-proxy
      release: auth
  template:
    metadata:
      annotations:
        checksum/config:value
        checksum/config-emails:value
        checksum/secret:value
        checksum/google-secret:value
      labels:
        app: oauth2-proxy
        release: "auth"
    spec:
      serviceAccountName: auth-oauth2-proxy
      containers:
      - name: oauth2-proxy
        image: "quay.io/oauth2-proxy/oauth2-proxy:7.2.1"
        imagePullPolicy: IfNotPresent
        args:
          - --http-address=0.0.0.0:4180
          - --email-domain=company.com
          - --set-authorization-header
          - --config=/etc/oauth2_proxy/oauth2_proxy.cfg
        env:
        - name: OAUTH2_PROXY_CLIENT_ID
          valueFrom:
            secretKeyRef:
              name:  auth-oauth2-proxy
              key: client-id
        - name: OAUTH2_PROXY_CLIENT_SECRET
          valueFrom:
            secretKeyRef:
              name:  auth-oauth2-proxy
              key: client-secret
        - name: OAUTH2_PROXY_COOKIE_SECRET
          valueFrom:
            secretKeyRef:
              name:  auth-oauth2-proxy
              key: cookie-secret
        ports:
          - containerPort: 4180
            name: http
            protocol: TCP
        livenessProbe:
          httpGet:
            path: /ping
            port: http
            scheme: HTTP
          initialDelaySeconds: 0
          timeoutSeconds: 1
        readinessProbe:
          httpGet:
            path: /ping
            port: http
            scheme: HTTP
          initialDelaySeconds: 0
          timeoutSeconds: 1
          successThreshold: 1
          periodSeconds: 10
        resources:
          {}

---

apiVersion: v1
kind: Service
metadata:
  labels:
    app: oauth2-proxy
    chart: oauth2-proxy-4.3.0
    release: auth
    heritage: Helm
  name: auth-oauth2-proxy
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app: oauth2-proxy
    release: auth

---

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: auth-oauth2-proxy
  namespace: namespace
  annotations:
    kubernetes.io/ingress.class: nginx
  labels:
    app: auth-oauth2-proxy
spec:
  rules:
  - host: site.example.com
    http:
      paths:
      - backend:
          serviceName: auth-oauth2-proxy
          servicePort: 80
        path: /oauth2

What am I doing wrong ? Am I missing out some very important step? Someone please help me out.

There are no error logs available in both the pods and this is the serivce status

NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
auth-oauth2-proxy             ClusterIP   10.108.150.27    <none>        80/TCP     165m
web-app                       ClusterIP   10.108.202.197   <none>        8000/TCP   2d15h