Managing multiple domains and services in a Kubernetes cluster can become complex, especially as the number of services grows. Clients typically interact with applications through domain names, and ensuring smooth routing and access control is essential. In this post, we’ll explore how using an Ingress Controller in Kubernetes simplifies domain management, compare it with traditional solutions, and demonstrate how to set up basic authentication for sensitive services.

Traditional Solutions Before Containers

Before the advent of containerization and Kubernetes, managing multiple services under different domains often involved:

  • Dedicated Load Balancers: Hardware or software load balancers were configured to route traffic to the appropriate backend servers.
  • Reverse Proxies: Tools like NGINX or Apache HTTP Server acted as reverse proxies, forwarding requests based on domain names or URL paths.
  • Manual Configuration: Each time a new service was added or updated, administrators manually updated the configuration files and reloaded the services.
  • Scaling Challenges: Managing high availability and scaling required additional effort and infrastructure.

These approaches were often static, time-consuming, and error-prone, especially in dynamic environments where services frequently change.

Introduction to Ingress in Kubernetes

Kubernetes introduces the concept of Ingress to handle external access to services within the cluster:

  • Ingress Resource: Defines rules for routing external HTTP/S traffic to services inside the cluster based on hostnames and paths.
  • Ingress Controller: A Kubernetes component that implements the Ingress rules. It listens for changes to Ingress resources and configures the underlying proxy accordingly.

By using Ingress, you can:

  • Simplify Domain Management: Route multiple domains and paths through a single load balancer or IP address.
  • Automate Configuration: Ingress controllers automatically update routing based on changes to Ingress resources.
  • Enhance Security: Implement TLS termination, authentication, and other security measures centrally.

Choosing an Ingress Controller

There are several Ingress controllers available for Kubernetes, including:

  • NGINX Ingress Controller
  • Traefik
  • HAProxy
  • Istio Ingress Gateway

For this setup, we’ll use the NGINX Ingress Controller because it’s:

  • Fast and Efficient: High performance with low resource consumption.
  • Widely Used: A large community and extensive documentation.
  • Feature-Rich: Supports a wide range of features, including SSL/TLS termination, URL rewriting, and authentication.

Deploying the NGINX Ingress Controller

Step 1: Add the Helm Repository

Helm is a package manager for Kubernetes that simplifies the deployment of complex applications.

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Step 2: Create a Namespace

Create a dedicated namespace for the ingress controller:

kubectl create namespace ingress-nginx

Step 3: Install the Ingress Controller

Install the NGINX Ingress Controller using Helm:

helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx
  • ingress-nginx: The release name.
  • ingress-nginx/ingress-nginx: The chart to install.
  • -namespace ingress-nginx: Specifies the namespace to deploy the controller.

By default, the ingress controller is configured with sensible defaults. If you need to customize settings, you can create a values.yaml file and include it in the installation command:

helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx -f values.yaml

Optional: Configuring TCP/UDP Services

If you need to expose TCP or UDP services (like databases or message brokers), you can define them in the values.yaml file:

tcp:
  "4222": "nats/nats-cluster:4222"
  • "4222": The external port to expose.
  • "nats/nats-cluster:4222": The Kubernetes service and port handling the traffic.

Managing Application Domains with Ingress Resources

With the ingress controller deployed, you can now create Ingress resources to route traffic to your services based on domain names.

Example: Exposing a Backend Service

Suppose you have a backend service named astring-backend-service running in the astring namespace, listening on port 4000.

Step 1: Create an Ingress Resource

Create a file named ingress-backend.yaml with the following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: astring-backend-ingress
  namespace: astring
spec:
  ingressClassName: nginx
  rules:
    - host: api.astring.app
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: astring-backend-service
                port:
                  number: 4000

Step 2: Apply the Ingress Resource

kubectl apply -f ingress-backend.yaml

Explanation

  • ingressClassName: nginx: Specifies the ingress controller to use.
  • host: api.astring.app: The domain name to route.
  • service.name: The name of the Kubernetes service.
  • service.port.number: The port on which the service is listening.

Now, when a client sends a request to api.astring.app, the ingress controller routes the traffic to astring-backend-service on port 4000.

Example: Exposing Prometheus with Authentication

Exposing sensitive services like Prometheus without authentication is a security risk. Let’s see how to add basic authentication to protect it.

Step 1: Create the Ingress Resource with Annotations

Create a file named ingress-prometheus.yaml:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prometheus-ingress
  namespace: prometheus
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: prometheus-basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - Prometheus'
spec:
  ingressClassName: nginx
  rules:
    - host: prometheus.astring.app
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: prometheus-kube-prometheus-prometheus
                port:
                  number: 9090

Step 2: Create the Basic Authentication Secret

First, generate a password file using htpasswd (you may need to install the apache2-utils package if not already installed):

htpasswd -c auth admin
  • c: Creates a new file.
  • auth: The output file name.
  • admin: The username.

You’ll be prompted to enter a password for the user admin.

Create the Kubernetes Secret:

kubectl create secret generic prometheus-basic-auth --from-file=auth -n prometheus
  • prometheus-basic-auth: The name of the secret.
  • -from-file=auth: Uses the auth file created earlier.
  • n prometheus: Specifies the namespace.

Step 3: Apply the Ingress Resource

kubectl apply -f ingress-prometheus.yaml

Explanation

  • Annotations: The annotations configure NGINX to use basic authentication.
    • nginx.ingress.kubernetes.io/auth-type: basic: Enables basic authentication.
    • nginx.ingress.kubernetes.io/auth-secret: prometheus-basic-auth: References the secret containing the credentials.
    • nginx.ingress.kubernetes.io/auth-realm: Customizes the authentication prompt.
  • Ingress Rules: Routes traffic from prometheus.astring.app to the Prometheus service.

Now, when someone tries to access prometheus.astring.app, they’ll be prompted for a username and password.

Benefits of Using Ingress Controllers

  • Centralized Management: Manage all routing rules in a single place.
  • Dynamic Updates: Changes to Ingress resources are automatically picked up by the controller.
  • Scalability: Easily handle a large number of services and domains.
  • Security: Implement authentication, SSL/TLS termination, and other security features centrally.
  • Simplified DNS Configuration: Point multiple domains to the ingress controller’s IP address.

Common Commands and Tips

Checking Ingress Resources

List all Ingress resources in a namespace:

kubectl get ingress -n your-namespace

Describe an Ingress resource for detailed information:

kubectl describe ingress your-ingress-name -n your-namespace

Troubleshooting

  • DNS Configuration: Ensure that your domain names point to the ingress controller’s external IP address.

  • Ingress Controller Status: Verify that the ingress controller pods are running:

    kubectl get pods -n ingress-nginx

  • Logs: Check the logs of the ingress controller for errors:

    kubectl logs -n ingress-nginx deploy/ingress-nginx-controller

  • Service Endpoints: Confirm that services are correctly defined and endpoints are available.

Conclusion

By deploying an ingress controller like NGINX in your Kubernetes cluster, you simplify the management of multiple domains and services. Ingress resources allow you to define routing rules declaratively, and the controller handles the rest.

Additionally, you can enhance security by adding authentication mechanisms, such as basic authentication, to protect sensitive services without modifying the underlying applications.