diff --git a/pages/kubernetes/reference-content/lb-ingress-controller.mdx b/pages/kubernetes/reference-content/lb-ingress-controller.mdx index 56c3453eda..a2ba23df41 100644 --- a/pages/kubernetes/reference-content/lb-ingress-controller.mdx +++ b/pages/kubernetes/reference-content/lb-ingress-controller.mdx @@ -1,22 +1,25 @@ --- meta: - title: Exposing a Kubernetes Kapsule ingress controller service with a Load Balancer + title: Deploying an NGINX ingress controller on Scaleway Kubernetes Kapsule with a LoadBalancer description: This page explains how to expose an application via an ingress object, and using a Load Balancer to make the IP persistent. content: h1: Exposing a Kubernetes Kapsule ingress controller service with a Load Balancer paragraph: This page explains how to expose an application via an ingress object, and using a Load Balancer to make the IP persistent. categories: - - network - kubernetes - - storage - load-balancer tags: compute kapsule kubernetes ingress-controller k8s Load-balancer wildcard dates: - validation: 2025-04-22 - posted: 2020-05-05 + validation: 2025-06-17 + posted: 2025-06-17 --- -This document will guide you through deploying a test application on a Kubernetes cluster, exposing it via an ingress object, and using a Scaleway Load Balancer to ensure persistent IP addressing. +This guide walks you through the process of deploying an NGINX ingress controller on Scaleway's Kubernetes Kapsule service. +We will configure a Load Balancer that uses a persistent IP address, which is essential for maintaining consistent routing. Additionally, we will enable the PROXY protocol to preserve client information such as the original IP address and port, which is recommended for applications that need to log or act on this data. + +The guide also delves into the differences between ephemeral and persistent IP addresses, helping you understand when and why to use each type. To complete the guide, we will deploy a demo application that illustrates the entire setup. + +By the end of this guide, you should have a robust and well-configured NGINX ingress controller running on Scaleway's Kubernetes platform. @@ -24,99 +27,191 @@ This document will guide you through deploying a test application on a Kubernete - [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization - Set up a [Kubernetes Kapsule cluster](/kubernetes/how-to/create-cluster/), deploying a TRAEFIK2 ingress controller via the application library using the [Easy Deploy function](/kubernetes/how-to/enable-easy-deploy/) - Obtained the [kubeconfig](/kubernetes/how-to/edit-cluster/) file for the cluster -- Installed [kubectl](/kubernetes/how-to/connect-cluster-kubectl/) on your local machine +- Helm installed on your local machine +- Installed [kubectl](/kubernetes/how-to/connect-cluster-kubectl/) and the Scaleway CLI on your local machine + +## Overview of key concepts + +### Ingress controller +An ingress controller manages external HTTP/HTTPS traffic to services within a Kubernetes cluster. The NGINX ingress controller routes traffic based on ingress resource rules. + +### LoadBalancer Service +On Scaleway Kapsule, the LoadBalancer service provisions a Scaleway Load Balancer with an external IP, exposing the ingress controller via the Scaleway Cloud Controller Manager (CCM). + +### Ephemeral vs. Persistent IPs +- Ephemeral IP: Dynamically assigned by Scaleway when a LoadBalancer service is created. It may change if the service is deleted and recreated, requiring DNS updates. +- Persistent IP: A flexible IP reserved via the Scaleway API, CLI or console, ensuring consistency across service recreations. This is recommended for production to maintain stable DNS records. -## Exposing the ingress controller using a Scaleway Load Balancer +### PROXY Protocol +The PROXY protocol allows the LoadBalancer to forward the client's original IP address to the ingress controller, preserving source information for logging and security. -By default, ingress controllers on Kapsule are deployed using a [hostPort](https://kubernetes.io/docs/concepts/services-networking/service/). This ensures accessibility on all cluster nodes via ports 80 and 443. However, for production readiness, you might prefer using a Load Balancer to expose your services to the internet. +## Deploying the ingress controller - -By default, a new security group that blocks all incoming traffic on the nodes for security purposes is created during cluster configuration. To allow incoming HTTP/80 and HTTPS/443 traffic, you need to modify the security group. +## Installation prework +Kapsule clusters use a default security group (`kubernetes-`) that blocks incoming traffic. To allow HTTP/HTTPS connections to the cluster: +1. Go to the Scaleway console and navigate to **Compute > CPU & GPU Instances > Security Groups**. +2. Locate the security group `kubernetes-`. +3. Add rules to allow: + - TCP port 80 (HTTP) from `0.0.0.0/0`. + - TCP port 443 (HTTPS) from `0.0.0.0/0`. -1. In the [Scaleway console](https://console.scaleway.com/instance/security-groups), navigate to the **Compute** > **Security groups** section and find the security group named `kubernetes `. -2. Modify the security group rules to allow incoming traffic on ports 80 (HTTP) and 443 (HTTPS). - - Allow TCP traffic on port 80 from all sources (0.0.0.0/0) for HTTP. - - Allow TCP traffic on port 443 from all sources (0.0.0.0/0) for HTTPS. - +## Reserve a flexible IP +To use a persistent IP with the ingress controller: +1. Create a flexible IP using the Scaleway CLI: + ```bash + scw lb ip create + ``` +2. Note the IP address (e.g., `195.154.72.226`) and IP ID for use in the LoadBalancer service. -### Deploying a test application +## Installing the NGINX ingress controller -1. Deploy the `cafe-ingress` test application: +Use Helm to deploy the NGINX ingress controller with Scaleway-specific configurations. +1. Add the NGINX ingress Helm repository ```bash - kubectl create -f https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/main/examples/ingress-resources/basic-auth/cafe.yaml + helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx + helm repo update ``` -2. Create the ingress object (`coffee-ingress.yaml`) using the DNS wildcard provided by Scaleway: - +2. Create a file named `ingress-values.yaml` with and edit the `loadBalancerIP` to your flexible IP: ```yaml - apiVersion: networking.k8s.io/v1 - kind: Ingress - metadata: - name: coffee-ingress - spec: - rules: - - host: YOUR_SCALEWAY_DNS_WILDCARD - http: - paths: - - path: /tea - pathType: Prefix - backend: - service: - name: tea-svc - port: - number: 80 - - path: /coffee - pathType: Prefix - backend: - service: - name: coffee-svc - port: - number: 80 + controller: + service: + type: LoadBalancer + # Specify reserved flexible IP + loadBalancerIP: "195.154.72.226" + annotations: + # Enable PROXY protocol v2 + service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2: "true" + # Use hostname for cert-manager compatibility + service.beta.kubernetes.io/scw-loadbalancer-use-hostname: "true" + config: + # Enable PROXY protocol in NGINX + use-proxy-protocol: "true" + use-forwarded-headers: "true" + compute-full-forwarded-for: "true" ``` - - Your DNS wildcard is composed of your cluster ID (e.g., `68362d3b-57c8-4bea-905a-aeb7f9ab95dc`) followed by `.nodes.k8s..scw.cloud`. For a cluster located in the Paris region, your DNS wildcard could be, for example: `hotdrinks.68362d3b-57c8-4bea-905a-aeb7f9ab95dc.nodes.k8s.fr-par.scw.cloud`. + - Replace `195.154.72.226` with your reserved flexible IP. Omitting `loadBalancerIP` results in an ephemeral IP. + - The `service.beta.kubernetes.io/scw-loadbalancer-proxy-protocol-v2` annotation enables PROXY protocol v2. + - The `service.beta.kubernetes.io/scw-loadbalancer-use-hostname` annotation supports cert-manager HTTP01 challenges. -3. Apply the configuration: - +3. Deploy the ingress controller: ```bash - kubectl create -f coffee-ingress.yaml + helm install ingress-nginx ingress-nginx/ingress-nginx -f ingress-values.yaml --namespace ingress-nginx --create-namespace ``` -4. Test the ingress: - +4. Verify the LoadBalancer IP using `kubectl`: ```bash - curl http://YOUR_SCALEWAY_DNS_WILDCARD/coffee + kubectl get svc -n ingress-nginx ingress-nginx-controller ``` -## Using a reserved IP with a Load Balancer - -Reserve a flexible Load Balancer IP address [through the Scaleway API](/kubernetes/reference-content/managing-load-balancer-ips/#reserve-a-load-balancer-flexible-ip-address-via-the-api). Take note of the IP address, referred to as `RESERVED_IP` from now on. + You will see an output similar to the following example: + ``` + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + ingress-nginx-controller LoadBalancer 10.100.0.1 195.154.72.226 80/TCP,443/TCP 5m + ``` + + - The `EXTERNAL-IP` should match your reserved flexible IP (e.g., `195.154.72.226`). + - If an ephemeral IP appears, verify that the `loadBalancerIP` field is correctly set and matches a valid Load Balancer flexible IP attached to your Scaleway Project. + - Confirm the LoadBalancer in the Scaleway console under **Network > Load Balancers**. + -### Using the reserved IP in Kubernetes +5. Configure DNS by setting the A-Record of your domain (e.g., `demo.example.com`) to the flexible IP via Scaleway's Domains & DNS product or your DNS provider. Persistent IPs ensure stability and will not change as long as they are reserved. -1. Patch `tea-svc` to use the reserved IP with a `LoadBalancer` service: +### Deploying a Demo Application - ```bash - kubectl patch svc tea-svc --type merge --patch '{"spec":{"loadBalancerIP": "RESERVED_IP","type":"LoadBalancer"}}' - ``` +1. Create a file named `demo-app.yaml` and copy the following content into it to deploy a simple web application to test the ingress controller: -2. Delete `tea-svc`: + ```yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: demo-app + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: demo-app + template: + metadata: + labels: + app: demo-app + spec: + containers: + - name: demo-app + image: nginx:1.21 + ports: + - containerPort: 80 + --- + apiVersion: v1 + kind: Service + metadata: + name: demo-app + namespace: default + spec: + selector: + app: demo-app + ports: + - protocol: TCP + port: 80 + targetPort: 80 + --- + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: demo-app-ingress + namespace: default + annotations: + nginx.ingress.kubernetes.io/rewrite-target: / + spec: + ingressClassName: nginx + rules: + - host: demo.example.com + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: demo-app + port: + number: 80 + ``` - ```bash - kubectl delete svc tea-svc - ``` + + - Replace `demo.example.com` with your domain name. + -3. Patch `coffee-svc` to use the reserved IP: +2. Apply the configuration: + ```bash + kubectl apply -f demo-app.yaml + ``` - ```bash - kubectl patch svc coffee-svc --type merge --patch '{"spec":{"loadBalancerIP": "RESERVED_IP","type":"LoadBalancer"}}' - ``` +## Test the Setup +1. Access the demo application: + ```bash + curl http://demo.example.com + # or + curl http://195.154.72.226/ + ``` -## Related tutorials +2. You should see the NGINX welcome page. Verify the PROXY protocol by checking logs for the client's real IP: + ```bash + kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx + ``` -- [Loki monitoring on Kubernetes](/tutorials/manage-k8s-logging-loki/) -- [Monitoring a Kubernetes Kapsule cluster](/tutorials/monitor-k8s-grafana/) -- [Deploy an image from a private registry](/kubernetes/how-to/deploy-image-from-container-registry/) \ No newline at end of file +## Cleanup (optional) +Once fininshed you can remove the demo application and ingress controller from your cluster: +```bash +kubectl delete -f demo-app.yaml +helm uninstall ingress-nginx -n ingress-nginx +kubectl delete namespace ingress-nginx +``` + +To release the flexible IP: +```bash +scw lb ip delete +``` diff --git a/pages/kubernetes/reference-content/wildcard-dns.mdx b/pages/kubernetes/reference-content/wildcard-dns.mdx index 38468e9d12..0b5c299f3f 100644 --- a/pages/kubernetes/reference-content/wildcard-dns.mdx +++ b/pages/kubernetes/reference-content/wildcard-dns.mdx @@ -1,14 +1,14 @@ --- meta: - title: Kubernetes service routing with wildcard DNS and ingress controller + title: Deploying an NGINX ingress controller on Scaleway Kubernetes Kapsule without a LoadBalancer description: This page explains how to route external traffic in Kubernetes using wildcard DNS and ingress controller content: - h1: Kubernetes service routing with wildcard DNS and ingress controller + h1: Deploying an NGINX ingress controller on Scaleway Kubernetes Kapsule without a LoadBalancer paragraph: This page explains how to route external traffic in Kubernetes using wildcard DNS and ingress controller tags: kubernetes load-balancer-service wildcard ingress dates: - validation: 2025-05-12 - posted: 2021-08-12 + validation: 2025-06-17 + posted: 2021-06-17 categories: - kubernetes --- @@ -29,108 +29,201 @@ In short, Kubernetes wildcard DNS, combined with an ingress controller, provides - [Owner](/iam/concepts/#owner) status or [IAM permissions](/iam/concepts/#permission) allowing you to perform actions in the intended Organization - [Created](/kubernetes/how-to/create-cluster/) a Scaleway Kubernetes cluster - Installed `helm` on your local computer -- A domain name - A `TCP` or `HTTP` service you want to expose -## Installing the ingress controller with helm - -1. Add the Helm repository: - ```bash - helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx - helm repo update - ``` - -2. Install the Nginx ingress controller using the helm packet manager: - ```bash - helm install nginx-ingress ingress-nginx/ingress-nginx \ - --namespace ingress-nginx --create-namespace - ``` - -3. Verify the installation: - ```bash - kubectl get pods -n ingress-nginx - ``` - -## Configuring wildcard DNS - -1. Retrieve the ingress IP: - ```bash - kubectl get svc -n ingress-nginx - ``` - - It may take a few minutes for the Load Balancer IP to be assigned. - - -2. Configure Wildcard DNS: - - Go to your DNS provider and add an A record for `*.yourdomain.com` pointing to the IP address of the ingress controller's load balancer. - -## Deploy a sample application - -1. Deploy a sample application by creating a file `hello-world.yaml`. Below is a simple deployment and service example. Copy the content in the file and save it: - ```yaml - apiVersion: apps/v1 - kind: Deployment - metadata: - name: hello-world - spec: - replicas: 2 - selector: - matchLabels: - app: hello-world - template: - metadata: - labels: - app: hello-world - spec: - containers: - - name: hello-world - image: nginxdemos/hello - ports: - - containerPort: 80 - --- - apiVersion: v1 - kind: Service - metadata: - name: hello-world - spec: - ports: - - port: 80 - selector: - app: hello-world - ``` - -2. Apply the configuration with `kubectl apply -f hello-world.yaml`. - -## Create an ingress resource - -1. Copy the following sample resource and paste and save it into a YAML file called `wildcard-ingress.yaml`. Remember to replace the domain name with your own. - ```yaml - apiVersion: networking.k8s.io/v1 - kind: Ingress - metadata: - name: wildcard-ingress - annotations: - nginx.ingress.kubernetes.io/rewrite-target: / - spec: - rules: - - host: "*.yourdomain.com" - http: - paths: - - path: / - pathType: Prefix - backend: - service: - name: hello-world - port: - number: 80 - ``` - -2. Apply the configuration with `kubectl apply -f wildcard-ingress.yaml`. - -## Test your setup - -1. Access your application by pointing your web browser to `http://anything.yourdomain.com` (replace this with your domain name). It should load the hello-world application. - -2. Test with different subdomains, they should all lead to your `hello-world` application due to the wildcard DNS setup. - -You have successfully set up an ingress controller with wildcard DNS on Scaleway Kubernetes Kapsule. This setup allows you to easily manage multiple subdomains and route traffic to the various services in your Kubernetes cluster. \ No newline at end of file +## Overview of key concepts + +### Ingress controller +An ingress controller manages external HTTP/HTTPS traffic to services within a Kubernetes cluster based on Ingress resource rules. The NGINX ingress controller is used here. + +### NodePort service +A `NodePort` service exposes the ingress controller on a specific port (e.g., 30000–32767) on each cluster node’s IP address, allowing external access without a Scaleway LoadBalancer. + +### Round-robin DNS +Scaleway provides a wildcard DNS record (`.nodes.k8s..scw.cloud`) that resolves to the public IPs of all nodes in the default pool. This acts as a round-robin DNS, distributing traffic randomly across nodes. +However, this is not true load balancing, as DNS resolution is client-dependent and may not evenly distribute traffic. Consider [deploying a LoadBalancer service](/kubernetes/reference-content/lb-ingress-controller/) for production use. + +### Full isolation node pools +Scaleway Kapsule supports “full isolation” node pools, where nodes lack public IPs. This is incompatible with `NodePort`-based ingress without additional configuration, as external traffic cannot reach nodes directly. A node selector is required to ensure the ingress controller runs on nodes with public IPs (e.g., in the default pool). + +### Security groups +Kapsule clusters use a default security group (`kubernetes-`) that blocks incoming traffic by default. You must open ports (e.g., 80 and 443) to allow external access to the `NodePort`. + +## Deployment of the ingress controller + +## Installation prework + +To allow HTTP/HTTPS traffic to the `NodePort`: +1. In the Scaleway console, navigate to **Compute > CPU & GPU Instances > Security Groups**. +2. Locate the security group `kubernetes-`. +3. Add rules to allow: + - TCP port 80 (HTTP) from `0.0.0.0/0`. + - TCP port 443 (HTTPS) from `0.0.0.0/0`. + - TCP port range 30000–32767 (`NodePort` range) from `0.0.0.0/0` for external access to the ingress controller. + + Opening the `NodePort` range exposes all `NodePort` services. For production, consider restricting the source IP range or using a more specific port if known. + + +## Installing the NGINX ingress controller +Use Helm to deploy the NGINX ingress controller with a `NodePort` service and a node selector to ensure it runs on nodes with public IPs. + +1. Add the NGINX Ingress Helm Repository + ```bash + helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx + helm repo update + ``` + +2. Create a file named `ingress-values.yaml` and copy the following values into it: + ```yaml + controller: + service: + type: NodePort + # Specify NodePort values (optional, Kubernetes assigns random ports if omitted) + nodePorts: + http: 30080 + https: 30443 + # Node selector to ensure pods run on nodes with public IPs + nodeSelector: + scaleway.com/pool-name: + config: + use-forwarded-headers: "true" + compute-full-forwarded-for: "true" + admissionWebhooks: + enabled: true + ``` + + - `type: `NodePort`` exposes the ingress controller on the specified ports (e.g., 30080 for HTTP, 30443 for HTTPS) on each node’s public IP. + - The `nodeSelector` ensures the ingress controller pods run on nodes in the `` pool, which have public IPs. Replace `scaleway.com/pool-name: ` if your public IP pool has a different name (check via `kubectl get nodes -o wide`). + - Full isolation node pools lack public IPs, making them incompatible with `NodePort`-based ingress unless routed through nodes with public IPs (e.g., default pool). + - `admissionWebhooks.enabled: true` ensures the validating webhook is enabled for ingress resource validation. + + +3. Deploy the ingress controller using `helm` + ```bash + helm install ingress-nginx ingress-nginx/ingress-nginx --version 4.7.0 -f ingress-values.yaml --namespace ingress-nginx --create-namespace + ``` + +4. Check the `NodePort` service: + ```bash + kubectl get svc -n ingress-nginx ingress-nginx-controller + ``` + + You will see an output similar to the following example: + ``` + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + ingress-nginx-controller `NodePort` 10.32.12.0 80:30080/TCP,443:30443/TCP 5m + ``` + + - The `PORT(S)` field shows the `NodePort`s (e.g., `30080` for HTTP, `30443` for HTTPS). + - The `EXTERNAL-IP` is `` because `NodePort` uses the nodes’ IPs. + +5. Get the public IPs of the nodes in the default pool: + ```bash + kubectl get nodes -o wide + ``` + + You will see an output similar to the following example + ``` + NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP + node-1 Ready 1h v1.32.3 172.16.28.3 195.154.111.100 + node-2 Ready 1h v1.32.3 172.16.28.4 195.154.111.101 + ``` + +6. Configure DNS with round-robin DNS: + - Use Scaleway’s wildcard DNS (`.nodes.k8s..scw.cloud`) for external access. + For example, for a cluster with ID `39b054d2-5255-4fe3-a327-7ccfac7ffdca` in the `fr-par` region, the DNS is `39b054d2-5255-4fe3-a327-7ccfac7ffdca.nodes.k8s.fr-par.scw.cloud`. This domain name resolves to the public IPs of all nodes in the default pool (e.g., `195.154.72.100`, `195.154.72.101`). + + -Alternatively, for a custom domain, map your domain (e.g., `demo.example.com`) to all node IPs via your DNS provider, creating multiple A records: + ``` + demo.example.com A 195.154.111.100 + demo.example.com A 195.154.111.101 + ``` + + DNS round-robin is not true load balancing. Client DNS resolution may favor one node, leading to uneven traffic distribution. For production, consider [using a Scaleway LoadBalancer](/kubernetes/reference-content/lb-ingress-controller/) instead. + + +7. Create a file named `demo-app.yaml` and copy the follwoing manifest into it to deploy a simple web application to test the ingress controller: + ```yaml + apiVersion: apps/v1 + kind: Deployment + metadata: + name: demo-app + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: demo-app + template: + metadata: + labels: + app: demo-app + spec: + containers: + - name: demo-app + image: nginx:1.21 + ports: + - containerPort: 80 + --- + apiVersion: v1 + kind: Service + metadata: + name: demo-app + namespace: default + spec: + selector: + app: demo-app + ports: + - protocol: TCP + port: 80 + targetPort: 80 + --- + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: demo-app-ingress + namespace: default + annotations: + nginx.ingress.kubernetes.io/rewrite-target: / + spec: + ingressClassName: nginx + rules: + - host: demo..nodes.k8s..scw.cloud + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: demo-app + port: + number: 80 + ``` + + Replace `` with your Kapsule cluster ID (found in the Scaleway console) and `` with the region of your cluster. For a custom domain, use `demo.example.com`. + + +8. Apply the configuration: + ```bash + kubectl apply -f demo-app.yaml + ``` + +9. Access the demo application using the wildcard DNS and the HTTP `NodePort` (e.g., 30080): + ```bash + curl http://demo..nodes.k8s.fr-par.scw.cloud:30080/ + # or, using a node’s public IP + curl http://195.154.111.100:30080/ + # or, with a custom domain + curl http://demo.example.com:30080/ + ``` + +You should see the NGINX welcome page. Test HTTPS if configured (port 30443). + + +## Cleanup (optional) +Once fininshed you can remove the demo application and ingress controller from your cluster: +```bash +kubectl delete -f demo-app.yaml +helm uninstall ingress-nginx -n ingress-nginx +kubectl delete namespace ingress-nginx +``` \ No newline at end of file