Will Ingress work in a simple http-only scenario?
Let us try a simple http ingress into a standard nginx webserver pod. To do this,create a DNS record called k8s, for yourdomain.tld. It should be an ‘A’ record pointing to your outgoing ip address (curl ifconfig.me). Wait for dns to update, and nslookup k8s.yourdomain.tld to return the IP address.
Next copy the script below into your favorite editor.
# Namespace
apiVersion: v1
kind: Namespace
metadata:
name: test
---
# Pod
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
namespace: test
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
---
# Service
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: test
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 5555
targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: test
name: nginx-ingress
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
rules:
- host: "k8s.yourdomain.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: nginx-service
port:
number: 5555
Replace the hostname in the ingress to your newly created name, the one you just looked up. Save the file as manifest.yaml. You will notice the service (traffic coming in) is at port 5555, an arbitrary high port number and not 80, or 443, so that we do not conflict with the hosts open ports. The service routes to the pod’s port 80, because that is where nginx is listening. Now the ingress points to the service name but also to the port, so the port numbers for ingress and service must match!
Now open a terminal and run kubectl as shown. The expected output is also shown.
$ kubectl apply -f manifest.yaml pod/nginx-pod configured service/nginx-service configured ingress.networking.k8s.io/nginx-ingress configured
Once this is cool, try to load the page. Modern web browsers give users severe warning when navigating to http only websites, it is easier to use wget. Below is an example of what the output should look like when it works.
$ wget -q -O - http://k8s.brunzema.com (k3d-k3d-cluster/test)
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
If the ingress is not working, do not proceed trying to install wordpress before it does. You will need to retrace your steps and troubleshoot. Use kubectl delete -f manifest.yaml to remove the test pod, service, ingress and namespace once things work.
Will https ingress, complete with TLS termination work for the nginx service?
We are going to run another experiment to make sure that https, certificates etc. work for the nginx sample site first. Reason for this is this is better than a complex application like WordPress, it is just easier to know what is going on.
The first part, the service and deployment are almost identical to the http version, I just changed the port number, so that it does not conflict with the previous example.
# Namespace
apiVersion: v1
kind: Namespace
metadata:
name: test
---
# Pod
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod
namespace: test
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
---
# Service
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: test
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 5678
targetPort: 80
Next comes the ingress, save this as ingress-cert-mgr.yaml and make sure you change the host name to your domain.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: test
name: nginx-ingress-cert-bot
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts:
- "k8s.yourdomain.tld"
secretName: nginx-tls
rules:
- host: "k8s.yourdomain.tld"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: nginx-service
port:
number: 5678
What’s happening in this setup? The ingress configuration is used as a workaround to help cert-manager obtain a secret. The “secretName” field in the ingress specifies the name of the secret that cert-manager will create to store the actual secret data. Make sure your pod and service are running because the ingress depends on them.
When you apply the ingress configuration (kubectl apply -f ingress-cert-mgr.yaml), cert-manager detects the cert-manager.io/cluster-issuer annotation in the ingress and finds a valid cluster issuer in your cluster. Cert-manager then sets up a temporary HTTP server. This server contacts the ACME server (Automated Certificate Management Engine, not the company from Wiley Coyote) to perform a challenge, demonstrating that you own the domain by pointing it to your server. ACME generates the TLS certificate and sends it back to the server, which stores the private key in the specified secret.
This process takes a few minutes. You can check the progress by running kubectl get secrets. Initially, you’ll see a secret name with a random hex suffix. After 1-2 minutes, it will change to nginx-tls, and you can view the private key by inspecting the nginx-tls secret.
Once the secret is created, you can delete the ingress (kubectl delete -f ingress.yaml) as it has served its purpose. Unfortunately, there is no direct integration with Traefik, so this workaround is necessary.
Now create the real ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: test
name: nginx-ingress
annotations:
traefik.ingress.kubernetes.io/router.middlewares: cert-manager-redirect-https@kubernetescrd
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
tls:
- hosts:
- "k8s.yourdomain.tld"
secretName: nginx-tls
rules:
- host: "k8s.yourdomain.tld"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: nginx-service
port:
number: 5678
You will notice a couple of changes. In the annotations, we tell traefik that we want http->https connections upgrades automatically, and that the traffic routes to web internally. The spec now has a tls: section, specifying the host name and secret name containing the valid tls cert that matches the host name. The rules: section is pretty vanilla after this.
Save the file as ingress-tls.yaml and apply it. If all goes well, navigating to https://k8s.yourdomain.tld will get you a TLS secured connection and a very happy browser, as can be seen below.
Now that we know the pattern we need to have in ingress and service, and about the trick to create the secret with a fake temporary ingress, we will be able to change the helm-generated service and ingress. Running curl with -vvv, will show the entire TLS negotiation in its full glory, and reveal interesting details about the certificate.
$ curl -vvv https://k8s.brunzema.com (k3d-k3d-cluster/test) * Host k8s.brunzema.com:443 was resolved. * IPv6: (none) * IPv4: 174.95.83.83 * Trying 174.95.83.83:443... * Connected to k8s.brunzema.com (174.95.83.83) port 443 * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/certs/ca-certificates.crt * CApath: none * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): * TLSv1.3 (IN), TLS handshake, Certificate (11): * TLSv1.3 (IN), TLS handshake, CERT verify (15): * TLSv1.3 (IN), TLS handshake, Finished (20): * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.3 (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / x25519 / RSASSA-PSS * ALPN: server accepted h2 * Server certificate: * subject: CN=k8s.brunzema.com * start date: Mar 3 05:10:56 2024 GMT * expire date: Jun 1 05:10:55 2024 GMT * subjectAltName: host "k8s.brunzema.com" matched cert's "k8s.brunzema.com" * issuer: C=US; O=Let's Encrypt; CN=R3 * SSL certificate verify ok. * Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): * using HTTP/2 * [HTTP/2] [1] OPENED stream for https://k8s.brunzema.com/ * [HTTP/2] [1] [:method: GET] * [HTTP/2] [1] [:scheme: https] * [HTTP/2] [1] [:authority: k8s.brunzema.com] * [HTTP/2] [1] [:path: /] * [HTTP/2] [1] [user-agent: curl/8.6.0] * [HTTP/2] [1] [accept: */*] > GET / HTTP/2 > Host: k8s.brunzema.com > User-Agent: curl/8.6.0 > Accept: */* > < HTTP/2 200 < accept-ranges: bytes < content-type: text/html < date: Sun, 03 Mar 2024 06:32:41 GMT < etag: "5c0692e1-264" < last-modified: Tue, 04 Dec 2018 14:44:49 GMT < server: nginx/1.14.2 < content-length: 612 < <!DOCTYPE html> ...
Next steps
In the next post, we will look at changing the helm-generated wordpress service and ingress to match our needs — we will replicate the tls ingress that works in the simpler service shown above.