Kubernetes TLS Security Hardening Guide (Traefik & Nginx)

Step-by-step guide on How to Secure Kubernetes TLS Security Hardening Guide (Traefik & Nginx)

Kubernetes TLS Security Hardening Guide

Comprehensive guide for hardening TLS configuration in Kubernetes using Traefik or Nginx Ingress Controller

Table of Contents


Introduction

The Security Problem

TLS 1.0 and TLS 1.1 are deprecated and insecure protocols that have been superseded by TLS 1.2 and TLS 1.3. Despite this, many web servers and load balancers continue to support these older protocols by default, creating security vulnerabilities.

Common Pentesting Findings

During penetration testing and security audits, assessors commonly flag:

  1. Weak TLS Protocol Support

    • TLS 1.0 (Released 1999, deprecated in 2020)
    • TLS 1.1 (Released 2006, deprecated in 2020)
    • Vulnerable to attacks like BEAST, POODLE, and Lucky13
  2. Weak Cipher Suites

    • Non-AEAD (Authenticated Encryption with Associated Data) ciphers
    • CBC-mode ciphers vulnerable to padding oracle attacks
    • Export-grade ciphers with weak key lengths
    • RC4-based ciphers
    • 3DES ciphers
  3. Missing Security Headers

    • Lack of strict SNI (Server Name Indication) validation

Why This Matters

Compliance Requirements

  • PCI DSS 3.2+: Requires TLS 1.2 or higher
  • NIST Guidelines: Recommend disabling TLS 1.0/1.1
  • HIPAA: Requires “strong cryptography”
  • GDPR: Requires state-of-the-art security measures
  • ISO 27001: Mandates secure communication protocols

Real-World Impact

  • Data Interception: Weak protocols can be exploited to decrypt traffic
  • Man-in-the-Middle Attacks: Downgrade attacks force clients to use weaker protocols
  • Compliance Penalties: Non-compliance can result in fines and audit failures
  • Reputation Damage: Security breaches erode customer trust
  • Browser Warnings: Modern browsers show warnings for sites using deprecated TLS

What We’re Going to Fix

This guide will show you how to:

  1. βœ… Disable TLS 1.0 and TLS 1.1 completely
  2. βœ… Enable only TLS 1.2 and TLS 1.3 (if supported)
  3. βœ… Use only modern AEAD cipher suites (GCM, ChaCha20-Poly1305)
  4. βœ… Enable strict SNI validation
  5. βœ… Use strong elliptic curves (P-384, P-521)
  6. βœ… Make these settings cluster-wide for all services

Prerequisites

Required Components

Before starting, ensure you have:

ComponentVersionPurpose
Kubernetes1.19+Container orchestration platform
Traefik OR Nginx Ingress2.0+ / 1.0+Ingress controller (choose one)
kubectlLatestKubernetes CLI tool
cert-manager1.0+ (optional)Automatic TLS certificate management
OpenSSL1.1.1+For testing and validation

Note: This guide covers both Traefik and Nginx Ingress Controller. Choose the section that matches your setup:

Required Permissions

You need the following Kubernetes permissions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Minimum RBAC permissions required
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
rules:
  - apiGroups: ["traefik.io"]
    resources: ["tlsoptions"]
    verbs: ["create", "get", "list", "update", "patch", "delete"]
  - apiGroups: ["networking.k8s.io"]
    resources: ["ingresses"]
    verbs: ["get", "list", "update", "patch"]
  - apiGroups: [""]
    resources: ["pods", "services"]
    verbs: ["get", "list"]

Verify Traefik Installation

1
2
3
4
5
6
7
8
# Check Traefik is installed
kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik

# Verify TLSOption CRD is available
kubectl get crd tlsoptions.traefik.io

# Check Traefik version
kubectl get deployment -n kube-system traefik -o jsonpath='{.spec.template.spec.containers[0].image}'

Expected output:

1
2
NAME                      READY   STATUS    RESTARTS   AGE
traefik-5d4f5c7d9-abcde   1/1     Running   0          10d

Choosing Your Ingress Controller

This guide covers TLS hardening for both Traefik and Nginx Ingress Controller. If you’re unsure which one you have, run:

1
2
3
4
5
6
7
# Check for Traefik
kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik
kubectl get crd tlsoptions.traefik.io 2>/dev/null

# Check for Nginx Ingress
kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller
kubectl get configmap -n ingress-nginx nginx-configuration 2>/dev/null

Quick Comparison

FeatureTraefikNginx Ingress
Configuration MethodCustom Resource (TLSOption CRD)ConfigMap + Annotations
ScopeNamespace-scopedCluster-wide (ConfigMap)
GranularityPer-namespace TLS policiesGlobal with per-ingress overrides
ComplexityMore Kubernetes-nativeMore traditional nginx config
Dynamic UpdatesAutomaticRequires controller reload
Best ForCloud-native, multiple teamsTraditional ops, single config

Which Section Should You Follow?


Implementation with Traefik

Note: If you’re using Nginx Ingress Controller instead, skip to Implementation with Nginx Ingress

Understanding Traefik Components

Traefik TLSOption CRD

Traefik uses a Custom Resource Definition (CRD) called TLSOption to configure TLS settings. This resource allows you to define:

  • Minimum and maximum TLS versions
  • Allowed cipher suites
  • Elliptic curve preferences
  • Client authentication settings
  • SNI strictness

Key Concept: TLSOptions are namespace-scoped, meaning each namespace needs its own TLSOption resource.

How Traefik Processes TLS

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Client    β”‚
β”‚  (Browser)  β”‚
β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
       β”‚ HTTPS Request
       β”‚
       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      Traefik Ingress Controller     β”‚
β”‚                                     β”‚
β”‚  1. Receives connection            β”‚
β”‚  2. Checks Ingress annotations     β”‚
β”‚  3. Loads TLSOption CRD            β”‚
β”‚  4. Negotiates TLS handshake       β”‚
β”‚  5. Applies cipher/version rules   β”‚
β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
       β”‚ HTTP (unencrypted)
       β”‚
       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Your Service   β”‚
β”‚   (Backend)     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Flow Explanation:

  1. Client initiates HTTPS connection to your domain
  2. Traefik receives the connection at the edge
  3. Traefik reads the Ingress resource to find TLS configuration
  4. The annotation traefik.ingress.kubernetes.io/router.tls.options points to a TLSOption
  5. Traefik applies the TLSOption rules during TLS handshake
  6. If negotiation succeeds, traffic is decrypted and forwarded to backend

Annotation Format

The annotation uses this format:

1
traefik.ingress.kubernetes.io/router.tls.options: <namespace>-<name>@kubernetescrd

Breaking it down:

  • <namespace>: The Kubernetes namespace where TLSOption is deployed
  • <name>: The name of the TLSOption resource
  • @kubernetescrd: Tells Traefik to look for a Kubernetes CRD (not a file-based config)

Example:

1
traefik.ingress.kubernetes.io/router.tls.options: prod-secure-tls@kubernetescrd

This references a TLSOption named secure-tls in the prod namespace.


Implementation Steps

Step 1: Create TLSOption Resources

Understanding the TLSOption Specification

Here’s a fully documented TLSOption resource:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: secure-tls
  namespace: your-namespace  # Must match your application namespace
  labels:
    security.policy: strict-tls
    managed-by: platform-team
spec:
  # Minimum TLS version allowed (TLS 1.2)
  minVersion: VersionTLS12

  # Maximum TLS version (optional, defaults to highest available)
  # maxVersion: VersionTLS13

  # Allowed cipher suites (AEAD only for maximum security)
  # Order matters: most preferred first
  cipherSuites:
    # ECDHE with AES-128 in GCM mode (fast, secure)
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

    # ECDHE with AES-256 in GCM mode (more secure, slightly slower)
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

    # ChaCha20-Poly1305 (modern, fast on mobile)
    - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
    - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305

  # Elliptic curves for key exchange (strongest first)
  curvePreferences:
    - CurveP521  # 521-bit curve (highest security)
    - CurveP384  # 384-bit curve (balanced)

  # Strict SNI: reject connections without valid SNI
  sniStrict: true

  # Client authentication (optional, for mTLS)
  # clientAuth:
  #   secretNames:
  #     - ca-cert-secret
  #   clientAuthType: RequireAndVerifyClientCert

Cipher Suite Explanation

Cipher SuiteKey ExchangeEncryptionAuthenticationHash
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256ECDHEAES-128-GCMECDSASHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256ECDHEAES-128-GCMRSASHA256
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384ECDHEAES-256-GCMECDSASHA384
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384ECDHEAES-256-GCMRSASHA384
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305ECDHEChaCha20ECDSAPoly1305
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305ECDHEChaCha20RSAPoly1305

Why these ciphers?

  • βœ… ECDHE: Perfect Forward Secrecy (PFS) - past sessions can’t be decrypted if key is compromised
  • βœ… GCM/Poly1305: AEAD modes provide encryption + authentication in one operation
  • βœ… AES-128/256: Industry standard, hardware-accelerated on most CPUs
  • βœ… ChaCha20: Fast on mobile devices without AES acceleration

Why NOT these ciphers?

  • ❌ CBC mode: Vulnerable to padding oracle attacks (Lucky13, POODLE)
  • ❌ RC4: Known weaknesses, deprecated
  • ❌ 3DES: 64-bit block size, sweet32 attack
  • ❌ Static RSA: No forward secrecy

Option A: Using kubectl apply

Create a TLSOption file for each namespace:

File: tlsoption-prod.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: secure-tls
  namespace: production
spec:
  minVersion: VersionTLS12
  cipherSuites:
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
    - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
  curvePreferences:
    - CurveP521
    - CurveP384
  sniStrict: true

Apply the resource:

1
2
3
4
5
6
7
8
# Apply to specific namespace
kubectl apply -f tlsoption-prod.yaml

# Verify creation
kubectl get tlsoption -n production

# View details
kubectl describe tlsoption secure-tls -n production

Option B: Using Kustomize (Recommended for Multiple Environments)

Kustomize allows you to define a base configuration and overlay it across multiple environments.

Directory Structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
tls-security/
β”œβ”€β”€ base/
β”‚   β”œβ”€β”€ kustomization.yaml
β”‚   └── tlsoption.yaml
└── overlays/
    β”œβ”€β”€ production/
    β”‚   β”œβ”€β”€ kustomization.yaml
    β”‚   └── namespace.yaml
    β”œβ”€β”€ staging/
    β”‚   β”œβ”€β”€ kustomization.yaml
    β”‚   └── namespace.yaml
    └── development/
        β”œβ”€β”€ kustomization.yaml
        └── namespace.yaml

File: base/tlsoption.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: secure-tls
spec:
  minVersion: VersionTLS12
  cipherSuites:
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
    - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
  curvePreferences:
    - CurveP521
    - CurveP384
  sniStrict: true

File: base/kustomization.yaml

1
2
3
4
5
6
7
8
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - tlsoption.yaml

commonLabels:
  security.policy: strict-tls
  managed-by: platform-team

File: overlays/production/kustomization.yaml

1
2
3
4
5
6
7
8
9
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: production

resources:
  - ../../base

commonLabels:
  environment: production

File: overlays/staging/kustomization.yaml

1
2
3
4
5
6
7
8
9
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: staging

resources:
  - ../../base

commonLabels:
  environment: staging

Deploy with Kustomize:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Preview what will be applied
kubectl kustomize overlays/production

# Apply to production
kubectl apply -k overlays/production

# Apply to staging
kubectl apply -k overlays/staging

# Apply to all environments
for env in production staging development; do
  echo "Deploying to $env..."
  kubectl apply -k overlays/$env
done

Verify:

1
2
3
4
5
6
7
8
# Check all TLSOptions across namespaces
kubectl get tlsoption -A

# Expected output:
# NAMESPACE    NAME          AGE
# production   secure-tls    30s
# staging      secure-tls    30s
# development  secure-tls    30s

Option C: Using Helm

If you’re using Helm charts, add the TLSOption as a template:

File: templates/tlsoption.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{{- if .Values.tls.enabled }}
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: {{ .Values.tls.optionName | default "secure-tls" }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
spec:
  minVersion: {{ .Values.tls.minVersion | default "VersionTLS12" }}
  cipherSuites:
    {{- toYaml .Values.tls.cipherSuites | nindent 4 }}
  curvePreferences:
    {{- toYaml .Values.tls.curvePreferences | nindent 4 }}
  sniStrict: {{ .Values.tls.sniStrict | default true }}
{{- end }}

File: values.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
tls:
  enabled: true
  optionName: secure-tls
  minVersion: VersionTLS12
  sniStrict: true
  cipherSuites:
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
    - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
  curvePreferences:
    - CurveP521
    - CurveP384

Step 2: Annotate Your Ingress Resources

Before: Insecure Ingress (Default Configuration)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: production
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    # No TLS options specified - uses Traefik defaults
    # This allows TLS 1.0/1.1 and weak ciphers!
spec:
  ingressClassName: traefik
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: myapp-service
                port:
                  number: 80
  tls:
    - hosts:
        - myapp.example.com
      secretName: myapp-tls-cert

Security Issues:

  • ⚠️ No TLS version restrictions
  • ⚠️ Allows weak cipher suites
  • ⚠️ Vulnerable to downgrade attacks
  • ⚠️ Fails PCI DSS compliance

After: Secure Ingress (With TLSOption)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: production
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    # Add this annotation to reference TLSOption
    traefik.ingress.kubernetes.io/router.tls.options: production-secure-tls@kubernetescrd
spec:
  ingressClassName: traefik
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: myapp-service
                port:
                  number: 80
  tls:
    - hosts:
        - myapp.example.com
      secretName: myapp-tls-cert

Security Improvements:

  • βœ… TLS 1.2+ enforced
  • βœ… Only AEAD cipher suites
  • βœ… Strong elliptic curves
  • βœ… Strict SNI validation
  • βœ… PCI DSS compliant

Annotation Breakdown

1
2
3
4
5
6
traefik.ingress.kubernetes.io/router.tls.options: production-secure-tls@kubernetescrd
β”‚                                                 β”‚         β”‚           β”‚
β”‚                                                 β”‚         β”‚           └─ Provider type (CRD)
β”‚                                                 β”‚         └─ Resource name
β”‚                                                 └─ Namespace
└─ Traefik annotation key

Important Notes:

  1. Namespace must match: The namespace in the annotation must match where the TLSOption is deployed
  2. Format is strict: Always use <namespace>-<name>@kubernetescrd format
  3. One TLSOption per Ingress: Each Ingress can only reference one TLSOption
  4. Default fallback: If TLSOption not found, Traefik uses default (insecure) settings

Updating Existing Ingresses

Option 1: Using kubectl patch

1
2
3
4
5
6
# Patch a single Ingress
kubectl patch ingress myapp-ingress -n production \
  -p '{"metadata":{"annotations":{"traefik.ingress.kubernetes.io/router.tls.options":"production-secure-tls@kubernetescrd"}}}'

# Verify the patch
kubectl get ingress myapp-ingress -n production -o yaml | grep tls.options

Option 2: Using kubectl edit

1
2
3
kubectl edit ingress myapp-ingress -n production
# Add the annotation manually in your editor
# Save and exit

Option 3: Script for bulk updates

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/bin/bash
# update-ingresses.sh

NAMESPACE="production"
TLSOPTION="production-secure-tls@kubernetescrd"

# Get all ingresses in namespace
INGRESSES=$(kubectl get ingress -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}')

for ingress in $INGRESSES; do
  echo "Updating $ingress..."
  kubectl patch ingress $ingress -n $NAMESPACE \
    -p "{\"metadata\":{\"annotations\":{\"traefik.ingress.kubernetes.io/router.tls.options\":\"$TLSOPTION\"}}}"
done

echo "All ingresses updated!"

Helm Chart Integration

If using Helm, update your ingress template:

File: templates/ingress.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ include "myapp.fullname" . }}
  namespace: {{ .Release.Namespace }}
  annotations:
    {{- if .Values.ingress.certManager.enabled }}
    cert-manager.io/cluster-issuer: {{ .Values.ingress.certManager.issuer }}
    {{- end }}
    {{- if .Values.ingress.tls.enabled }}
    traefik.ingress.kubernetes.io/router.tls.options: {{ .Release.Namespace }}-secure-tls@kubernetescrd
    {{- end }}
    {{- with .Values.ingress.annotations }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
spec:
  ingressClassName: {{ .Values.ingress.className }}
  rules:
    {{- range .Values.ingress.hosts }}
    - host: {{ .host | quote }}
      http:
        paths:
          {{- range .paths }}
          - path: {{ .path }}
            pathType: {{ .pathType }}
            backend:
              service:
                name: {{ include "myapp.fullname" $ }}
                port:
                  number: {{ $.Values.service.port }}
          {{- end }}
    {{- end }}
  {{- if .Values.ingress.tls.enabled }}
  tls:
    {{- range .Values.ingress.hosts }}
    - hosts:
        - {{ .host }}
      secretName: {{ .host | replace "." "-" }}-tls
    {{- end }}
  {{- end }}
{{- end }}

File: values.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
ingress:
  enabled: true
  className: traefik
  annotations: {}
  tls:
    enabled: true
  certManager:
    enabled: true
    issuer: letsencrypt-prod
  hosts:
    - host: myapp.example.com
      paths:
        - path: /
          pathType: Prefix

Step 3: Deploy the Changes

Pre-Deployment Checklist

  • TLSOption resources created in all namespaces
  • Ingress resources updated with annotations
  • Changes tested in non-production first
  • Monitoring/alerting configured
  • Rollback plan prepared
  • Team notified of maintenance window

Deployment Strategy

Strategy 1: Gradual Rollout (Recommended)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# Step 1: Deploy to development
kubectl apply -k overlays/development
kubectl patch ingress -n development --all \
  -p '{"metadata":{"annotations":{"traefik.ingress.kubernetes.io/router.tls.options":"development-secure-tls@kubernetescrd"}}}'

# Wait 10 minutes, monitor logs
sleep 600

# Step 2: Deploy to staging
kubectl apply -k overlays/staging
kubectl patch ingress -n staging --all \
  -p '{"metadata":{"annotations":{"traefik.ingress.kubernetes.io/router.tls.options":"staging-secure-tls@kubernetescrd"}}}'

# Wait 30 minutes, run validation
sleep 1800

# Step 3: Deploy to production (during maintenance window)
kubectl apply -k overlays/production
kubectl patch ingress -n production --all \
  -p '{"metadata":{"annotations":{"traefik.ingress.kubernetes.io/router.tls.options":"production-secure-tls@kubernetescrd"}}}'

Strategy 2: Blue-Green Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Create new ingress with TLS hardening
kubectl apply -f ingress-secure.yaml

# Test the new ingress
curl -v --tls-max 1.1 https://myapp-secure.example.com  # Should fail

# Switch traffic (update DNS or load balancer)
# Monitor for 24 hours

# Delete old insecure ingress
kubectl delete ingress myapp-ingress-old -n production

Strategy 3: Canary Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Use Traefik weighted routing
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
  name: myapp-weighted
  namespace: production
spec:
  weighted:
    services:
      - name: myapp-secure    # New secure ingress
        weight: 10            # 10% traffic
      - name: myapp-insecure  # Old ingress
        weight: 90            # 90% traffic

Gradually increase weight to secure service:

1
2
3
# 10% β†’ 50% β†’ 100%
kubectl patch traefikservice myapp-weighted -n production \
  --type=json -p='[{"op": "replace", "path": "/spec/weighted/services/0/weight", "value": 50}]'

Post-Deployment Verification

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# 1. Verify TLSOptions are active
kubectl get tlsoption -A

# 2. Check Traefik logs for errors
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik --tail=100

# 3. Verify ingress annotations
kubectl get ingress -A -o jsonpath='{range .items[*]}{.metadata.namespace}{"\t"}{.metadata.name}{"\t"}{.metadata.annotations.traefik\.ingress\.kubernetes\.io/router\.tls\.options}{"\n"}{end}'

# 4. Test TLS connection
openssl s_client -connect myapp.example.com:443 -tls1_2

# 5. Monitor error rates
kubectl top pods -n production

Verification (Traefik)

Immediate Verification

1. Check TLSOption Resource Status

1
2
3
4
5
6
7
8
# List all TLSOptions
kubectl get tlsoption -A

# Describe specific TLSOption
kubectl describe tlsoption secure-tls -n production

# View YAML
kubectl get tlsoption secure-tls -n production -o yaml

Expected Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: secure-tls
  namespace: production
  creationTimestamp: "2024-01-15T10:00:00Z"
  generation: 1
  resourceVersion: "12345"
  uid: abc-def-ghi
spec:
  minVersion: VersionTLS12
  cipherSuites:
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    # ... other ciphers
  curvePreferences:
    - CurveP521
    - CurveP384
  sniStrict: true

2. Verify Ingress Annotations

1
2
3
4
5
6
7
8
# Check single ingress
kubectl get ingress myapp-ingress -n production -o yaml | grep -A 3 annotations

# List all ingresses with TLS options
kubectl get ingress -A -o custom-columns=\
NAMESPACE:.metadata.namespace,\
NAME:.metadata.name,\
TLS-OPTION:.metadata.annotations.'traefik\.ingress\.kubernetes\.io/router\.tls\.options'

Expected Output:

1
2
3
4
NAMESPACE    NAME              TLS-OPTION
production   myapp-ingress     production-secure-tls@kubernetescrd
production   api-ingress       production-secure-tls@kubernetescrd
staging      test-ingress      staging-secure-tls@kubernetescrd

3. Check Traefik Configuration

1
2
3
4
5
6
7
8
# Access Traefik dashboard (if enabled)
kubectl port-forward -n kube-system $(kubectl get pods -n kube-system -l app.kubernetes.io/name=traefik -o name) 9000:9000

# Visit http://localhost:9000/dashboard/
# Navigate to: HTTP > Routers > Your Router > TLS

# Or query Traefik API
curl http://localhost:9000/api/http/routers | jq '.[] | select(.name=="myapp") | .tls'

4. Monitor Traefik Logs

1
2
3
4
5
6
7
8
# Watch Traefik logs in real-time
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik -f

# Look for TLS handshake errors
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik | grep -i "tls"

# Check for protocol version rejections
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik | grep -i "protocol version"

Good Log Example:

1
time="2024-01-15T10:05:00Z" level=info msg="TLS connection established" protocol=TLS1.3 cipher=TLS_AES_128_GCM_SHA256

Bad Log Example (indicates old clients being rejected):

1
time="2024-01-15T10:05:00Z" level=warning msg="TLS handshake error" error="tls: protocol version not supported"

Implementation with Nginx Ingress

Note: If you’re using Traefik instead, see Implementation with Traefik

Understanding Nginx Components

How Nginx Ingress Handles TLS

Unlike Traefik’s CRD-based approach, Nginx Ingress Controller uses:

  1. ConfigMap for global TLS settings (cluster-wide)
  2. Annotations for per-ingress overrides (optional)
  3. nginx.conf generated dynamically from ConfigMap values

Key Differences from Traefik:

AspectNginx IngressTraefik
ConfigurationConfigMapCustom Resource (TLSOption)
ScopeCluster-wideNamespace-scoped
SyntaxOpenSSL directivesTraefik DSL
UpdatesRequires reloadAutomatic

How Nginx Processes TLS

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Client    β”‚
β”‚  (Browser)  β”‚
β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜
       β”‚ HTTPS Request
       β”‚
       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Nginx Ingress Controller          β”‚
β”‚                                     β”‚
β”‚  1. Receives connection            β”‚
β”‚  2. Reads ConfigMap settings       β”‚
β”‚  3. Applies ssl_protocols          β”‚
β”‚  4. Applies ssl_ciphers            β”‚
β”‚  5. Negotiates TLS handshake       β”‚
β”‚  6. Applies per-ingress overrides  β”‚
β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
       β”‚ HTTP (unencrypted)
       β”‚
       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Your Service   β”‚
β”‚   (Backend)     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Configuration Hierarchy:

  1. Global ConfigMap β†’ Applied to all ingresses by default
  2. Ingress Annotations β†’ Override global settings for specific ingresses (optional)
  3. Nginx conf snippets β†’ Advanced customization (not recommended)

Implementation Steps

Step 1: Configure Global TLS Settings

The primary method for configuring TLS in Nginx Ingress is through the controller’s ConfigMap.

Understanding Nginx TLS Directives

Nginx uses OpenSSL directives for TLS configuration:

DirectivePurposeExample
ssl-protocolsAllowed TLS versionsTLSv1.2 TLSv1.3
ssl-ciphersCipher suite listECDHE-ECDSA-AES128-GCM-SHA256:...
ssl-prefer-server-ciphersUse server cipher order"true"
ssl-ecdh-curveElliptic curve preferencessecp521r1:secp384r1

Option A: Update Existing ConfigMap

First, identify your Nginx Ingress ConfigMap:

1
2
3
4
5
6
7
8
# Find the ConfigMap (common names)
kubectl get configmap -n ingress-nginx
kubectl get configmap -n kube-system | grep nginx

# Common ConfigMap names:
# - nginx-configuration
# - nginx-ingress-controller
# - ingress-nginx-controller

Create or update the ConfigMap with secure TLS settings:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-configuration  # Use your actual ConfigMap name
  namespace: ingress-nginx    # Use your actual namespace
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
data:
  # TLS Protocol versions - Only allow TLS 1.2 and 1.3
  ssl-protocols: "TLSv1.2 TLSv1.3"

  # Cipher suites - AEAD only (GCM and ChaCha20-Poly1305)
  # OpenSSL format (space or colon separated)
  ssl-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305"

  # Prefer server cipher order (not client's)
  ssl-prefer-server-ciphers: "true"

  # Elliptic curves for key exchange (strongest first)
  ssl-ecdh-curve: "secp521r1:secp384r1"

  # Optional: Enable TLS session caching for performance
  ssl-session-cache: "shared:SSL:10m"
  ssl-session-timeout: "10m"

  # Optional: Enable HSTS (HTTP Strict Transport Security)
  hsts: "true"
  hsts-max-age: "31536000"
  hsts-include-subdomains: "true"

Apply the ConfigMap:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Method 1: Apply from file
kubectl apply -f nginx-tls-configmap.yaml

# Method 2: Patch existing ConfigMap
kubectl patch configmap nginx-configuration -n ingress-nginx \
  --type merge \
  -p '{"data":{"ssl-protocols":"TLSv1.2 TLSv1.3","ssl-ciphers":"ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305","ssl-prefer-server-ciphers":"true","ssl-ecdh-curve":"secp521r1:secp384r1"}}'

# Verify the ConfigMap was updated
kubectl get configmap nginx-configuration -n ingress-nginx -o yaml

Cipher Suite Format Conversion

Nginx uses OpenSSL cipher names, which differ slightly from Traefik:

Traefik FormatNginx/OpenSSL Format
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256ECDHE-ECDSA-AES128-GCM-SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256ECDHE-RSA-AES128-GCM-SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384ECDHE-ECDSA-AES256-GCM-SHA384
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384ECDHE-RSA-AES256-GCM-SHA384
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305ECDHE-ECDSA-CHACHA20-POLY1305
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305ECDHE-RSA-CHACHA20-POLY1305

Option B: Using Helm Values (If Installed via Helm)

If you installed Nginx Ingress using Helm, update your values.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
controller:
  config:
    ssl-protocols: "TLSv1.2 TLSv1.3"
    ssl-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305"
    ssl-prefer-server-ciphers: "true"
    ssl-ecdh-curve: "secp521r1:secp384r1"
    ssl-session-cache: "shared:SSL:10m"
    ssl-session-timeout: "10m"
    hsts: "true"
    hsts-max-age: "31536000"
    hsts-include-subdomains: "true"

Apply with Helm:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Upgrade with new values
helm upgrade nginx-ingress ingress-nginx/ingress-nginx \
  -n ingress-nginx \
  -f values.yaml

# Or set values directly
helm upgrade nginx-ingress ingress-nginx/ingress-nginx \
  -n ingress-nginx \
  --set controller.config.ssl-protocols="TLSv1.2 TLSv1.3" \
  --set controller.config.ssl-ciphers="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:..."

Option C: Using Kustomize

Create a Kustomize patch for the ConfigMap:

File: nginx-tls-patch.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
data:
  ssl-protocols: "TLSv1.2 TLSv1.3"
  ssl-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305"
  ssl-prefer-server-ciphers: "true"
  ssl-ecdh-curve: "secp521r1:secp384r1"

File: kustomization.yaml

1
2
3
4
5
6
7
8
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - nginx-tls-patch.yaml

patchesStrategicMerge:
  - nginx-tls-patch.yaml

Apply:

1
kubectl apply -k .

Step 2: Reload Nginx Ingress Controller

Important: Unlike Traefik, Nginx requires the controller pod to reload configuration after ConfigMap changes.

Option A: Automatic Reload (Preferred)

The Nginx Ingress Controller watches ConfigMap changes and auto-reloads. Wait 30-60 seconds:

1
2
# Watch for reload in logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller -f | grep -i reload

Expected output:

1
2
I0115 10:05:30 backend_ssl.go:42] "Reloading TLS configuration"
I0115 10:05:30 controller.go:168] "Configuration changes detected, backend reload required"

Option B: Manual Reload (If Needed)

Force a reload by sending a signal to the controller:

1
2
3
4
5
6
7
8
# Find the controller pod
NGINX_POD=$(kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller -o jsonpath='{.items[0].metadata.name}')

# Send reload signal
kubectl exec -n ingress-nginx $NGINX_POD -- /usr/bin/nginx -s reload

# Or restart the pod (more disruptive)
kubectl rollout restart deployment nginx-ingress-controller -n ingress-nginx

Step 3: (Optional) Per-Ingress TLS Overrides

If you need different TLS settings for specific ingresses, use annotations:

Example: Stricter TLS for API Ingress

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: production
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    # Override global TLS settings for this ingress only
    nginx.ingress.kubernetes.io/ssl-protocols: "TLSv1.3"  # TLS 1.3 only
    nginx.ingress.kubernetes.io/ssl-ciphers: "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256"
spec:
  ingressClassName: nginx
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 80
  tls:
    - hosts:
        - api.example.com
      secretName: api-tls-cert

Example: Standard Web Application

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  namespace: production
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    # No TLS overrides - uses global ConfigMap settings
spec:
  ingressClassName: nginx
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-service
                port:
                  number: 80
  tls:
    - hosts:
        - myapp.example.com
      secretName: web-tls-cert

Available Nginx TLS Annotations:

AnnotationPurposeExample
nginx.ingress.kubernetes.io/ssl-protocolsOverride TLS versions"TLSv1.3"
nginx.ingress.kubernetes.io/ssl-ciphersOverride cipher suites"ECDHE-RSA-..."
nginx.ingress.kubernetes.io/ssl-redirectForce HTTPS redirect"true"
nginx.ingress.kubernetes.io/force-ssl-redirectForce HTTPS even with X-Forwarded-Proto"true"
nginx.ingress.kubernetes.io/backend-protocolBackend protocol"HTTPS"

Verification (Nginx)

1. Verify ConfigMap Settings

1
2
3
4
5
# Check ConfigMap values
kubectl get configmap nginx-configuration -n ingress-nginx -o yaml

# Extract just the TLS settings
kubectl get configmap nginx-configuration -n ingress-nginx -o jsonpath='{.data}' | jq -r 'to_entries[] | select(.key | startswith("ssl")) | "\(.key): \(.value)"'

Expected output:

1
2
3
4
5
6
ssl-ciphers: ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:...
ssl-ecdh-curve: secp521r1:secp384r1
ssl-prefer-server-ciphers: true
ssl-protocols: TLSv1.2 TLSv1.3
ssl-session-cache: shared:SSL:10m
ssl-session-timeout: 10m

2. Verify Nginx Configuration

Check that the ConfigMap settings were applied to nginx.conf:

1
2
3
4
5
# Get the nginx pod name
NGINX_POD=$(kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller -o jsonpath='{.items[0].metadata.name}')

# View the generated nginx.conf
kubectl exec -n ingress-nginx $NGINX_POD -- cat /etc/nginx/nginx.conf | grep -A 5 "ssl_"

Expected output:

1
2
3
4
5
6
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305';
ssl_prefer_server_ciphers on;
ssl_ecdh_curve secp521r1:secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

3. Test Nginx Configuration Syntax

Verify there are no syntax errors:

1
2
# Test nginx configuration
kubectl exec -n ingress-nginx $NGINX_POD -- nginx -t

Expected output:

1
2
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

4. Monitor Nginx Logs

1
2
3
4
5
6
7
8
# Watch controller logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller -f

# Check for TLS-related errors
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller | grep -i "ssl\|tls"

# Check for handshake failures
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller | grep -i "handshake"

Good log example:

1
2024/01/15 10:05:00 [info] TLS connection using TLSv1.3 / TLS_AES_128_GCM_SHA256

Bad log example (indicates old clients being rejected):

1
2024/01/15 10:05:00 [error] SSL_do_handshake() failed (SSL: error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher)

5. Verify Ingress Resources

1
2
3
4
5
6
7
8
# List all ingresses
kubectl get ingress -A

# Check specific ingress
kubectl describe ingress myapp-ingress -n production

# Check if TLS is properly configured
kubectl get ingress myapp-ingress -n production -o jsonpath='{.spec.tls}'

Troubleshooting (Nginx)

Issue 1: ConfigMap Changes Not Applied

Symptom: TLS settings not taking effect after updating ConfigMap

Diagnosis:

1
2
3
4
5
# Check if ConfigMap was updated
kubectl get configmap nginx-configuration -n ingress-nginx -o yaml

# Check controller logs for reload
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller --tail=50 | grep reload

Solutions:

  1. Wait for auto-reload (30-60 seconds):
1
2
# Watch for reload signal
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller -f | grep -i "reload\|backend"
  1. Force manual reload:
1
2
NGINX_POD=$(kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller -o jsonpath='{.items[0].metadata.name}')
kubectl exec -n ingress-nginx $NGINX_POD -- nginx -s reload
  1. Restart controller (last resort):
1
kubectl rollout restart deployment nginx-ingress-controller -n ingress-nginx

Issue 2: Syntax Error in Cipher List

Symptom: Nginx configuration test fails

Diagnosis:

1
kubectl exec -n ingress-nginx $NGINX_POD -- nginx -t

Error example:

1
2
nginx: [emerg] SSL_CTX_set_cipher_list("INVALID_CIPHER") failed
nginx: configuration file /etc/nginx/nginx.conf test failed

Solutions:

  1. Check cipher syntax:
1
2
# Valid OpenSSL cipher names
openssl ciphers -v 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256'
  1. Use colon-separated format (not spaces):
1
2
3
4
5
# βœ… Correct
ssl-ciphers: "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384"

# ❌ Wrong
ssl-ciphers: "ECDHE-RSA-AES128-GCM-SHA256 ECDHE-RSA-AES256-GCM-SHA384"
  1. Verify cipher names:
1
2
# Test if ciphers are valid
openssl ciphers -v 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305'

Issue 3: ConfigMap Name Mismatch

Symptom: Changes to ConfigMap don’t affect nginx

Diagnosis:

1
2
# Check what ConfigMap the controller is using
kubectl describe deployment nginx-ingress-controller -n ingress-nginx | grep -i configmap

Solutions:

Find the correct ConfigMap name:

1
2
3
4
5
# Common names to check
kubectl get configmap -n ingress-nginx | grep -E "nginx|ingress"

# Check controller args for ConfigMap reference
kubectl get deployment nginx-ingress-controller -n ingress-nginx -o yaml | grep -A 5 "configmap"

Update the correct ConfigMap or update the controller to reference your ConfigMap.

Issue 4: TLS Still Accepts Old Protocols

Symptom: TLS 1.0/1.1 still working after configuration

Diagnosis:

1
2
3
4
5
# Test from outside the cluster
openssl s_client -connect myapp.example.com:443 -tls1_1

# Check actual nginx config
kubectl exec -n ingress-nginx $NGINX_POD -- cat /etc/nginx/nginx.conf | grep ssl_protocols

Solutions:

  1. Ensure ConfigMap data is correctly formatted:
1
2
data:
  ssl-protocols: "TLSv1.2 TLSv1.3"  # Must be quoted
  1. Check for per-ingress overrides:
1
2
# Look for annotations that might override global settings
kubectl get ingress -A -o yaml | grep "ssl-protocols"
  1. Verify reload happened:
1
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller | tail -20

Issue 5: Certificate Errors

Symptom: ERR_SSL_VERSION_OR_CIPHER_MISMATCH or certificate warnings

Diagnosis:

1
2
3
4
5
# Check certificate
kubectl get secret myapp-tls -n production -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout

# Check if secret exists and is referenced
kubectl get ingress myapp-ingress -n production -o yaml | grep -A 5 tls:

Solutions:

  1. Verify TLS secret exists:
1
kubectl get secret myapp-tls -n production
  1. Check cert-manager status (if using):
1
2
kubectl get certificate -n production
kubectl describe certificate myapp-tls -n production
  1. Test certificate validity:
1
openssl s_client -connect myapp.example.com:443 -showcerts

Validation Methods

1. OpenSSL Command-Line Testing

Test TLS 1.0 (Should Fail)

1
openssl s_client -connect myapp.example.com:443 -tls1

Expected Result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Connecting to 203.0.113.1
CONNECTED(00000003)
140736242796096:error:1400442E:SSL routines:CONNECT_CR_SRVR_HELLO:tlsv1 alert protocol version:ssl/statem/statem_clnt.c:1269:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 107 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Start Time: 1673778000
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

βœ… PASS: Connection rejected (protocol version error)

Test TLS 1.1 (Should Fail)

1
openssl s_client -connect myapp.example.com:443 -tls1_1

Expected: Similar error as TLS 1.0

Test TLS 1.2 (Should Succeed)

1
openssl s_client -connect myapp.example.com:443 -tls1_2

Expected Result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Connecting to 203.0.113.1
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
verify return:1
depth=0 CN = myapp.example.com
verify return:1
---
Certificate chain
 0 s:CN = myapp.example.com
   i:C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFmDCCBICgAwIBAgIQBrCWwSZLqqQZV...
-----END CERTIFICATE-----
subject=CN = myapp.example.com
issuer=C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3286 bytes and written 395 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 2F3A4B5C6D7E8F...
    Session-ID-ctx:
    Master-Key: 9E8D7C6B5A4F3E...
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - ab cd ef 12 34 56 78 90-...

    Start Time: 1673778100
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---

βœ… PASS:

  • Protocol: TLSv1.2
  • Cipher: ECDHE-RSA-AES128-GCM-SHA256 (approved)
  • Verification: OK

Test TLS 1.3 (Should Succeed)

1
openssl s_client -connect myapp.example.com:443 -tls1_3

Expected: Similar success, with Protocol: TLSv1.3

Test Specific Cipher Suite

1
2
3
4
5
6
7
8
9
# Test approved cipher
openssl s_client -connect myapp.example.com:443 \
  -tls1_2 -cipher 'ECDHE-RSA-AES128-GCM-SHA256'
# Should succeed

# Test weak cipher (CBC mode)
openssl s_client -connect myapp.example.com:443 \
  -tls1_2 -cipher 'AES128-SHA'
# Should fail with "no ciphers available"

Comprehensive OpenSSL Test Script

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
#!/bin/bash
# tls-test.sh

DOMAIN="$1"

if [ -z "$DOMAIN" ]; then
  echo "Usage: $0 <domain>"
  exit 1
fi

echo "Testing TLS configuration for $DOMAIN"
echo "======================================="

# Test TLS versions
echo ""
echo "1. Testing TLS Versions"
echo "-----------------------"

for version in tls1 tls1_1 tls1_2 tls1_3; do
  echo -n "Testing $version: "
  result=$(echo "Q" | timeout 5 openssl s_client -connect $DOMAIN:443 -$version 2>&1)

  if echo "$result" | grep -q "Cipher is (NONE)"; then
    echo "❌ REJECTED (Good!)"
  elif echo "$result" | grep -q "Cipher is"; then
    cipher=$(echo "$result" | grep "Cipher is" | head -1 | awk '{print $3}')
    echo "βœ… ACCEPTED - Cipher: $cipher"
  else
    echo "⚠️  ERROR - Connection failed"
  fi
done

# Test weak ciphers
echo ""
echo "2. Testing Weak Ciphers (should be rejected)"
echo "---------------------------------------------"

WEAK_CIPHERS=(
  "DES-CBC3-SHA"         # 3DES
  "AES128-SHA"           # CBC mode
  "AES256-SHA"           # CBC mode
  "RC4-SHA"              # RC4
  "NULL-SHA256"          # NULL encryption
)

for cipher in "${WEAK_CIPHERS[@]}"; do
  echo -n "Testing $cipher: "
  result=$(echo "Q" | timeout 5 openssl s_client -connect $DOMAIN:443 -cipher $cipher 2>&1)

  if echo "$result" | grep -q "Cipher is (NONE)"; then
    echo "βœ… REJECTED"
  else
    echo "❌ ACCEPTED (Bad!)"
  fi
done

# Test strong ciphers
echo ""
echo "3. Testing Strong Ciphers (should be accepted)"
echo "-----------------------------------------------"

STRONG_CIPHERS=(
  "ECDHE-RSA-AES128-GCM-SHA256"
  "ECDHE-RSA-AES256-GCM-SHA384"
  "ECDHE-ECDSA-AES128-GCM-SHA256"
)

for cipher in "${STRONG_CIPHERS[@]}"; do
  echo -n "Testing $cipher: "
  result=$(echo "Q" | timeout 5 openssl s_client -connect $DOMAIN:443 -cipher $cipher 2>&1)

  if echo "$result" | grep -q "Cipher is" && ! echo "$result" | grep -q "Cipher is (NONE)"; then
    echo "βœ… ACCEPTED"
  else
    echo "⚠️  REJECTED or ERROR"
  fi
done

echo ""
echo "Testing complete!"

Usage:

1
2
chmod +x tls-test.sh
./tls-test.sh myapp.example.com

2. nmap Testing

1
2
3
4
5
6
# Install nmap
# Ubuntu/Debian: apt install nmap
# macOS: brew install nmap

# Scan TLS protocols and ciphers
nmap --script ssl-enum-ciphers -p 443 myapp.example.com

Expected Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Starting Nmap 7.93 ( https://nmap.org )
Nmap scan report for myapp.example.com (203.0.113.1)
Host is up (0.012s latency).

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers:
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A
|       TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
|     compressors:
|       NULL
|     cipher preference: server
|     warnings:
|       Forward Secrecy not supported by all ciphers
|   TLSv1.3:
|     ciphers:
|       TLS_AKE_WITH_AES_128_GCM_SHA256 (secp256r1) - A
|       TLS_AKE_WITH_AES_256_GCM_SHA384 (secp384r1) - A
|       TLS_AKE_WITH_CHACHA20_POLY1305_SHA256 (secp256r1) - A
|     cipher preference: server
|_  least strength: A

Nmap done: 1 IP address (1 host up) scanned in 2.84 seconds

βœ… PASS Criteria:

  • Only TLSv1.2 and TLSv1.3 present
  • All ciphers rated “A”
  • No TLSv1.0 or TLSv1.1
  • No weak ciphers (rated below “A”)

3. testssl.sh (Comprehensive)

testssl.sh is a command-line tool that checks TLS/SSL encryption.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Install testssl.sh
git clone --depth 1 https://github.com/drwetter/testssl.sh.git
cd testssl.sh

# Run full test
./testssl.sh --full myapp.example.com

# Quick test (protocols and ciphers only)
./testssl.sh --protocols --ciphers myapp.example.com

# Test specific aspects
./testssl.sh --protocols myapp.example.com          # Protocol versions
./testssl.sh --standard myapp.example.com            # Standard cipher suites
./testssl.sh --server-preference myapp.example.com   # Cipher order
./testssl.sh --pfs myapp.example.com                 # Perfect Forward Secrecy

Expected Output (partial):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
###########################################################
    testssl.sh       3.0.8

      This program is free software. Distribution and
             modification under GPLv2 permitted.

    Please file bugs @ https://testssl.sh/bugs/
###########################################################

 Start 2024-01-15 10:15:00        -->> 203.0.113.1:443 (myapp.example.com) <<--

 rDNS (203.0.113.1):     myapp.example.com.

 Service detected:       HTTP

 Testing protocols via sockets

 SSLv2      not offered (OK)
 SSLv3      not offered (OK)
 TLS 1      not offered (OK)
 TLS 1.1    not offered (OK)
 TLS 1.2    offered (OK)
 TLS 1.3    offered (OK): final

 Testing cipher categories

 NULL ciphers (no encryption)                        not offered (OK)
 Anonymous NULL Ciphers (no authentication)          not offered (OK)
 Export ciphers (w/o ADH+NULL)                       not offered (OK)
 LOW: 64 Bit + DES, RC[2,4], MD5 (w/o export)       not offered (OK)
 Triple DES Ciphers / IDEA                           not offered (OK)
 Obsolete CBC ciphers (AES, ARIA etc.)               not offered (OK)
 Strong encryption (AEAD ciphers) with no FS         not offered (OK)
 Forward Secrecy strong encryption (AEAD ciphers)    offered (OK)

 Testing server preferences

 Has server cipher order?                            yes (OK)
 Negotiated protocol                                 TLSv1.3
 Negotiated cipher                                   TLS_AES_128_GCM_SHA256

 Testing ciphers per protocol via OpenSSL

 Hexcode  Cipher Suite Name (OpenSSL)       KeyExch.   Encryption  Bits
---------------------------------------------------------------------
 TLSv1.2 (server order)
 xc02f   ECDHE-RSA-AES128-GCM-SHA256       ECDH 256   AESGCM      128
 xc030   ECDHE-RSA-AES256-GCM-SHA384       ECDH 256   AESGCM      256
 xcca8   ECDHE-RSA-CHACHA20-POLY1305       ECDH 256   ChaCha20    256

 TLSv1.3 (server order)
 x1301   TLS_AES_128_GCM_SHA256            ECDH 253   AESGCM      128
 x1302   TLS_AES_256_GCM_SHA384            ECDH 253   AESGCM      256
 x1303   TLS_CHACHA20_POLY1305_SHA256      ECDH 253   ChaCha20    256

 Rating (experimental)

 Rating specs (not complete)  SSL Labs's 'SSL Server Rating Guide' (version 2009q)
 Specification documentation  https://github.com/ssllabs/research/wiki/SSL-Server-Rating-Guide
 Protocol Support (weighted)  100 (30)
 Key Exchange     (weighted)  100 (30)
 Cipher Strength  (weighted)  100 (40)
 Final Score                  100
 Overall Grade                A+

 Done 2024-01-15 10:17:23 [ 143s] -->> 203.0.113.1:443 (myapp.example.com) <<--

βœ… PASS Criteria:

  • SSLv2, SSLv3, TLS 1.0, TLS 1.1 β†’ “not offered (OK)”
  • TLS 1.2, TLS 1.3 β†’ “offered (OK)”
  • Only AEAD ciphers listed
  • Forward Secrecy β†’ “offered (OK)”
  • Overall Grade β†’ A or A+

4. curl Testing

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# Test with curl (should work)
curl -v https://myapp.example.com

# Force TLS 1.0 (should fail)
curl --tlsv1.0 --tls-max 1.0 https://myapp.example.com
# Expected: curl: (35) error:1400442E:SSL routines:CONNECT_CR_SRVR_HELLO:tlsv1 alert protocol version

# Force TLS 1.1 (should fail)
curl --tlsv1.1 --tls-max 1.1 https://myapp.example.com

# Force TLS 1.2 (should work)
curl --tlsv1.2 https://myapp.example.com

# Force TLS 1.3 (should work)
curl --tlsv1.3 https://myapp.example.com

# Verbose output shows negotiated cipher
curl -v https://myapp.example.com 2>&1 | grep -E "(SSL|TLS|cipher)"

Expected Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256

5. SSL Labs Online Testing

SSL Labs provides a comprehensive online TLS testing service.

URL: https://www.ssllabs.com/ssltest/

Steps:

  1. Go to https://www.ssllabs.com/ssltest/analyze.html
  2. Enter your domain: myapp.example.com
  3. Check “Do not show the results on the boards”
  4. Click “Submit”
  5. Wait 2-5 minutes for scan to complete

Expected Results:

Overall Rating: A or A+

Protocol Support:

  • βœ… TLS 1.3: Yes
  • βœ… TLS 1.2: Yes
  • ❌ TLS 1.1: No
  • ❌ TLS 1.0: No

Cipher Suites:

1
2
3
4
5
6
7
8
9
TLS 1.3 (suites in server-preferred order)
  TLS_AES_128_GCM_SHA256 (0x1301)   ECDH x25519 (eq. 3072 bits RSA)   FS    128
  TLS_AES_256_GCM_SHA384 (0x1302)   ECDH x25519 (eq. 3072 bits RSA)   FS    256
  TLS_CHACHA20_POLY1305_SHA256 (0x1303) ECDH x25519 (eq. 3072 bits RSA) FS  256

TLS 1.2 (suites in server-preferred order)
  TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)   ECDH secp256r1   FS    128
  TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)   ECDH secp256r1   FS    256
  TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8) ECDH secp256r1 FS  256

βœ… All ciphers should show “FS” (Forward Secrecy)

Protocol Details:

  • Secure Renegotiation: Supported
  • BEAST attack: Not applicable (TLS 1.0/1.1 disabled)
  • POODLE (TLS): No
  • Downgrade attack prevention: Yes
  • Forward Secrecy: Yes (with most browsers)

Certificate:

  • Trusted: Yes
  • Chain issues: None

6. Browser DevTools Testing

Modern browsers show TLS information in DevTools.

Chrome/Edge:

  1. Open DevTools (F12)
  2. Go to Security tab
  3. Click on main origin
  4. Check “Connection” section

Expected:

1
2
3
Connection: TLS 1.3
Cipher suite: TLS_AES_128_GCM_SHA256
Key exchange: X25519

Firefox:

  1. Click lock icon in address bar
  2. Click “Connection secure”
  3. Click “More information”
  4. Click “Technical Details”

Expected:

1
Connection Encrypted (TLS 1.3, TLS_AES_128_GCM_SHA256, 128 bit keys)

7. Automated Monitoring Script

Create a script to continuously monitor TLS configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
#!/bin/bash
# monitor-tls.sh

DOMAIN="$1"
SLACK_WEBHOOK="${2:-}"  # Optional Slack webhook for alerts

check_tls() {
  local domain=$1
  local issues=0

  echo "Checking TLS configuration for $domain..."

  # Check TLS 1.0 (should be rejected)
  if echo "Q" | timeout 5 openssl s_client -connect $domain:443 -tls1 2>&1 | grep -q "Cipher is" | grep -v "(NONE)"; then
    echo "❌ ALERT: TLS 1.0 is enabled!"
    ((issues++))
  fi

  # Check TLS 1.1 (should be rejected)
  if echo "Q" | timeout 5 openssl s_client -connect $domain:443 -tls1_1 2>&1 | grep -q "Cipher is" | grep -v "(NONE)"; then
    echo "❌ ALERT: TLS 1.1 is enabled!"
    ((issues++))
  fi

  # Check TLS 1.2 (should be enabled)
  if ! echo "Q" | timeout 5 openssl s_client -connect $domain:443 -tls1_2 2>&1 | grep -q "Cipher is" | grep -v "(NONE)"; then
    echo "❌ ALERT: TLS 1.2 is NOT enabled!"
    ((issues++))
  fi

  # Check for weak cipher (CBC mode)
  if echo "Q" | timeout 5 openssl s_client -connect $domain:443 -cipher 'AES128-SHA' 2>&1 | grep -q "Cipher is" | grep -v "(NONE)"; then
    echo "❌ ALERT: Weak CBC cipher is enabled!"
    ((issues++))
  fi

  if [ $issues -eq 0 ]; then
    echo "βœ… TLS configuration is secure"
    return 0
  else
    echo "⚠️  Found $issues issue(s)"

    # Send Slack alert if webhook provided
    if [ -n "$SLACK_WEBHOOK" ]; then
      curl -X POST -H 'Content-type: application/json' \
        --data "{\"text\":\"🚨 TLS Configuration Issue on $domain: $issues problems detected\"}" \
        "$SLACK_WEBHOOK"
    fi

    return 1
  fi
}

check_tls "$DOMAIN"

Run as cron job:

1
2
# Add to crontab -e
0 */6 * * * /path/to/monitor-tls.sh myapp.example.com https://hooks.slack.com/services/YOUR/WEBHOOK/URL

Troubleshooting

Issue 1: Ingress Not Using TLSOption

Symptom: Old protocols still accepted after applying TLSOption

Diagnosis:

1
2
3
4
5
# Check if annotation is present
kubectl get ingress myapp-ingress -n production -o jsonpath='{.metadata.annotations}'

# Check Traefik logs
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik | grep tlsoption

Solutions:

  1. Verify annotation format:
1
2
# Must be exactly this format
traefik.ingress.kubernetes.io/router.tls.options: <namespace>-<name>@kubernetescrd
  1. Check namespace match:
1
2
3
# TLSOption namespace must match ingress namespace
kubectl get tlsoption -n production
kubectl get ingress -n production
  1. Restart Traefik:
1
kubectl rollout restart deployment traefik -n kube-system

Issue 2: TLSOption Not Found

Symptom: Traefik logs show “TLSOption not found”

Diagnosis:

1
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik | grep "not found"

Solutions:

  1. Verify TLSOption exists:
1
kubectl get tlsoption -A
  1. Check CRD is installed:
1
kubectl get crd tlsoptions.traefik.io
  1. Recreate TLSOption:
1
2
kubectl delete tlsoption secure-tls -n production
kubectl apply -f tlsoption-prod.yaml

Issue 3: Certificate Errors After Applying TLSOption

Symptom: ERR_SSL_VERSION_OR_CIPHER_MISMATCH in browser

Diagnosis:

1
2
3
4
5
# Check certificate
openssl s_client -connect myapp.example.com:443 -showcerts

# Check cert-manager logs
kubectl logs -n cert-manager -l app=cert-manager

Solutions:

  1. Verify certificate is valid:
1
2
kubectl get certificate -n production
kubectl describe certificate myapp-tls -n production
  1. Check certificate issuer:
1
2
kubectl get clusterissuer
kubectl describe clusterissuer letsencrypt-prod
  1. Force certificate renewal:
1
2
kubectl delete secret myapp-tls -n production
kubectl annotate certificate myapp-tls -n production cert-manager.io/issue-temporary-certificate="true"

Issue 4: Some Clients Can’t Connect

Symptom: Old browsers/clients fail to connect

Diagnosis:

1
2
# Check client's TLS capabilities
openssl s_client -connect myapp.example.com:443 -debug

Solutions:

  1. Check if client supports TLS 1.2+:

    • Windows XP: No TLS 1.2 support
    • Android 4.4 and older: No TLS 1.2 by default
    • Java 7 and older: No TLS 1.2 by default
  2. Temporarily enable TLS 1.1 (not recommended):

1
2
spec:
  minVersion: VersionTLS11  # Only if absolutely necessary
  1. Document minimum client requirements:
    • Windows 7+, macOS 10.13+, iOS 11+, Android 5.0+
    • Modern browsers (Chrome 30+, Firefox 27+, Safari 7+, Edge all versions)

Issue 5: Performance Degradation

Symptom: Slower response times after enabling strict TLS

Diagnosis:

1
2
3
4
5
# Measure TLS handshake time
time openssl s_client -connect myapp.example.com:443 < /dev/null

# Check Traefik resource usage
kubectl top pod -n kube-system -l app.kubernetes.io/name=traefik

Solutions:

  1. Enable TLS session resumption (already enabled by default):
1
2
3
# In Traefik values
additionalArguments:
  - "--experimental.http3=true"  # HTTP/3 for faster handshakes
  1. Use ECDSA certificates (faster than RSA):
1
2
3
4
spec:
  privateKey:
    algorithm: ECDSA
    size: 256
  1. Increase Traefik resources:
1
2
3
4
5
6
7
resources:
  limits:
    cpu: 2000m
    memory: 2Gi
  requests:
    cpu: 500m
    memory: 512Mi

Issue 6: Kustomize Overlay Not Working

Symptom: TLSOption deployed to wrong namespace

Diagnosis:

1
2
3
4
5
# Preview what will be applied
kubectl kustomize overlays/production

# Check namespace in output
kubectl kustomize overlays/production | grep namespace

Solutions:

  1. Verify kustomization.yaml has namespace:
1
namespace: production  # Must be set
  1. Check directory structure:
1
2
tree tls-security/
# Should match documented structure
  1. Use explicit namespace:
1
kubectl apply -k overlays/production --namespace=production

Common Error Messages

ErrorCauseSolution
tls: protocol version not supportedClient using TLS 1.0/1.1Expected behavior, client needs upgrade
tls: no cipher suite supported by both client and serverClient doesn’t support AEAD ciphersClient needs modern TLS stack
TLSOption not found: production-secure-tls@kubernetescrdTLSOption missing or wrong nameCreate TLSOption, check annotation format
x509: certificate has expiredTLS certificate expiredRenew certificate, check cert-manager
remote error: tls: handshake failureSNI mismatch or invalid certCheck certificate SAN, verify hostname

Best Practices

1. Security Best Practices

Use Strong Cipher Suites Only

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# βœ… GOOD: Only AEAD ciphers
cipherSuites:
  - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
  - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305

# ❌ BAD: Includes CBC mode
cipherSuites:
  - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256  # Vulnerable to padding oracle

Enable Strict SNI

1
2
# Always enable SNI strictness
sniStrict: true

This prevents:

  • IP-based access to your service
  • SNI downgrade attacks
  • Certificate enumeration

Use Strong Curves

1
2
3
4
5
6
7
8
9
# βœ… GOOD: Strong curves only
curvePreferences:
  - CurveP521  # 521-bit
  - CurveP384  # 384-bit

# ❌ BAD: Weak curves included
curvePreferences:
  - CurveP256  # Only 256-bit, not ideal
  - CurveP224  # Too weak

2. Operational Best Practices

Version Control All Configurations

1
2
3
4
# Store in git
git add helm/tlsoption-*.yaml
git commit -m "Add TLS hardening configuration"
git push

Use Consistent Naming

1
2
3
4
5
6
# Follow this pattern across all namespaces
<environment>-<region>-secure-tls

# Examples:
prod-secure-tls
stage-secure-tls

Label All Resources

1
2
3
4
5
6
metadata:
  labels:
    security.policy: strict-tls
    managed-by: platform-team
    environment: production
    compliance: pci-dss

Document Exceptions

If you must support older protocols temporarily:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: legacy-support-tls
  annotations:
    security.exception: "temporary-support-for-legacy-client"
    security.expires: "2024-06-30"
    security.approved-by: "security-team"
spec:
  minVersion: VersionTLS11  # Exception: legacy client support

3. Deployment Best Practices

Always Test in Non-Production First

1
2
3
4
5
6
# Deployment order
1. Development
2. Staging
3. Production

# Never skip staging!

Use Canary Deployments

1
2
3
4
# Gradual rollout
10% β†’ 25% β†’ 50% β†’ 100%

# Monitor error rates at each step

Have a Rollback Plan

1
2
3
4
5
# Save old configuration
kubectl get ingress myapp-ingress -n production -o yaml > ingress-backup.yaml

# Quick rollback command
kubectl apply -f ingress-backup.yaml

4. Monitoring Best Practices

Monitor TLS Handshake Failures

1
2
3
4
# Alert on handshake failures
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik \
  | grep "tls: handshake failure" \
  | wc -l

Track Protocol Usage

1
2
3
4
5
# Log which protocols are being used
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik \
  | grep "TLS connection" \
  | awk '{print $NF}' \
  | sort | uniq -c

Set Up Alerts

1
2
3
4
5
6
7
8
9
# Prometheus alert example
- alert: WeakTLSProtocol
  expr: |
    count(traefik_tls_connections_total{protocol=~"TLS1[01]"}) > 0
  for: 5m
  labels:
    severity: critical
  annotations:
    summary: "Weak TLS protocol detected"

5. Compliance Best Practices

Regular Audits

1
2
# Schedule monthly TLS audits
0 0 1 * * ./tls-audit.sh

Document Configuration

Maintain a security configuration document:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# TLS Configuration Standard

## Policy
- Minimum TLS version: 1.2
- Allowed ciphers: AEAD only
- Certificate validity: Max 90 days
- Audits: Monthly

## Exceptions
None currently

## Last Review
2024-01-15 by Security Team

Keep Evidence

1
2
3
4
5
# Save test results for compliance
./testssl.sh myapp.example.com > tls-report-$(date +%Y%m%d).txt

# Store in compliance folder
aws s3 cp tls-report-*.txt s3://compliance-evidence/tls-reports/

Advanced Topics

1. Mutual TLS (mTLS)

For API-to-API communication, enable client certificate authentication:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: mtls-strict
  namespace: production
spec:
  minVersion: VersionTLS12
  cipherSuites:
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  clientAuth:
    secretNames:
      - client-ca-cert  # Secret containing CA certificate
    clientAuthType: RequireAndVerifyClientCert

Create CA secret:

1
2
3
kubectl create secret generic client-ca-cert \
  --from-file=tls.ca=/path/to/ca.crt \
  -n production

2. Per-Service TLS Configuration

Different services may need different TLS settings:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# Strict for APIs
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: api-strict-tls
  namespace: production
spec:
  minVersion: VersionTLS13  # TLS 1.3 only
  sniStrict: true

---
# Moderate for web apps
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: web-standard-tls
  namespace: production
spec:
  minVersion: VersionTLS12  # TLS 1.2+
  sniStrict: true

3. Integration with Service Mesh

If using Istio or Linkerd, coordinate TLS settings:

1
2
3
4
5
6
7
8
9
# Traefik handles edge TLS
# Service mesh handles internal TLS

# Disable TLS in backend services
apiVersion: v1
kind: Service
metadata:
  annotations:
    traefik.ingress.kubernetes.io/service.serversscheme: http  # Backend is HTTP

4. ACME Certificate Automation

Use cert-manager with TLSOption:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: myapp-tls
  namespace: production
spec:
  secretName: myapp-tls
  issuer: letsencrypt-prod
  dnsNames:
    - myapp.example.com
  privateKey:
    algorithm: ECDSA  # Faster than RSA
    size: 256
  duration: 2160h  # 90 days
  renewBefore: 720h  # Renew 30 days before expiry

5. Geographic TLS Configuration

Different regions may have different compliance requirements:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# EU: GDPR compliant
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: eu-compliant-tls
  namespace: prod-eu
spec:
  minVersion: VersionTLS12
  # ... standard config

---
# US: FIPS compliant
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: us-fips-tls
  namespace: prod-us
spec:
  minVersion: VersionTLS12
  cipherSuites:
    # Only FIPS-approved ciphers
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

6. CI/CD Integration

Automate TLS configuration deployment:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# .gitlab-ci.yml
deploy-tls-security:
  stage: deploy
  script:
    - kubectl apply -k overlays/$ENVIRONMENT
    - ./validate-tls.sh $DOMAIN
  only:
    changes:
      - tls-security/**/*
  environment:
    name: $ENVIRONMENT
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# GitHub Actions
name: Deploy TLS Configuration
on:
  push:
    paths:
      - 'tls-security/**'
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Deploy TLSOptions
        run: |
          kubectl apply -k overlays/production
      - name: Validate
        run: |
          ./tls-test.sh myapp.example.com

Rollback Procedures

Quick Rollback

If issues arise immediately after deployment:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# 1. Backup current config
kubectl get tlsoption -n production -o yaml > tlsoption-backup.yaml
kubectl get ingress -n production -o yaml > ingress-backup.yaml

# 2. Remove TLS annotation
kubectl annotate ingress myapp-ingress -n production \
  traefik.ingress.kubernetes.io/router.tls.options-

# 3. Delete TLSOption
kubectl delete tlsoption secure-tls -n production

# 4. Verify old behavior restored
openssl s_client -connect myapp.example.com:443 -tls1

Gradual Rollback

For controlled rollback:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Step 1: Switch 50% of traffic back to old config
# Create parallel ingress without TLSOption
kubectl apply -f ingress-legacy.yaml

# Step 2: Update TraefikService weights
kubectl patch traefikservice myapp-weighted -n production \
  --type=json -p='[
    {"op": "replace", "path": "/spec/weighted/services/0/weight", "value": 50},
    {"op": "replace", "path": "/spec/weighted/services/1/weight", "value": 50}
  ]'

# Step 3: Monitor for 1 hour
# Step 4: If stable, complete rollback
# Step 5: If issues resolved, roll forward again

Emergency Rollback Script

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/bin/bash
# emergency-rollback.sh

NAMESPACE="$1"

if [ -z "$NAMESPACE" ]; then
  echo "Usage: $0 <namespace>"
  exit 1
fi

echo "🚨 EMERGENCY ROLLBACK for $NAMESPACE"
echo "Removing TLS security configurations..."

# Remove annotations from all ingresses
for ingress in $(kubectl get ingress -n $NAMESPACE -o name); do
  echo "Rolling back $ingress..."
  kubectl annotate $ingress -n $NAMESPACE \
    traefik.ingress.kubernetes.io/router.tls.options- \
    --overwrite
done

# Delete TLSOptions
kubectl delete tlsoption --all -n $NAMESPACE

echo "βœ… Rollback complete"
echo "⚠️  WARNING: TLS security is now disabled"
echo "   Please investigate and reapply when ready"

Appendix

A. Cipher Suite Reference

Cipher Suite NameKey ExchangeEncryptionAEADForward SecrecyPerformanceSecurity
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256ECDHEAES-128-GCMYesYesFastHigh
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256ECDHEAES-128-GCMYesYesFastHigh
TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384ECDHEAES-256-GCMYesYesMediumVery High
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384ECDHEAES-256-GCMYesYesMediumVery High
TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305ECDHEChaCha20YesYesFast (mobile)High
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305ECDHEChaCha20YesYesFast (mobile)High

B. TLS Version Timeline

VersionReleasedDeprecatedStatusSecurity Issues
SSL 2.019952011❌ ForbiddenMultiple critical flaws
SSL 3.019962015❌ ForbiddenPOODLE attack
TLS 1.019992020❌ DeprecatedBEAST, downgrade attacks
TLS 1.120062020❌ DeprecatedLimited cipher suites
TLS 1.22008-βœ… CurrentSecure with AEAD ciphers
TLS 1.32018-βœ… RecommendedMost secure, fastest

C. Compliance Mapping

StandardRequirementThis Configuration
PCI DSS 3.2+TLS 1.2+βœ… Enforced
PCI DSS 3.2+Strong cryptographyβœ… AEAD only
NIST SP 800-52 Rev. 2Disable TLS 1.0/1.1βœ… Disabled
NIST SP 800-52 Rev. 2Use AEAD ciphersβœ… Only AEAD
HIPAAEncryption in transitβœ… TLS 1.2+
GDPR Art. 32State-of-the-art securityβœ… Modern ciphers
ISO 27001 A.13.1.1Network security controlsβœ… Strict TLS

D. Client Compatibility Matrix

ClientTLS 1.2TLS 1.3GCM CiphersChaCha20Compatible?
Chrome 30+βœ…βœ…βœ…βœ…βœ… Yes
Firefox 27+βœ…βœ…βœ…βœ…βœ… Yes
Safari 7+βœ…βœ…βœ…βœ…βœ… Yes
Edge (all)βœ…βœ…βœ…βœ…βœ… Yes
IE 11βœ…βŒβœ…βŒβœ… Yes
IE 10βœ…βŒβš οΈβŒβš οΈ Limited
Android 5.0+βœ…βœ…βœ…βœ…βœ… Yes
Android 4.4⚠️❌⚠️❌⚠️ Limited
iOS 11+βœ…βœ…βœ…βœ…βœ… Yes
iOS 9-10βœ…βŒβœ…βŒβœ… Yes
Windows 7 SP1+βœ…βŒβœ…βŒβœ… Yes
Windows XP❌❌❌❌❌ No
Java 8+βœ…βœ…βœ…βœ…βœ… Yes
Java 7⚠️❌⚠️❌⚠️ Limited
Python 3.6+βœ…βœ…βœ…βœ…βœ… Yes
Python 2.7⚠️❌⚠️❌⚠️ Limited
curl 7.34+βœ…βœ…βœ…βœ…βœ… Yes
wget 1.14+βœ…βœ…βœ…βœ…βœ… Yes

E. Useful Commands Reference

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
# Kubernetes Commands
kubectl get tlsoption -A
kubectl describe tlsoption <name> -n <namespace>
kubectl get ingress -A -o yaml
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik -f

# TLS Testing
openssl s_client -connect domain:443 -tls1_2
openssl s_client -connect domain:443 -showcerts
curl -v --tls-max 1.1 https://domain
nmap --script ssl-enum-ciphers -p 443 domain

# Certificate Management
kubectl get certificate -A
kubectl describe certificate <name> -n <namespace>
openssl x509 -in cert.pem -text -noout

# Debugging
kubectl get events -n <namespace> --sort-by='.lastTimestamp'
kubectl port-forward -n kube-system svc/traefik 9000:9000

F. Additional Resources

Ingress Controller Documentation:

General Kubernetes:

Security Standards:

Testing Tools:

Learning Resources:

G. Traefik vs Nginx: Detailed Comparison

This section provides a side-by-side comparison to help you understand the differences between implementing TLS hardening with Traefik versus Nginx Ingress Controller.

Configuration Approach

AspectTraefikNginx Ingress
Primary MethodCustom Resource Definition (TLSOption)ConfigMap
Configuration ScopeNamespace-scopedCluster-wide
Syntax StyleKubernetes-native YAMLOpenSSL directives
File LocationSeparate CRD per namespaceSingle ConfigMap
VersioningPer namespace versionsSingle global version

Implementation Complexity

TaskTraefikNginx Ingress
Initial SetupCreate TLSOption CRD per namespaceUpdate single ConfigMap
Multi-Team EnvironmentsEasy (namespace isolation)More complex (shared config)
Per-Service CustomizationCreate different TLSOptionsUse ingress annotations
Testing ChangesDeploy to test namespaceTest on non-prod cluster
RollbackDelete/revert specific TLSOptionRevert ConfigMap

Cipher Suite Syntax

1
2
3
4
5
6
7
# Traefik Format
cipherSuites:
  - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

# Nginx Format
ssl-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256"

Operational Characteristics

OperationTraefikNginx Ingress
Config UpdatesAutomatic detectionRequires reload (auto or manual)
Reload TimeImmediate30-60 seconds (automatic)
Downtime on UpdateZero downtimeZero downtime (graceful reload)
Monitoring ChangesWatch TLSOption resourcesWatch ConfigMap + nginx reloads
DebuggingCheck TLSOption statusCheck nginx.conf + logs

Example: Setting TLS 1.2 Minimum

Traefik:

1
2
3
4
5
6
7
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
  name: secure-tls
  namespace: production
spec:
  minVersion: VersionTLS12

Nginx:

1
2
3
4
5
6
7
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
data:
  ssl-protocols: "TLSv1.2 TLSv1.3"

When to Choose Which

Choose Traefik if:

  • You need namespace-level TLS policy isolation
  • Multiple teams manage their own ingresses
  • You prefer Kubernetes-native CRD approach
  • You want automatic updates without reloads
  • You’re already using Traefik features (middleware, etc.)

Choose Nginx if:

  • You prefer centralized TLS configuration
  • You’re familiar with nginx/OpenSSL syntax
  • You want simple cluster-wide defaults
  • Your team has nginx expertise
  • You need nginx-specific features (Lua, modules)

Migration Between Controllers

Traefik β†’ Nginx:

  1. Export TLSOption settings from each namespace
  2. Merge into single nginx ConfigMap
  3. Convert cipher syntax (TLS_ β†’ ECDHE-)
  4. Update ingress class annotations
  5. Test thoroughly before switching

Nginx β†’ Traefik:

  1. Extract ConfigMap ssl-* settings
  2. Create TLSOption CRD per namespace
  3. Convert cipher syntax (ECDHE- β†’ TLS_)
  4. Add TLS annotation to ingresses
  5. Test namespace by namespace

Conclusion

By following this guide, you have successfully hardened TLS security in your Kubernetes cluster using either Traefik or Nginx Ingress Controller. You have:

  1. βœ… Disabled insecure TLS 1.0 and TLS 1.1 protocols
  2. βœ… Enabled only modern AEAD cipher suites (GCM, ChaCha20-Poly1305)
  3. βœ… Configured strong elliptic curves (P-384, P-521)
  4. βœ… Enabled strict SNI validation (Traefik) or server cipher preference (Nginx)
  5. βœ… Applied configuration consistently across environments
  6. βœ… Validated configuration with multiple methods
  7. βœ… Established monitoring and alerting
  8. βœ… Documented configuration for compliance

Your Kubernetes cluster now enforces modern TLS security standards, protecting against known vulnerabilities and meeting compliance requirements for PCI DSS, HIPAA, GDPR, and other security frameworks.

Implementation Summary

If you used Traefik:

  • βœ… Created TLSOption CRDs in each namespace
  • βœ… Applied TLS annotations to ingress resources
  • βœ… Verified TLSOptions are active

If you used Nginx Ingress:

  • βœ… Updated the nginx ConfigMap with secure TLS settings
  • βœ… Verified nginx reloaded the configuration
  • βœ… Confirmed settings in nginx.conf

Next Steps:

  1. Schedule monthly security audits using SSL Labs or testssl.sh
  2. Monitor for deprecated protocol connection attempts in controller logs
  3. Keep cipher suite list updated with current recommendations
  4. Review client compatibility if adding new integrations
  5. Document any exceptions with security team approval
  6. Consider implementing mutual TLS (mTLS) for service-to-service communication

Questions or Issues?

  • Check the controller-specific Troubleshooting sections:
  • Review controller logs:
    • Traefik: kubectl logs -n kube-system -l app.kubernetes.io/name=traefik
    • Nginx: kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx-controller
  • Consult official documentation (see Additional Resources)
  • Test configuration changes in non-production first
  • Use the Validation Methods section to verify your configuration

Remember: Security is an ongoing process. Regularly review and update your TLS configuration as new vulnerabilities are discovered and security standards evolve.

comments powered by Disqus
Citizix Ltd
Built with Hugo
Theme Stack designed by Jimmy