AKS Private Cluster

Prerequisite

1. Resource group name

2. AKS Cluster name

3. Application gateway name

4. VM Size (Standard_D4v3, Standard_B2s, etc...)

5. AKS Subnet Id

6. Application Gateway subnet id

Get Subnet Ids

az network vnet subnet list --resource-group EX-TEST --vnet-name ex-test-ag-vnet

Install AKS Clusters

Creating Public AKS Cluster with Azure CNI

Below script will create a new AKS cluster with application gateway with predefined subnets for both AKS and Application Gateway.

Benefits of using this script for customers to deploy the cluster with predefined network architecture of their choice and gives freedom to select whatever the IP ranges they wish for.

az aks create --name ex-pri-stg `
              --resource-group EX-TEST `
              --load-balancer-sku standard `
              --node-count 1 `
              --vnet-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-app-vnet/subnets/ex-in-test1-app-subnet" `
              --docker-bridge-address 172.17.0.1/16 `
              --dns-name-prefix ex-pri-stg-dns `
              --dns-service-ip 10.2.0.10 `
              --service-cidr 10.2.0.0/24 `
              --network-plugin azure `
              --enable-managed-identity `
              -a ingress-appgw `
              --appgw-name ex-pri-ag-test-1 `
              --appgw-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-ag-vnet/subnets/ex-in-test1-ag-subnet" `
              --node-vm-size Standard_B2s `
              --generate-ssh-keys

Creating Private AKS Cluster with Azure CNI

Below script will create a new AKS cluster with application gateway with predefined subnets for both AKS and Application Gateway. It registers the AKS API Server with private DNS.

Prerequisite for private DNS

1. Create Private DNS Zone with privatelink.<region>.azmk8s.io

2. Create User Managed Identity

3. Assign managed identity as

   • Private DNS zone contributor in private dns zone

   • Network Contributor in vnet or specific subnet

4. Create jump server in same subnet or vnet (or ensure the jump has access to Private DNS and VNet)

Register EnablePrivateClusterFQDNSubdomain to use custom private DNS

-------------------------
     Enable Feature
-------------------------

az feature register --namespace "Microsoft.ContainerService" --name "EnablePrivateClusterFQDNSubdomain"

-------------------------
Check registration status
-------------------------

az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnablePrivateClusterFQDNSubdomain')].{Name:name,State:properties.state}"

-------------------------
  Refresh the provider
-------------------------
az provider register --namespace Microsoft.ContainerService

az aks create --name ex-pri-stg `
              --resource-group EX-TEST `
              --load-balancer-sku standard `
              --node-count 1 `
              --vnet-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-app-vnet/subnets/ex-in-test1-app-subnet" `
              --docker-bridge-address 172.17.0.1/16 `
              --dns-service-ip 10.2.0.10 `
              --service-cidr 10.2.0.0/24 `
              --network-plugin azure `
              --enable-managed-identity `
              --assign-identity "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-mi" `
              -a ingress-appgw `
              --appgw-name ex-pri-ag-test-1 `
              --appgw-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-ag-vnet/subnets/ex-in-test1-ag-subnet" `
              --node-vm-size Standard_B2s `
              --generate-ssh-keys `
              --enable-private-cluster `
              --private-dns-zone "/subscriptions/<subscriptionid>/resourceGroups/gaeaglobal/providers/Microsoft.Network/privateDnsZones/privatelink.centralindia.azmk8s.io"`
              --fqdn-subdomain ex-pri-stg

Creating Private AKS Cluster with Kubenet

az aks create --name ex-proj-01 \
              --resource-group EX-PROJECTS-US \
              --location westus2 \
              --load-balancer-sku standard \
              --enable-cluster-autoscaler \
              --min-count 2 \
              --max-count 4 \
              --kubernetes-version 1.21.2 \
              --vnet-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-PROJECTS-US/providers/Microsoft.Network/virtualNetworks/ex-proj-us-vnet/subnets/ex-proj-01" \
              --assign-identity="/subscriptions/<subscriptionid>/resourceGroups/EX-PROJECTS-US/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ex-proj-us-umi" \
              --docker-bridge-address 172.17.0.1/16 \
              --dns-service-ip 10.2.0.10 \
              --pod-cidr 10.244.0.0/24 \
              --service-cidr 10.2.0.0/24 \
              --network-plugin kubenet \
              --node-vm-size Standard_D2s_v3 \
              --generate-ssh-keys

Delete AKS Cluster

az aks delete -g EX-TEST -n ex-pri-stg