> For the complete documentation index, see [llms.txt](https://docs.exto360.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.exto360.com/installation/install-exto-on-kubernetes/tips-and-tricks/aks-private-cluster.md).

# AKS Private Cluster

## Prerequisite

1. Resource group name
2. AKS Cluster name
3. Application gateway name
4. VM Size (Standard\_D4v3, Standard\_B2s, etc...)
5. AKS Subnet Id
6. Application Gateway subnet id

#### Get Subnet Ids

```
az network vnet subnet list --resource-group EX-TEST --vnet-name ex-test-ag-vnet
```

## Install AKS Clusters

### Creating Public AKS Cluster with Azure CNI

Below script will create a new AKS cluster with application gateway with predefined subnets for both AKS and Application Gateway.

Benefits of using this script for customers to deploy the cluster with predefined network architecture of their choice and gives freedom to select whatever the IP ranges they wish for.

```
az aks create --name ex-pri-stg `
              --resource-group EX-TEST `
              --load-balancer-sku standard `
              --node-count 1 `
              --vnet-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-app-vnet/subnets/ex-in-test1-app-subnet" `
              --docker-bridge-address 172.17.0.1/16 `
              --dns-name-prefix ex-pri-stg-dns `
              --dns-service-ip 10.2.0.10 `
              --service-cidr 10.2.0.0/24 `
              --network-plugin azure `
              --enable-managed-identity `
              -a ingress-appgw `
              --appgw-name ex-pri-ag-test-1 `
              --appgw-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-ag-vnet/subnets/ex-in-test1-ag-subnet" `
              --node-vm-size Standard_B2s `
              --generate-ssh-keys
```

### Creating Private AKS Cluster with Azure CNI

Below script will create a new AKS cluster with application gateway with predefined subnets for both AKS and Application Gateway. It registers the AKS API Server with private DNS.

#### Prerequisite for private DNS

1. Create Private DNS Zone with privatelink.\<region>.azmk8s.io
2. Create User Managed Identity
3. Assign managed identity as
   * **Private DNS zone contributor** in private dns zone
   * **Network Contributor** in vnet or specific subnet
4. Create jump server in same subnet or vnet (or ensure the jump has access to Private DNS and VNet)

#### Register EnablePrivateClusterFQDNSubdomain to use custom private DNS

```
-------------------------
     Enable Feature
-------------------------

az feature register --namespace "Microsoft.ContainerService" --name "EnablePrivateClusterFQDNSubdomain"

-------------------------
Check registration status
-------------------------

az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/EnablePrivateClusterFQDNSubdomain')].{Name:name,State:properties.state}"

-------------------------
  Refresh the provider
-------------------------
az provider register --namespace Microsoft.ContainerService
```

```
az aks create --name ex-pri-stg `
              --resource-group EX-TEST `
              --load-balancer-sku standard `
              --node-count 1 `
              --vnet-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-app-vnet/subnets/ex-in-test1-app-subnet" `
              --docker-bridge-address 172.17.0.1/16 `
              --dns-service-ip 10.2.0.10 `
              --service-cidr 10.2.0.0/24 `
              --network-plugin azure `
              --enable-managed-identity `
              --assign-identity "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.ManagedIdentity/userAssignedIdentities/aks-mi" `
              -a ingress-appgw `
              --appgw-name ex-pri-ag-test-1 `
              --appgw-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-TEST/providers/Microsoft.Network/virtualNetworks/ex-in-test-ag-vnet/subnets/ex-in-test1-ag-subnet" `
              --node-vm-size Standard_B2s `
              --generate-ssh-keys `
              --enable-private-cluster `
              --private-dns-zone "/subscriptions/<subscriptionid>/resourceGroups/gaeaglobal/providers/Microsoft.Network/privateDnsZones/privatelink.centralindia.azmk8s.io"`
              --fqdn-subdomain ex-pri-stg
```

### Creating Private AKS Cluster with Kubenet

```bash
az aks create --name ex-proj-01 \
              --resource-group EX-PROJECTS-US \
              --location westus2 \
              --load-balancer-sku standard \
              --enable-cluster-autoscaler \
              --min-count 2 \
              --max-count 4 \
              --kubernetes-version 1.21.2 \
              --vnet-subnet-id "/subscriptions/<subscriptionid>/resourceGroups/EX-PROJECTS-US/providers/Microsoft.Network/virtualNetworks/ex-proj-us-vnet/subnets/ex-proj-01" \
              --assign-identity="/subscriptions/<subscriptionid>/resourceGroups/EX-PROJECTS-US/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ex-proj-us-umi" \
              --docker-bridge-address 172.17.0.1/16 \
              --dns-service-ip 10.2.0.10 \
              --pod-cidr 10.244.0.0/24 \
              --service-cidr 10.2.0.0/24 \
              --network-plugin kubenet \
              --node-vm-size Standard_D2s_v3 \
              --generate-ssh-keys
```

## Delete AKS Cluster

```
az aks delete -g EX-TEST -n ex-pri-stg
```