# eks-lab
This module creates the following resources
- VPC
- Public and private subnets
- NAT gateway
- EKS cluster
- EKS nodegroup
- EKS bastion
- Install eksctl and kubectl on EKS bastion

## How to use eksctl and kubectl
By default, AWS EKS are installed with an aws-auth configmap which allows only the cluster creator
to work with the cluster. Therefore, one must first assume to the creator IAM role before running eksctl or kubectl.
For example, to create kube config, run these commands:

```bash
export AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY="yyyy" AWS_DEFAULT_REGION=ap-northeast-1
aws eks update-kubeconfig --name lab-apne1-xpk-iac-cluster01
```

## Configure VPC CNI to use custom networking
```bash
kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
kubectl set env daemonset aws-node -n kube-system ENI_CONFIG_LABEL_DEF=failure-domain.beta.kubernetes.io/zone

cat <<EOF  | kubectl apply -f -
apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
 name: ap-northeast-1a
spec:
  subnet: subnet-0d015cc72715685ca
EOF

cat <<EOF | kubectl apply -f -
apiVersion: crd.k8s.amazonaws.com/v1alpha1
kind: ENIConfig
metadata:
 name: ap-northeast-1c
spec:
  subnet: subnet-030ee2c3e2b730fcc
EOF
```

Then redeploy the nodegroup
```bash
terraform apply -replace="aws_eks_node_group.eks-nodegroup"
```

If successfully done, you will start to see 100.64.0.0 addresses being used on the EKS worker nodes. You can also see it with kubectl:


```bash
root@ip-192-168-123-48:~# kubectl  get pods --all-namespaces -o wide
NAMESPACE     NAME                                READY   STATUS    RESTARTS   AGE     IP                NODE                                                 NOMINATED NODE   READINESS GATES
kube-system   aws-node-5892k                      1/1     Running   0          4m9s    192.168.123.245   ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>
kube-system   coredns-5fc8d4cdcf-c75z6            1/1     Running   0          13m     100.64.9.249      ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>
kube-system   coredns-5fc8d4cdcf-h5lnl            1/1     Running   0          13m     100.64.13.41      ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>
kube-system   ebs-csi-controller-d6bff959-8459z   6/6     Running   0          13m     100.64.8.74       ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>
kube-system   ebs-csi-controller-d6bff959-vnwlf   6/6     Running   0          5m28s   100.64.11.124     ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>
kube-system   ebs-csi-node-h7w8r                  3/3     Running   0          4m9s    100.64.11.188     ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>
kube-system   kube-proxy-vgmdf                    1/1     Running   0          4m9s    192.168.123.245   ip-192-168-123-245.ap-northeast-1.compute.internal   <none>           <none>

```

## Edit configmap/aws-auth
```
kubectl edit -n kube-system configmap/aws-auth
```
Add a group with system:master role
```yaml
apiVersion: v1
data:
  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::040216112220:role/clusterCreator
      username: system:node:Template:EC2PrivateDNSName
    - groups:
      - system:masters
      rolearn: arn:aws:iam::040216112220:role/lab-apne1-xpk-iac-bast-role
      username: lab-apne1-xpk-iac-bast-role
kind: ConfigMap
metadata:
  creationTimestamp: "2022-12-29T11:02:15Z"
  name: aws-auth
  namespace: kube-system
  resourceVersion: "59670"
  uid: 7cf9d889-8ed2-4c8d-ac0f-092184cede8a
```

## Addon updates
When updating addons, please select advanced options and choose preserve settings.