94 lines
3.7 KiB
Markdown
94 lines
3.7 KiB
Markdown
|
# eks-lab
|
||
|
|
This module creates the following resources
|
||
|
|
- VPC
|
||
|
|
- Public and private subnets
|
||
|
|
- NAT gateway
|
||
|
|
- EKS cluster
|
||
|
|
- EKS nodegroup
|
||
|
|
- EKS bastion
|
||
|
|
- Install eksctl and kubectl on EKS bastion
|
||
|
|
|
||
|
|
## How to use eksctl and kubectl
|
||
|
|
By default, AWS EKS are installed with an aws-auth configmap which allows only the cluster creator
|
||
|
|
to work with the cluster. Therefore, one must first assume to the creator IAM role before running eksctl or kubectl.
|
||
|
|
For example, to create kube config, run these commands:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
export AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY="yyyy" AWS_DEFAULT_REGION=ap-northeast-1
|
||
|
|
aws eks update-kubeconfig --name lab-apne1-xpk-iac-cluster01
|
||
|
|
```
|
||
|
|
|
||
|
|
## Configure VPC CNI to use custom networking
|
||
|
|
```bash
|
||
|
|
kubectl set env daemonset aws-node -n kube-system AWS_VPC_K8S_CNI_CUSTOM_NETWORK_CFG=true
|
||
|
|
kubectl set env daemonset aws-node -n kube-system ENI_CONFIG_LABEL_DEF=failure-domain.beta.kubernetes.io/zone
|
||
|
|
|
||
|
|
cat <<EOF | kubectl apply -f -
|
||
|
|
apiVersion: crd.k8s.amazonaws.com/v1alpha1
|
||
|
|
kind: ENIConfig
|
||
|
|
metadata:
|
||
|
|
name: ap-northeast-1a
|
||
|
|
spec:
|
||
|
|
subnet: subnet-0d015cc72715685ca
|
||
|
|
EOF
|
||
|
|
|
||
|
|
cat <<EOF | kubectl apply -f -
|
||
|
|
apiVersion: crd.k8s.amazonaws.com/v1alpha1
|
||
|
|
kind: ENIConfig
|
||
|
|
metadata:
|
||
|
|
name: ap-northeast-1c
|
||
|
|
spec:
|
||
|
|
subnet: subnet-030ee2c3e2b730fcc
|
||
|
|
EOF
|
||
|
|
```
|
||
|
|
|
||
|
|
Then redeploy the nodegroup
|
||
|
|
```bash
|
||
|
|
terraform apply -replace="aws_eks_node_group.eks-nodegroup"
|
||
|
|
```
|
||
|
|
|
||
|
|
If successfully done, you will start to see 100.64.0.0 addresses being used on the EKS worker nodes. You can also see it with kubectl:
|
||
|
|
|
||
|
|
|
||
|
|
```bash
|
||
|
|
root@ip-192-168-123-48:~# kubectl get pods --all-namespaces -o wide
|
||
|
|
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
|
||
|
|
kube-system aws-node-5892k 1/1 Running 0 4m9s 192.168.123.245 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
kube-system coredns-5fc8d4cdcf-c75z6 1/1 Running 0 13m 100.64.9.249 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
kube-system coredns-5fc8d4cdcf-h5lnl 1/1 Running 0 13m 100.64.13.41 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
kube-system ebs-csi-controller-d6bff959-8459z 6/6 Running 0 13m 100.64.8.74 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
kube-system ebs-csi-controller-d6bff959-vnwlf 6/6 Running 0 5m28s 100.64.11.124 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
kube-system ebs-csi-node-h7w8r 3/3 Running 0 4m9s 100.64.11.188 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
kube-system kube-proxy-vgmdf 1/1 Running 0 4m9s 192.168.123.245 ip-192-168-123-245.ap-northeast-1.compute.internal <none> <none>
|
||
|
|
|
||
|
|
```
|
||
|
|
|
||
|
|
## Edit configmap/aws-auth
|
||
|
|
```
|
||
|
|
kubectl edit -n kube-system configmap/aws-auth
|
||
|
|
```
|
||
|
|
Add a group with system:master role
|
||
|
|
```yaml
|
||
|
|
apiVersion: v1
|
||
|
|
data:
|
||
|
|
mapRoles: |
|
||
|
|
- groups:
|
||
|
|
- system:bootstrappers
|
||
|
|
- system:nodes
|
||
|
|
rolearn: arn:aws:iam::040216112220:role/clusterCreator
|
||
|
|
username: system:node:Template:EC2PrivateDNSName
|
||
|
|
- groups:
|
||
|
|
- system:masters
|
||
|
|
rolearn: arn:aws:iam::040216112220:role/lab-apne1-xpk-iac-bast-role
|
||
|
|
username: lab-apne1-xpk-iac-bast-role
|
||
|
|
kind: ConfigMap
|
||
|
|
metadata:
|
||
|
|
creationTimestamp: "2022-12-29T11:02:15Z"
|
||
|
|
name: aws-auth
|
||
|
|
namespace: kube-system
|
||
|
|
resourceVersion: "59670"
|
||
|
|
uid: 7cf9d889-8ed2-4c8d-ac0f-092184cede8a
|
||
|
|
```
|
||
|
|
|
||
|
|
## Addon updates
|
||
|
|
When updating addons, please select advanced options and choose preserve settings.
|