120 lines
4.0 KiB
Markdown
120 lines
4.0 KiB
Markdown
# eks-lab/eks
|
||
This layer creates the following resources
|
||
- EKS cluster using ipv6 for service network
|
||
- EKS nodegroup
|
||
- EKS bastion
|
||
- Install eksctl, kubectl, awscliv2, helm on EKS bastion with user_data script
|
||
|
||
Be patient. EKS cluster takes 12min to provision. Node group will take another 5 min. And the cluster addon takes another ?? min.
|
||
|
||
## Worker node instance size
|
||
Choose t3.large at the minimum. This is due to AWS's limitation on number of IPs. Smaller instanecs are limited with 6 IP
|
||
which is not enough. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI
|
||
|
||
## How to use eksctl and kubectl
|
||
By default, AWS EKS are installed with an aws-auth configmap which allows only the cluster creator
|
||
to work with the cluster. Therefore, one must first assume to the creator IAM role before running eksctl or kubectl.
|
||
For example, to create kube config, run these commands:
|
||
|
||
```bash
|
||
export AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY="yyyy" AWS_DEFAULT_REGION=ap-northeast-1
|
||
aws eks update-kubeconfig --name lab-apne1-xpk-iac-cluster01
|
||
```
|
||
|
||
## Edit configmap/aws-auth
|
||
```
|
||
kubectl edit -n kube-system configmap/aws-auth
|
||
```
|
||
Add a group with system:master role
|
||
```yaml
|
||
apiVersion: v1
|
||
data:
|
||
mapRoles: |
|
||
- groups:
|
||
- system:bootstrappers
|
||
- system:nodes
|
||
rolearn: arn:aws:iam::040216112220:role/clusterCreator
|
||
username: system:node:Template:EC2PrivateDNSName
|
||
- groups:
|
||
- system:masters
|
||
rolearn: arn:aws:iam::040216112220:role/lab-apne1-xpk-iac-bast-role
|
||
username: lab-apne1-xpk-iac-bast-role
|
||
kind: ConfigMap
|
||
metadata:
|
||
creationTimestamp: "2022-12-29T11:02:15Z"
|
||
name: aws-auth
|
||
namespace: kube-system
|
||
resourceVersion: "59670"
|
||
uid: 7cf9d889-8ed2-4c8d-ac0f-092184cede8a
|
||
```
|
||
|
||
## Addon updates
|
||
When updating addons, please select advanced options and choose preserve settings.
|
||
|
||
## Install ALB ingress controller
|
||
AWS provides documentation on how to deploy a sample application with ingress (ALB)
|
||
https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
|
||
|
||
That depends on the load balancer container, which can be deployed by
|
||
|
||
```bash
|
||
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/install/iam_policy.json
|
||
|
||
aws iam create-policy \
|
||
--policy-name AWSLoadBalancerControllerIAMPolicy \
|
||
--policy-document file://iam_policy.json
|
||
|
||
```
|
||
Create an openid provider on iam
|
||
https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html
|
||
|
||
```
|
||
eksctl create iamserviceaccount \
|
||
--cluster=lab-apne1-xpk-iac-cluster01 \
|
||
--namespace=kube-system \
|
||
--name=aws-load-balancer-controller \
|
||
--role-name AmazonEKSLoadBalancerControllerRole \
|
||
--attach-policy-arn=arn:aws:iam::040216112220:policy/AWSLoadBalancerControllerIAMPolicy \
|
||
--approve
|
||
|
||
helm repo add eks https://aws.github.io/eks-charts
|
||
helm repo update
|
||
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
|
||
-n kube-system \
|
||
--set clusterName=lab-apne1-xpk-iac-cluster01 \
|
||
--set serviceAccount.create=false \
|
||
--set serviceAccount.name=aws-load-balancer-controller
|
||
```
|
||
|
||
## Tag subnets
|
||
Reference: https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
|
||
|
||
The following tags are set in the network layer:
|
||
|
||
On private subnets:
|
||
Key – kubernetes.io/role/internal-elb
|
||
Value – 1
|
||
|
||
On public subnets:
|
||
Key – kubernetes.io/role/elb
|
||
Value – 1
|
||
|
||
|
||
## Install sample app the 2048 game
|
||
See https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
|
||
```bash
|
||
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/examples/2048/2048_full.yaml
|
||
edit the file
|
||
kubectl apply -f 2048_full.yaml
|
||
kubectl get ingress/ingress-2048 -n game-2048
|
||
```
|
||
|
||
In a moment, the lb address should be displayed
|
||
```bash
|
||
root@ip-192-168-123-187:~# kubectl get ingress/ingress-2048 -n game-2048
|
||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||
ingress-2048 alb * internal-k8s-game2048-ingress2-5f196824a1-20502803.ap-northeast-1.elb.amazonaws.com 80 7s
|
||
```
|
||
|
||
|