120 lines
4.0 KiB
Markdown
120 lines
4.0 KiB
Markdown
|
# eks-lab/eks
|
|||
|
|
This layer creates the following resources
|
|||
|
|
- EKS cluster using ipv6 for service network
|
|||
|
|
- EKS nodegroup
|
|||
|
|
- EKS bastion
|
|||
|
|
- Install eksctl, kubectl, awscliv2, helm on EKS bastion with user_data script
|
|||
|
|
|
|||
|
|
Be patient. EKS cluster takes 12min to provision. Node group will take another 5 min. And the cluster addon takes another ?? min.
|
|||
|
|
|
|||
|
|
## Worker node instance size
|
|||
|
|
Choose t3.large at the minimum. This is due to AWS's limitation on number of IPs. Smaller instanecs are limited with 6 IP
|
|||
|
|
which is not enough. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI
|
|||
|
|
|
|||
|
|
## How to use eksctl and kubectl
|
|||
|
|
By default, AWS EKS are installed with an aws-auth configmap which allows only the cluster creator
|
|||
|
|
to work with the cluster. Therefore, one must first assume to the creator IAM role before running eksctl or kubectl.
|
|||
|
|
For example, to create kube config, run these commands:
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
export AWS_ACCESS_KEY_ID=xxxx AWS_SECRET_ACCESS_KEY="yyyy" AWS_DEFAULT_REGION=ap-northeast-1
|
|||
|
|
aws eks update-kubeconfig --name lab-apne1-xpk-iac-cluster01
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Edit configmap/aws-auth
|
|||
|
|
```
|
|||
|
|
kubectl edit -n kube-system configmap/aws-auth
|
|||
|
|
```
|
|||
|
|
Add a group with system:master role
|
|||
|
|
```yaml
|
|||
|
|
apiVersion: v1
|
|||
|
|
data:
|
|||
|
|
mapRoles: |
|
|||
|
|
- groups:
|
|||
|
|
- system:bootstrappers
|
|||
|
|
- system:nodes
|
|||
|
|
rolearn: arn:aws:iam::040216112220:role/clusterCreator
|
|||
|
|
username: system:node:Template:EC2PrivateDNSName
|
|||
|
|
- groups:
|
|||
|
|
- system:masters
|
|||
|
|
rolearn: arn:aws:iam::040216112220:role/lab-apne1-xpk-iac-bast-role
|
|||
|
|
username: lab-apne1-xpk-iac-bast-role
|
|||
|
|
kind: ConfigMap
|
|||
|
|
metadata:
|
|||
|
|
creationTimestamp: "2022-12-29T11:02:15Z"
|
|||
|
|
name: aws-auth
|
|||
|
|
namespace: kube-system
|
|||
|
|
resourceVersion: "59670"
|
|||
|
|
uid: 7cf9d889-8ed2-4c8d-ac0f-092184cede8a
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Addon updates
|
|||
|
|
When updating addons, please select advanced options and choose preserve settings.
|
|||
|
|
|
|||
|
|
## Install ALB ingress controller
|
|||
|
|
AWS provides documentation on how to deploy a sample application with ingress (ALB)
|
|||
|
|
https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
|
|||
|
|
|
|||
|
|
That depends on the load balancer container, which can be deployed by
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/install/iam_policy.json
|
|||
|
|
|
|||
|
|
aws iam create-policy \
|
|||
|
|
--policy-name AWSLoadBalancerControllerIAMPolicy \
|
|||
|
|
--policy-document file://iam_policy.json
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
Create an openid provider on iam
|
|||
|
|
https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
eksctl create iamserviceaccount \
|
|||
|
|
--cluster=lab-apne1-xpk-iac-cluster01 \
|
|||
|
|
--namespace=kube-system \
|
|||
|
|
--name=aws-load-balancer-controller \
|
|||
|
|
--role-name AmazonEKSLoadBalancerControllerRole \
|
|||
|
|
--attach-policy-arn=arn:aws:iam::040216112220:policy/AWSLoadBalancerControllerIAMPolicy \
|
|||
|
|
--approve
|
|||
|
|
|
|||
|
|
helm repo add eks https://aws.github.io/eks-charts
|
|||
|
|
helm repo update
|
|||
|
|
helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
|
|||
|
|
-n kube-system \
|
|||
|
|
--set clusterName=lab-apne1-xpk-iac-cluster01 \
|
|||
|
|
--set serviceAccount.create=false \
|
|||
|
|
--set serviceAccount.name=aws-load-balancer-controller
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Tag subnets
|
|||
|
|
Reference: https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
|
|||
|
|
|
|||
|
|
The following tags are set in the network layer:
|
|||
|
|
|
|||
|
|
On private subnets:
|
|||
|
|
Key – kubernetes.io/role/internal-elb
|
|||
|
|
Value – 1
|
|||
|
|
|
|||
|
|
On public subnets:
|
|||
|
|
Key – kubernetes.io/role/elb
|
|||
|
|
Value – 1
|
|||
|
|
|
|||
|
|
|
|||
|
|
## Install sample app the 2048 game
|
|||
|
|
See https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
|
|||
|
|
```bash
|
|||
|
|
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/examples/2048/2048_full.yaml
|
|||
|
|
edit the file
|
|||
|
|
kubectl apply -f 2048_full.yaml
|
|||
|
|
kubectl get ingress/ingress-2048 -n game-2048
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
In a moment, the lb address should be displayed
|
|||
|
|
```bash
|
|||
|
|
root@ip-192-168-123-187:~# kubectl get ingress/ingress-2048 -n game-2048
|
|||
|
|
NAME CLASS HOSTS ADDRESS PORTS AGE
|
|||
|
|
ingress-2048 alb * internal-k8s-game2048-ingress2-5f196824a1-20502803.ap-northeast-1.elb.amazonaws.com 80 7s
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|