RANCHER-681 Investigate approach to reduce number of loadbalancers
Limits and price
Price
https://aws.amazon.com/elasticloadbalancing/pricing/?nc1=h_ls
Load Balancer | Price | LCU |
---|---|---|
Application | 0,0225 USD | 0,008 USD for Hour |
Network | 0.0225 USD | 0,006 USD for Hour |
Classic | 0.0225 USD | 0,008 USD for Gb |
Your AWS account has the following quotas related to Application Load Balancers.
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html
Name | Default | Adjustable |
---|---|---|
Application Load Balancers per Region | 50 | Yes |
Certificates per Application Load Balancer (excluding default certificates) | 25 | Yes |
Listeners per Application Load Balancer | 50 | Yes |
Number of times a target can be registered per Application Load Balancer | 1,000 | No |
Target Groups per Action per Application Load Balancer | 5 | No |
Target Groups per Application Load Balancer | 100 | No |
Targets per Application Load Balancer | 1,000 | Yes |
The following quotas are for rules.
Name | Default | Adjustable |
---|---|---|
Rules per Application Load Balancer (excluding default rules) | 100 | Yes |
Condition Values per Rule | 5 | No |
Condition Wildcards per Rule | 5 | No |
Match evaluations per rule | 5 | No |
Your AWS account has the following quotas related to Network Load Balancers.
https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-limits.html
Name | Default | Adjustable |
---|---|---|
Certificates per Network Load Balancer | 25 | Yes |
Listeners per Network Load Balancer | 50 | No |
Network Load Balancer ENIs per VPC | 1,200 ₁ | Yes |
Network Load Balancers per Region | 50 | Yes |
Target Groups per Action per Network Load Balancer | 1 | No |
Targets per Availability Zone per Network Load Balancer | 500 ₂, ₃ | Yes |
Targets per Network Load Balancer | 3,000 ₃ | Yes |
The following quotas are for target groups.
Name | Default | Adjustable |
---|---|---|
Target Groups per Region | 3,000 ₁ | Yes |
Targets per Target Group per Region (instances or IP addresses) | 1,000 | Yes |
Targets per Target Group per Region (Application Load Balancers) | 1 | No |
Your AWS account has the following quotas related to Classic Load Balancers.
https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-limits.html
Name | Default | Adjustable |
---|---|---|
Classic Load Balancers per Region | 20 | Yes |
Listeners per Classic Load Balancer | 100 | Yes |
Registered Instances per Classic Load Balancer | 1,000 | Yes |
How to do
https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html
Current
Create A record in Router53 for service
Template where we do it https://github.com/folio-org/folio-helm-v2/blob/07d49293a2bbd75a44c89f2af1965d92ba577b7a/charts/common/templates/_service.yaml
How we add data to it https://github.com/folio-org/pipelines-shared-library/blob/80b63af619da71ab8ada14eb5e074bfd3bbf4ebd/vars/helm.groovy
config[(module_name)]['ingress']['hosts'][0] += [host: domain]
Create load balancer add rules to it
Template where we do it https://github.com/folio-org/folio-helm-v2/blob/07d49293a2bbd75a44c89f2af1965d92ba577b7a/charts/common/templates/_ingress.yaml
Attach service to specific load balancer via group.
config[(module_name)]['ingress']['annotations'] += ['alb.ingress.kubernetes.io/group.name': "${project_config.getClusterName()}.${project_config.getProjectName()}"]
decrease alb
use only one static name for alb.ingress.kubernetes.io/group.name it allow to don't create a new alb for each namespace
config[(module_name)]['ingress']['annotations'] += ['alb.ingress.kubernetes.io/group.name': "${project_config.getClusterName()}.${project_config.getProjectName()}"]
Rules will be the same as current because we don't touch the Router53 records that has been creating by EKS .
We can reduce count records of Routers53 (Currently we pay for them about 4$).
For it we need to add into our ingress more http and modify the records.
For create new internal load balancer in Kubernetes
kubernetes.io/ingress.class: nginx
for external alb need to change routing to internal nginx lb
path: /namespace
backend:
resource:
apiGroup: k8s.example.com
name: namespace
Conclusion
We have about 15 alb on dev env.
In each of them 1 listener with about 17 rules that go to unique target group as result we have about 255 rules/target groups in limits we can see that we can increase count of rules but we can't increase the count of unique target group per alb.
Target Groups per Application Load Balancer | 100 |
I think it possible to increase this limit for us but for it we must communicate with AWS Support to ask them to ask their developers (because it hard code) increase this limit for us. For it we need to provide reasons why we need it and what the benefits it will give to us.