RANCHER-681 Investigate approach to reduce number of loadbalancers

Limits and price

Price

https://aws.amazon.com/elasticloadbalancing/pricing/?nc1=h_ls

Load BalancerPriceLCU
Application0,0225 USD0,008 USD for Hour
Network0.0225 USD0,006 USD for Hour
Classic0.0225 USD0,008 USD for Gb

Your AWS account has the following quotas related to Application Load Balancers.

https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html

NameDefaultAdjustable
Application Load Balancers per Region50Yes
Certificates per Application Load Balancer (excluding default certificates)25Yes
Listeners per Application Load Balancer50Yes
Number of times a target can be registered per Application Load Balancer1,000No
Target Groups per Action per Application Load Balancer5No
Target Groups per Application Load Balancer100No
Targets per Application Load Balancer1,000Yes

The following quotas are for rules.

NameDefaultAdjustable
Rules per Application Load Balancer (excluding default rules)100Yes
Condition Values per Rule5No
Condition Wildcards per Rule5No
Match evaluations per rule5No

Your AWS account has the following quotas related to Network Load Balancers.

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-limits.html

NameDefaultAdjustable
Certificates per Network Load Balancer25Yes
Listeners per Network Load Balancer50No
Network Load Balancer ENIs per VPC1,200 ₁Yes
Network Load Balancers per Region50Yes
Target Groups per Action per Network Load Balancer1No
Targets per Availability Zone per Network Load Balancer500 ₂, ₃Yes
Targets per Network Load Balancer3,000 ₃Yes

The following quotas are for target groups.

NameDefaultAdjustable
Target Groups per Region3,000 ₁Yes
Targets per Target Group per Region (instances or IP addresses)1,000Yes
Targets per Target Group per Region (Application Load Balancers)1No

Your AWS account has the following quotas related to Classic Load Balancers.

https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-limits.html

NameDefaultAdjustable
Classic Load Balancers per Region20Yes
Listeners per Classic Load Balancer100Yes
Registered Instances per Classic Load Balancer1,000Yes

How to do

https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html

Current

Create A record in Router53 for service

Template where we do it https://github.com/folio-org/folio-helm-v2/blob/07d49293a2bbd75a44c89f2af1965d92ba577b7a/charts/common/templates/_service.yaml

How we add data to it https://github.com/folio-org/pipelines-shared-library/blob/80b63af619da71ab8ada14eb5e074bfd3bbf4ebd/vars/helm.groovy

config[(module_name)]['ingress']['hosts'][0] += [host: domain]

Create load balancer add rules to it

Template where we do it https://github.com/folio-org/folio-helm-v2/blob/07d49293a2bbd75a44c89f2af1965d92ba577b7a/charts/common/templates/_ingress.yaml

Attach service to specific load balancer via group.

config[(module_name)]['ingress']['annotations'] += ['alb.ingress.kubernetes.io/group.name': "${project_config.getClusterName()}.${project_config.getProjectName()}"]


decrease alb

use only one static name for alb.ingress.kubernetes.io/group.name it allow to don't create a new alb for each namespace

config[(module_name)]['ingress']['annotations'] += ['alb.ingress.kubernetes.io/group.name': "${project_config.getClusterName()}.${project_config.getProjectName()}"]


Rules will be the same as current because we don't touch the Router53 records that has been creating by EKS .


We can reduce count records of Routers53 (Currently we pay for them about 4$).

For it we need to add into our ingress more http and modify the records.


For create new internal load balancer in Kubernetes 

kubernetes.io/ingress.class: nginx

for external alb need to change routing to internal nginx lb
path: /namespace
backend:
resource:
apiGroup: k8s.example.com
name: namespace


Conclusion

We have about 15 alb on dev env.

In each of them 1 listener with about 17 rules that go to unique target group as result we have about 255 rules/target groups in limits we can see that we can increase count of rules but we can't increase the count of unique target group per alb.

Target Groups per Application Load Balancer100

I think it possible to increase this limit for us but for it we must communicate with AWS Support to ask them to ask their developers (because it hard code) increase this limit for us. For it we need to provide reasons why we need it and what the benefits it will give to us.