RANCHER-681 Investigate approach to reduce number of loadbalancers

RANCHER-681 Investigate approach to reduce number of loadbalancers

Limits and price

Price

https://aws.amazon.com/elasticloadbalancing/pricing/?nc1=h_ls

Load Balancer

Price

LCU

Application

0,0225 USD

0,008 USD for Hour

Network

0.0225 USD

0,006 USD for Hour

Classic

0.0225 USD

0,008 USD for Gb

Your AWS account has the following quotas related to Application Load Balancers.

https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html

Name

Default

Adjustable

Application Load Balancers per Region

50

Yes

Certificates per Application Load Balancer (excluding default certificates)

25

Yes

Listeners per Application Load Balancer

50

Yes

Number of times a target can be registered per Application Load Balancer

1,000

No

Target Groups per Action per Application Load Balancer

5

No

Target Groups per Application Load Balancer

100

No

Targets per Application Load Balancer

1,000

Yes

The following quotas are for rules.

Name

Default

Adjustable

Rules per Application Load Balancer (excluding default rules)

100

Yes

Condition Values per Rule

5

No

Condition Wildcards per Rule

5

No

Match evaluations per rule

5

No

Your AWS account has the following quotas related to Network Load Balancers.

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-limits.html

Name

Default

Adjustable

Certificates per Network Load Balancer

25

Yes

Listeners per Network Load Balancer

50

No

Network Load Balancer ENIs per VPC

1,200 ₁

Yes

Network Load Balancers per Region

50

Yes

Target Groups per Action per Network Load Balancer

1

No

Targets per Availability Zone per Network Load Balancer

500 ₂, ₃

Yes

Targets per Network Load Balancer

3,000 ₃

Yes

The following quotas are for target groups.

Name

Default

Adjustable

Target Groups per Region

3,000 ₁

Yes

Targets per Target Group per Region (instances or IP addresses)

1,000

Yes

Targets per Target Group per Region (Application Load Balancers)

1

No

Your AWS account has the following quotas related to Classic Load Balancers.

https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-limits.html

Name

Default

Adjustable

Classic Load Balancers per Region

20

Yes

Listeners per Classic Load Balancer

100

Yes

Registered Instances per Classic Load Balancer

1,000

Yes

How to do

https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html

Current

Create A record in Router53 for service

Template where we do it https://github.com/folio-org/folio-helm-v2/blob/07d49293a2bbd75a44c89f2af1965d92ba577b7a/charts/common/templates/_service.yaml

How we add data to it https://github.com/folio-org/pipelines-shared-library/blob/80b63af619da71ab8ada14eb5e074bfd3bbf4ebd/vars/helm.groovy

config[(module_name)]['ingress']['hosts'][0] += [host: domain]

Create load balancer add rules to it

Template where we do it https://github.com/folio-org/folio-helm-v2/blob/07d49293a2bbd75a44c89f2af1965d92ba577b7a/charts/common/templates/_ingress.yaml

Attach service to specific load balancer via group.

config[(module_name)]['ingress']['annotations'] += ['alb.ingress.kubernetes.io/group.name': "${project_config.getClusterName()}.${project_config.getProjectName()}"]

 

decrease alb

use only one static name for alb.ingress.kubernetes.io/group.name it allow to don't create a new alb for each namespace

config[(module_name)]['ingress']['annotations'] += ['alb.ingress.kubernetes.io/group.name': "${project_config.getClusterName()}.${project_config.getProjectName()}"]

 

Rules will be the same as current because we don't touch the Router53 records that has been creating by EKS .

 

We can reduce count records of Routers53 (Currently we pay for them about 4$).

For it we need to add into our ingress more http and modify the records.

 

For create new internal load balancer in Kubernetes 

kubernetes.io/ingress.class: nginx

for external alb need to change routing to internal nginx lb
path: /namespace
backend:
resource:
apiGroup: k8s.example.com
name: namespace

 

Conclusion

We have about 15 alb on dev env.

In each of them 1 listener with about 17 rules that go to unique target group as result we have about 255 rules/target groups in limits we can see that we can increase count of rules but we can't increase the count of unique target group per alb.

Target Groups per Application Load Balancer

100

I think it possible to increase this limit for us but for it we must communicate with AWS Support to ask them to ask their developers (because it hard code) increase this limit for us. For it we need to provide reasons why we need it and what the benefits it will give to us.