Existing tags for monitoring costs in Kubecost
EKS tags:
- kubernetes_cluster = cluster_name - added to all EKS resources created by terraform-aws-eks Terraform module
RDS, MSK, ES tags:
- kubernetes_cluster =
...
- cluster_name (ex. folio-dev, folio-perf)
- kubernetes_namespace =
...
- namespace_
...
- name (ex. volaris, folijet)
- kubernetes_label_team =
...
- team_name (ex. volaris, folijet)
- kubernetes_service =
...
- name_of_service (ex., ES-Dashboard
...
- )
Resources created by Terraform
| Resource Name | Existing Tags | Add Tags | Notes |
|---|---|---|---|
| Cluster folder | |||
| EKS Cluster | Terraform = true Team = kitfox Project = folio | Terraform = true Team = kitfox Project = folio Env = folio-testing Region = us-west-2 | service tag cannot add https://github.com/terraform-aws-modules/terraform-aws-eks/blob/v18.26.6/main.tf#L46 |
| Node Group | Terraform = true Team = kitfox Project = folio Name = folio-testing | Terraform = true Team = kitfox Project = folio Name = folio-testing Env = folio-testing Region = us-west-2 Service = node-group ? | Service tag in file terraform\rancher\cluster\eks.tf try to add tags. https://github.com/terraform-aws-modules/terraform-aws-eks/blob/v18.26.6/node_groups.tf#L223 |
| EC2 | Terraform = true Team = kitfox Project = folio Name = folio-testing | Terraform = true Team = kitfox Project = folio Name = folio-testing Env = folio-testing Region = us-west-2 | https://github.com/terraform-aws-modules/terraform-aws-eks/blob/v18.26.6/main.tf#L65 service tag cannot add |
| IAM roles | Terraform = true Team = kitfox Project = folio Name = load-balancer-controller-role | Terraform = true Team = kitfox Project = folio Name = load-balancer-controller-role Env = folio-testing Region = us-west-2 Service = iam-role | In file terraform\rancher\cluster\iam.tf in tags block add Service tag. |
| Network folder | |||
| VPC | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc Env = folio-testing Region = us-west-2 Service = vpc | Service tag https://github.com/terraform-aws-modules/terraform-aws-vpc/blob/v3.14.0/main.tf#L31 |
| Private subnet | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-private-us-west-2c | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-private-us-west-2c Env = folio-testing Region = us-west-2 Service = subnet | In file terraform\rancher\network\main.tf in block private_subnet_tags add service tag |
| Public subnet | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-public-us-west-2c | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-public-us-west-2c Env = folio-testing Region = us-west-2 Service =subnet | In file terraform\rancher\network\main.tf in block public_subnet_tags add service tag |
| Database subnet | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-db-us-west-2c | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-db-us-west-2c Env = folio-testing Region = us-west-2 Service = subnet | In file terraform\rancher\network\main.tf in block database_subnet_tags add service tag |
| Elastic IP | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-nat-eip | Terraform = true Team = kitfox Project = folio Name = folio-rancher-vpc-nat-eip Env = folio-testing Region = us-west-2 Service = elastic-ip | In terraform\rancher\network\main.tf in aws_eip resource add tag service in tag block |
| Project folder | |||
| Security Group | Environment = dev Name = allow_es Terraform = true | Env = folio-testing Name = allow_es Terraform = true Region = us-west-2 Service = security_group | Changes in file terraform\rancher\project\elasticsearch.tf and in file terraform\rancher\project\kafka.tf |
| Amazon OpenSearch | Name = es-perf-folijet Terraform = true Service = ElasticSearch Version = 7.10 | Name = es-perf-folijet Terraform = true Service = ElasticSearch Version = 7.10 Region = us-west-2 Env = folio-testing | Changes in file terraform\rancher\project\elasticsearch.tf |
| Amazon MSK | Name = KAFKA-PERF-bulk-edit service = kafka | Name = kafka-bulk-edit Terraform = true Service = msk Version = 7.10 Region = us-west-2 Env = folio-testing | Changes in file terraform\rancher\project\kafka.tf |
| Amazon RDS | Terraform = true Environment = dev | Terraform = true Env = folio-testing Service = rds Region = us-west-2 | Changes in terraform\rancher\project\postgresql.tf Tags do not match with Terraform code. |
...
Kubecost configuration
SPOT instances
Kubecost will reconcile your spot prices with CUR billing reports as they become available (usually 1-2 days), but pricing data can be pulled hourly by integrating directly with the AWS spot feed.
For enabling hourly integration for SPOT:
- Create a bucket for SPOT logs
- Create user and grant read access to the bucket (copy API key and secret)
- In terraform\rancher\cluster\kubecost.tf file add values:
Additional info:
https://guide.kubecost.com/hc/en-us/articles/4407595928087#spot-data-feed-integration
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-data-feeds.html
Authentification using AWS Cognito
Manually steps:
For cost saving, we use one User Pool for all our clusters, but different App Clients.
Before deploying Kubecost check that Kubecost user pool exists in AWS Cognito or create a new one.
Configuration for the user account, policies and others can be set up for project purposes.
Create a domain name using Cognito. The name prefix must be unique. In our configuration using folio-kubecost.
App client creates and configured by Terraform code automatically.
| Info |
|---|
| Please, pay attention. If you change User Pool or domain prefix name change it also in code (terraform\rancher\cluster\kubecost.tf file). |