Attendees:

• @Maccabee Levine 

• @Steffen Köhler 

• @Yogesh Kumar 

• @Peter Murray 

Off-Hours Guidelines

D4. Define guidelines/best practices around pausing/stopping environments when they're not in use - e.g. off-hours/weekends/etc.

Last week: 

• @Yogesh Kumar still working on a few details with Kitfox

• Should release environments also have off-hours?  (From Kitfox sprint review: @Peter Murray discussed bringing to PC and TC whether we bring the same weekend suspension to the hosted release environments.  Peter will look at what savings that would be.)

  • Maybe start with weekends only?   @Peter Murray will estimate cost savings.  No actual env changes for that right now.

    • Look at logs also, see who accesses them over the weekend.

    • Maybe higher use nearer a flower release.

    • Look at snapshot, snapshot-2, and the two release environments

Today:

• @Yogesh Kumar still a few things to wrap up with Kitfox (updating document), hopefully wrapping up today.  

• @Peter Murray still to look at the weekend cost savings estimate for reference release environments.

Budgets / Cost Anomaly Detection

D5. Create AWS Budgets and AWS Budget Alerts for daily and monthly spend rates

D6. Explore AWS Cost Anomaly Detection and Rightsizing Recommendations

Last time:

• How would teams estimate costs?  AWS Calculator lets you put together a set of AWS services, create a link to share estimate.  But you have to know what you are asking for.  X size databases, open search, etc.

  • @Yogesh Kumar ask Kitfox to determine standard environment cost, i.e. Bama.  Maybe a price list, where the standard environment uses X resources, has X monthly cost.  Team may need to increase size of DB, so higher proportion of cost. 

  • Or simplify.  Maybe just two "recipes", and what the monthly cost was for those recipes.  "Standard" and "Premium".  Or "Dev" vs "Testing".  Start from there.

Today:

• Review draft environment "price list" / "recipes" from @Yogesh Kumar if ready.

• Yogesh presented: Namespace configurations costs estimation

• Consensus looks good!

  • @Yogesh Kumar will double-check Kafka embedded vs. shared – why is shared more expensive?

  • Add "daily" by price in each table.

Reviewing Environments to Shut Down

D2. Define a process for reviewing existing tools and environments for candidates to be shut down (e.g. when a team leaves the project or the env is no longer needed)

Last time / Prior:

• @Mark Veksler draft guidelines on who should have permissions to what operations in AWS.  What will each team be allowed to do.  Link from ACRG doc.

• Permissions on operations in AWS?  Right now AWS is just Kitfox, but Jenkins jobs are available to dev teams.

• Kitfox prefers a self-service model.

• @Yogesh Kumar will update the environment lifecycle document to indicate what dev teams can do.

Today:

• TODO

Off-hours shutdown during weekday evenings

Last time:

• Kitfox is looking at this.  What would work best for each team?  Ticket pending.

• We can at least look at the findings, and decide to do something or not.  Harder b/c geographical spread.

Today:

• @Yogesh Kumar still investigating

One-time estimate?

• So ACRG has something to base it on in future years.  Estimate and how we made it.

• Treasurer probably about to ask anyway.

• @Peter Murray think about a process.  Reconstruct from past years, bring to the group.