2023-06-02 Meeting Notes
Attendees:
@Maccabee Levine
@Peter Murray
@Steffen Köhler
@Yogesh Kumar
Off-Hours Guidelines
D4. Define guidelines/best practices around pausing/stopping environments when they're not in use - e.g. off-hours/weekends/etc.
Last week & prior:
@Yogesh Kumar still working on a few details with Kitfox
Should release environments also have off-hours? (From Kitfox sprint review: @Peter Murray discussed bringing to PC and TC whether we bring the same weekend suspension to the hosted release environments. Peter will look at what savings that would be.)
Maybe start with weekends only? @Peter Murray will estimate cost savings. No actual env changes for that right now.
Look at logs also, see who accesses them over the weekend.
Maybe higher use nearer a flower release.
Look at snapshot, snapshot-2, and the two release environments
@Yogesh Kumar still a few things to wrap up with Kitfox (updating document), hopefully wrapping up today.
@Peter Murray still to look at the weekend cost savings estimate for reference release environments.
Today:
@Maccabee Levine From discussion with PC, prefer not to consider reference environments for night/weekend shutdowns. Consensus agrees.
@Peter Murray didn't calculate estimate since councils didn't want to do it. If time allows still put into report, otherwise never mind.
@Yogesh Kumar what was left? weekday schedule? Yogesh will review through Monday. Let us know in Slack one way or the other.
Budgets / Cost Anomaly Detection
D5. Create AWS Budgets and AWS Budget Alerts for daily and monthly spend rates
D6. Explore AWS Cost Anomaly Detection and Rightsizing Recommendations
Last time:
Review draft environment "price list" / "recipes" from @Yogesh Kumar if ready.
Yogesh presented: Namespace configurations costs estimation
Consensus looks good!
@Yogesh Kumar will double-check Kafka embedded vs. shared – why is shared more expensive?
Add "daily" by price in each table.
Today:
@Yogesh Kumar will ask again about Kafka difference. Or forget about it.
Reviewing Environments to Shut Down
D2. Define a process for reviewing existing tools and environments for candidates to be shut down (e.g. when a team leaves the project or the env is no longer needed)
Last time / Prior:
@Mark Veksler draft guidelines on who should have permissions to what operations in AWS. What will each team be allowed to do. Link from ACRG doc.
Permissions on operations in AWS? Right now AWS is just Kitfox, but Jenkins jobs are available to dev teams.
Kitfox prefers a self-service model.
@Yogesh Kumar will update the environment lifecycle document to indicate what dev teams can do.
Today:
@Yogesh Kumar will wrap up dev permissions by early next week.
Off-hours shutdown during weekday evenings (team environments)
Last time / prior:
Kitfox is looking at this. What would work best for each team? Ticket pending.
We can at least look at the findings, and decide to do something or not. Harder b/c geographical spread.
@Yogesh Kumar still investigating
Today:
If time. Otherwise just go with weekends for phase 1.
One-time estimate?
Last time:
So ACRG has something to base it on in future years. Estimate and how we made it.
Treasurer probably about to ask anyway.
@Peter Murray think about a process. Reconstruct from past years, bring to the group.
Today: