Attendees:
Off-Hours Guidelines
D4. Define guidelines/best practices around pausing/stopping environments when they're not in use - e.g. off-hours/weekends/etc.
Last time:
- Mark Veksler draft off-hours guidelines
- Mark drafted them: https://docs.google.com/document/d/11lEgNrrQUetm1zq3Kmo1cWf4AnmGeL8J7k7iUue84vM/edit#heading=h.xwjbpjkuamcy
Today:
Budgets / Cost Anomaly Detection
D5. Create AWS Budgets and AWS Budget Alerts for daily and monthly spend rates
D6. Explore AWS Cost Anomaly Detection and Rightsizing Recommendations
Last time:
- Are there Jiras for these yet? Sprints identified?
- Mark Veksler Peter was setting up anomaly detection? Set up the budgets in AWS Cost Explorer.
- At some point, will have to assign budget to each team. How do we do that? Maybe need a spike to do that. Yogesh can create Jiras, and work with the team to prioritize those things.
- AWS Cost Review Group should periodically review the actuals vs. budget. And review the budget on a different cadence, maybe around each flower release. And revising it annually (strategic).
- Maccabee Levine asked: Are you suggesting the group just estimate from the current active environments (which should be doable -- each one will have a monthly budget, and end date, listed in the environment request)? But how would they take into account any new requests that might crop up later in the year? I imagine there would be a lot of guess work going on, but maybe you have a metric in mind?
- Yogesh Kumar noted: We could use the historical new requests data to determine what's needed. Would it work to chat about it on Friday? I'd appreciate more context so I can better understand the request.
- How would the dev team estimate costs. Maybe want to develop a calculator? Or an excel spreadsheet? Including all of the AWS components involved. Yogesh Kumar or Peter Murray ? Make as simple as possible for development team.
- Peter Murray noted: AWS does have a calculator, conveniently named https://calculator.aws/.
- Mark Veksler or Yogesh Kumar Need reports that includes the three budget points for each environment: end date, total cost, total monthly cost. May need to work with Kitfox to have a breakdown by team.
Mark Veksler you suggested this on 4/28 and that Yogesh Kumar might lead this. Would one of you please define more of the specifics here? A few questions I'm thinking of:
- Is this scheduled push (i.e. Kitfox generates the reports for the ACRG and dev teams every month) or on request?
- How can this be done to be as little work as possible for Kitfox? I.e. hopefully some initial setup and then just press a button (if that) to refresh the reports?
Today:
Reviewing Environments to Shut Down
D2. Define a process for reviewing existing tools and environments for candidates to be shut down (e.g. when a team leaves the project or the env is no longer needed)
Last time:
- Mark Veksler draft guidelines on who should have permissions to what operations in AWS. What will each team be allowed to do. Link from ACRG doc.
Today:
Future Topics
- Off-hours shutdown during weekday evenings
- Environment requests for the existing environments when the new procedures are approved