2023-04-21 Meeting Notes

Owned by Maccabee Levine
Last updated: Apr 21, 2023

Attendees:


Agenda:


Plans for D4, D5, D6

D4. Define guidelines/best practices around pausing/stopping environments when they're not in use - e.g. off-hours/weekends/etc.

D5. Create AWS Budgets and AWS Budget Alerts for daily and monthly spend rates

D6. Explore AWS Cost Anomaly Detection and Rightsizing Recommendations

From last week:

  • Mark has asked the team to add these three items to their backlog.  Mark Veksler  Which sprint?

This week:

  • This Monday they will demo and talk about it.
  • Starting in the next env to shut some environments down nights & weekends: "This will take effect from 6PM EST (Friday) until 6:30 PM EST (Sunday) starting from 21st of April."
  • Mark has asked the team to add these three items to their backlog.  Mark Veksler  Which sprint?

AWS Cost Review Group

Last week in Slack (excerpt https://folio-project.slack.com/archives/C050YSN4N68/p1681483340287699)

  • Regarding the proposed "cost review group".  We had consensus today (I think) on a few points:

    • Some human(s) need to actually make the yes/no decision on each request. 

    • Preferably a very small, agile group, and reacting only to the specified approval criteria.

    • Since these environments are only being created based on project needs, it would be a pretty bad situation if the group denied a request.

    • Community money is involved, and a very large environment request would have a large budget impact, so some review is still needed.

  • So far we have two proposed models for the cost review group:

    • Described in the draft Google Doc.  4 people: designees from CC, TC, PC, and someone who actually manages AWS stuff.

    • The four of us on the current group, for six months.

    • I'd like to ask everyone to see if we can come up with a third option.  I like the expediency of #2 but as @Mark Veksler joked it's hard to get rid of responsibilities on this project once you take them on (witness @peter-murray still doing AWS work all these years!).  Personally I volunteered to lead the subgroup in getting our process deliverables done, which was TC's intent.

    • I am fine with a smaller group than what I initially proposed, but personally I do think there must be some CC designee since each decision has significant budget impact (AWS as a whole is projected to be more than 50% of total community budget).  Unless CC themselves disagrees when we present this to them!  To @Mark Veksler’s point about RACI, I could see some of the other stakeholders (TC, PC) potentially being accountable / consulted / informed, but not actually being the decision makers.

This week:

  • Peter: three councils represented.  Kitfox makes four.  "Invited" to participate from each council.
  • New QA coordinator stepping in, Yogesh Kumar.  Responsibilities were distributed among several folks.  He will work with Kitfox to support development pipelines, integrating tests.  Running tests before PRs are merged.  Yogesh may replace Mark in this role – he would be the Kitfox representative.  He is a Kitfox PO.
  • Consensus to proceed.

Start Planning D2

D2. Define a process for reviewing existing tools and environments for candidates to be shut down (e.g. when a team leaves the project or the env is no longer needed)

  • Each new flower release, new teams are involved with different projects.  Group needs to be aware of teams involved in each flower release.
    • i.e. planning for Q release has already started, months ahead of Poppy being done.
  • What if duration has exceeded original intent?
    • Dev teams look at previous request and update it during sprint planning.
    • Recommend resubmit if you will be one sprint+ over.
  • Two weeks before X release, look at temas who will be involved in X+1 release.  CRG look at team makeup for X+1 release.
    • Message teams to see if they still need the environments.  Is the team still active in the environment?  They make take a release off, or may have disbanded.  Or maybe team still exists but doesn't need the environment anymore.
    • Request updated proposal if not already submitted.
  • Maccabee Levine draft

How do we socialize this?

  • Communicate to POs in their bi-weekly meetings?  Sysops?