2023-10-19 Product Council Agenda and Meeting Notes

Owned by Alexis Manheim
Last updated: Oct 24, 2023

Date

Attendees


Discussion items

TimeItemWhoNotesDecisions and Actions
5 minAnnouncements

Kristin Martin 
all

SIG Reports 2023-10-19

  • SIG liaisons should check in with SIGs that seem to be not meeting or that have not provided a SIG report

Update on Quesnalia release timing

From Community Council: Please will you ask your councils to ratify the following change to our governance model concerning the CC size and quorum:

Current text:
    The council will have 15 members but can operate with a minimum of 9. The council will seek to have a diverse membership.
Proposed replacement:
    The community council will have 13 members but can operate with a minimum of 7. The council will seek to have a diverse membership.

This was approved by CC as noted in 2023-09-25 Meeting notes

Rationale is that initially a large CC was usefully representative but with 43 members that is no longer possible, it is better to have a smaller/tighter/more-engaged CC that handles necessary business effectively, works with PC and TC, and  works to support engagement from all members





DECISION: release cycle update is approved


ACTION: Change to CC governance model: no direct feedback in the meeting

  • Kristin Martin will ask PC members to vote on change of CC governance model
25 min

Current state of FOLIO Quality Assurance testing

(link to ongoing discussion page)

 Yogesh Kumar Oleksii Petrenko Khalilah Gambrell 

Presentation by Yogesh of testing process with all its components

  • testing happens during development - those are manual and automated tests
  • there are manual test cases written by manual QA tster and POs
  • POs review test cases and are stored in testrail repository
  • In Sprint testing: unit tests, integration tests, daily e2e tests
  • Release testing: Bugfest → repeat all tests for release ; consists of new features and regression testing
  • timline: 2 week sprints
  • manual test results: during sprints new features are tested and impacted test cases due to functional changes
  • bugs are filed for failures; bugs go to triage process
  • 904 automated test cases run daily
  • ~ 9-10 bugs are found daily
  • test cases in testrail are categorized: Smoke, Critical Path and Extended Path tests
  • tests are repeated as part of the release cycle - by MQA and Community testers
  • test cases are at least executed once until release
  • number of automated test cases is growing (e.g. Critical Path tests currently 50% manual, 50% automated)
  • more stats in Appendix slides
  • Kristin: what are the definitions for the different test types
  • Smoke = basic test, e.g. is the data correct
    • Defines most critical functions related to the feature availability or stability.  
  • Critical Path = Test case covers critical data set that functionality should correctly process and returns an expected response. 
  • Extended Path = Test case that covers other scenarios that make no impact on the feature's work or edge cases.
  • definitions from: /wiki/spaces/DQA/pages/2661490
  • Tod is there any plan of bringing community test to an earlier stage in the process? Maybe in the  iddle in the release cycle?
    • Khalilah: Yes, UATs should happen earlier, are organised by POs
    • but there are some challenges; will be topic at an upcoming PO meeting
  • Jenn: it is more for the POs and more for the SMEs
    • looking at testrail runs: is there effort in decreasing the "noise" in bugfest? Would be good to get bugfest more focused as the automation increases
    • Yogesh: this might be an error, please ping Yogesh if you see something like this
  • Jenn: SMEs could do more UATs;
  • Khalilah in chat: Also sometimes we have trouble getting UAT participation.
  • Jenn in chat:  As a sporadically participating SME, I know! This might be another conversation to have about why with the SMEs
  • Kristin in chat: I think the Acq SIG has had a lot of success with UAT testing.
  • Jenn in chat:  UAT becoming more prominent than bugfest seems like it would be great
  • Yogesh: all test cases in testrail are manual tests, the aim is to bring the number down

Presentation by Oleksii on timeline

  • Timeline includes: Development freeze, Module release, Bugfest, Bugfix, General announcement
  • goal: up to dev freeze everything is finished
  • module release is to test dependencies and whether modules are comaptible with next release; release notes should be updated
  • bugfest: bugfest environment prep and test case execution; collecting feedback
  • bugfix. fixing issues, final smoke test, final update of release notes
  • general announcement of release via Slack
  • Brooks Travis will post something to the PC slack channel later today, regarding how to follow-up on the testing/release presentation/discussion.
20 minFollow up on Re-Architecting Impact on the Product Council report
Jesse Koennecke 
  • There was a discussion on the proposal in Tri-Council meeting on  
  • Question to decide on as PC: should PC endorse the vision to move ahead with the model and the answering the questions raised
  • document lists key problems to solve: e.g. definition of product
  • key overarching objectives to suuport from the doc:
    • Ensuring that FOLIO library functionality is fit for purpose and performant
    • Ensuring that FOLIO library functionality is a good fit for its target market(s)
    • Ensuring that there is a clear roadmap for FOLIO library functionality
  • The subgroup recommends 

    that PC endorses this approach and to progress to at least the next stage where the follwoing questions should be answered:

    • There should be more stakeholder engagement
    • Questions raised by the PC and TC should be addressed
    • Terminology is revisited and preferably terms other than Platform and Application used
    • A clear process and timeline need to be defined including:
      • Reaching agreement on the best path forwards
      • Tasks and Timeline
      • Tasks and Resourcing
  • Alexis: we should be clear about what should happen with the progression
  • define which parts need to be clarified 
  • Brooks in chat: I think it’s more “we think this is a good idea”
    With these caveats
  • Kristin: PC recognizes that we have a problem with the way flower releases are happening = part 1
  • part 2: we agree that the idea of formalizing applications and plattforms could help
  • how can we continue to stay engaged in this shift
  • Martina S.: formalizing applications and formalizing platforms should be evaluated as separate ideas; they have been separated out in TC paper as well as in PC paper
    • our plan to have a group of stakeholders working with Vince and Craig: there should be a clear call for members with a deadline
  • Marc in chat: “This is a good idea, let’s explore it more” is a very different framing to “yes, we want applications and platforms to be implemented (and we expect it to solve the challenge with flower releases)”
    The proposals stack upon each other, platforms are compositions of applications, thus they only make sense with applications
    There is a thread in the Tri-council channel for signing up to the cross council group
  • Charlotte: we should be clear that it is not one size fits all; we have different situations 
  • Jenn in chat: Charlotte’s point is very important, imo, given the ‘modules can belong to only one application’ condition
  • Tod: when initially presented, it did not seem the ideas were necessarily linked, rather 2 steps

No PC endorsement today

Decision is postponed

The Tri-Council group is the place to iron out where we are not sure

  • Alexis Manheim will follow up on slack to identify a PC representative(s) for the Tri-Council re-architecting working group
5 minFuture topics


Chat

16:04:49 From Index Data To Everyone:
    Yogesh - will you share a link to the slidedeck?
16:18:41 From Charlotte Whitt To Everyone:
    Link to definitions on the wiki: https://folio-org.atlassian.net/wiki/display/DQA/Test+case+group+definition
16:19:03 From Oleksii Petrenko To Everyone:
    Definition of Smoke/Critical_Path/Extended - https://folio-org.atlassian.net/wiki/display/DQA/FOLIO+test+cases+guideline#FOLIOtestcasesguideline-4Testcasescreation
16:19:18 From Charlotte Whitt To Everyone:
    Reacted to "Definition of Smoke/..." with 👍🏻
16:21:53 From Tod Olson To Everyone:
    Thanks, this was helpful.
16:23:08 From Khalilah Gambrell (EBSCO) To Everyone:
    Also sometimes we have trouble getting UAT participation.
16:23:15 From Jenn Colt To Everyone:
    Reacted to "Also sometimes we ha..." with ➕
16:23:20 From Tod Olson To Everyone:
    Reacted to "Also sometimes we ha..." with ➕
16:23:55 From Jenn Colt To Everyone:
    Replying to "Also sometimes we ha..."
    
    As a sporadically participating SME, I know! This might be another conversation to have about why with the SMEs
16:24:19 From Charlotte Whitt To Everyone:
    Reacted to "As a sporadically pa..." with 💯
16:24:49 From Kristin Martin (UChicago she/her) To Everyone:
    Replying to "Also sometimes we ha..."
    
    I think the Acq SIG has had a lot of success with UAT testing.
16:25:08 From Martina Schildt | VZG To Everyone:
    Reacted to "I think the Acq SIG ..." with 💯
16:25:22 From Khalilah Gambrell (EBSCO) To Everyone:
    Reacted to "I think the Acq SIG ..." with 💯
16:33:58 From Jenn Colt To Everyone:
    UAT becoming more prominent than bugfest seems like it would be great
16:34:10 From Tod Olson To Everyone:
    Thank you! This was great!
16:35:55 From Brooks Travis To Everyone:
    I’ll post something to the PC slack channel later today, regarding how to follow-up on the testing/release presentation/discussion.
16:40:31 From Jesse Koennecke (he/him) To Everyone:
    https://docs.google.com/document/d/1yxF-D47tpTmiusJYn8XtWbFKnk2ZapSvxH2kxEmE-F8/edit?usp=sharing
16:41:03 From Alexis Manheim To Everyone:
    last week
16:51:42 From Brooks Travis To Everyone:
    I think it’s more “we think this is a good idea”
16:51:58 From Brooks Travis To Everyone:
    With these caveats
16:55:15 From Marc Johnson To Everyone:
    “This is a good idea, let’s explore it more” is a very different framing to “yes, we want applications and platforms to be implemented (and we expect it to solve the challenge with flower releases)”
16:56:28 From Marc Johnson To Everyone:
    The proposals stack upon each other, platforms are compositions of applications, thus they only make sense with applications
16:56:56 From Marc Johnson To Everyone:
    There is a thread in the Tri-council channel for signing up to the cross council group
16:58:04 From Jenn Colt To Everyone:
    Charlotte’s point is very important, imo, given the ‘modules can belong to only one application’ condition
16:58:14 From Marc Johnson To Everyone:
    Reacted to "Charlotte’s point is…" with 💯
16:58:24 From Charlotte Whitt To Everyone:
    Reacted to "Charlotte’s point is..." with 🙏🏻
16:58:48 From Martina Schildt | VZG To Everyone:
    Reacted to "Charlotte’s point is..." with 💯

Action items

  •