EPAM-EBSCO Team Leads' Forum
- Lee Braginsky
- Ostap Voitsekhovskyi
- Matt Weaver
Permanent Forum Members:
| Name | Team |
|---|---|
| Firebird | |
| Volaris | |
| Folijet | |
| Vega | |
| Thunderjet | |
| Spitfire | |
| Citation | |
| Corsair | |
| PTF | |
| Kitfox | |
| Mriya (AQA) | |
| QA Manager | |
| Roman_Fedynyshyn | PTF |
| Oleksii Kuzminov | Eureka |
| Craig McNally | Eureka technical PO |
Meeting Minutes
Meeting December 10
- Tjet exploring remote debugging on Rancher; building local UI; starting experimenting with Telepresense
- Citation 16GB is not enough
- Volaris problems building and running UI modules; trying to figure out problems with running DCB module
- Folijet no progress, using deploying to Rancher
- Firebird. no progress
- Corsair 16 GB limitation as well
- Spitfire no need for tools
- Vega no progress
Meeting December 3
Attendees: PK, Sasha K, Gurleen, Matt, Ostap, Serhii, Boburbek Kadirkhodjaevlava, Kate, Yogesh, Pavlo, Craig McNally, Martin, Yogesh
Topics: Local development on Eureka platform. Teams' readiness to develop for Eureka and on Eureka
Quick notes:
Several Teams have tried the Golang tool and there are the results:
- Tjet (the tool author) can run the Acquisition app locally
- Citation tried and got some container errors. PK will troubleshoot with Boburbek
- Volaris was able to run some internal modules, but not the DCB (also container errors). Gurleen or her deputee will work with Boburbek
- Folijet tried to deploy the Import app locally, but seemingly ran into memory issues. Kate is going to report more results shortly
- Firebird. Slava intends to try the Bash tool in the next sprint
- Corsair is using Rancher to deal with Eureka issues for now, but going to try the Go tool in the next sprint
- Spitfire. Pavlo stated that Spitfire is mostly focusing on developing microservices independent on other modules. When they need to test integration they are doing well with Rancher, so they do not need local Eureka env.
- Vega is still struggling with the Ramsons and Ramsons+ scope. Too soon to even dream about Eureka
In general, all team leads understand that in Sunflower we will be developing features exclusively on the Eureka platform, and we will touch Okapi only support production (which I'm afraid we're going to do quite a lot judging by the frenzy in which we're wrapping up Ramsons).
Please remember that the team leads are responsible for creating the most favorable development conditions for your teams. If your team chooses to continue using Rancher as Spitfire does - I respect that, however please do not complain that developing and troubleshooting on Eureka is slow and painful.
Meeting November 26
Attendees: PK, Sasha K, Sasha H, Gurleen, Matt, Ostap, Serhii, Boburbek Kadirkhodjaevlava, Kate, Yogesh, Olamide, Pavlo, Oleksii Petrenko, Oleksii Kuzminov, Craig McNally, Steve Ellis
Topics: Local development on Eureka platform and tooling for that. In the meeting, we discussed the Go tool developed by Boburbek Kadirkhodjaev and Serhii Nosko
Quick notes:
We saw a very impressive presentation of the Go tool. The tool appears to be ready for prime time, but we are all anxious to hear from the users of the tool about their experience working with the tool. The Eureka team is going to record their tool demo later. Also, I scheduled another leads' meeting next week, 12/3, and I invite Eureka to present their solution then.
Another very important comment: Teams, please do not dump issues in Eureka's lap automatically when something works on Okapi but fails on Eureka. Please do some troubleshooting first. Please remember we're in it together. This is a link to the Eureka troubleshooting guide: https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/628916278/How+to+triage+permissions+or+403+401+problems+on+Eureka+envs
Comments:
- The ECS module is not loading automatically, so it needs to loaded manually (should not be a big deal, I'm hearing)
- From oleksandr_haimanov: can we figure out how to not load some standard components locally, like Kafka and postgres, to reclaim some memory space? See this tool https://www.telepresence.io/#gsc.tab=0
Team's readiness to start using the Go tool.
Teams ready to get started on the Go tool: Volaris, Thunderjet, Citation, Corsair
Teams ready to get started on the Bash Bootstrap tool: Firebird
Teams not ready to get started on Eureka work: Folijet, Vega
Teams who do not think any tool is necessary for local development on Eureka: Spitfire (Pavlo Smahin: I'm looking forward to hearing about their way of developing applications on the Eureka platform)
Links
Meeting Recording:
Serhii/Boburbek presentation
Eureka's guide to troubleshooting issues
Meeting October 29
Attendees: PK, Sasha K, Sasha H, Gurleen, Matt, Serhii, Boburbek Kadirkhodjaevlava, Kate, Yogesh
Topic: 1) Discuss Github Copilot tool to improve teams' productivity. Use cases considered included generating unit and Karate tests, code refactoring, unfamiliar code understanding (Python). Hoping to see a live demo next time
2) Discuss LoC support model. How do we allow developers to download LoC logs? How do we mask sensitive data pieces in the application logs?
Meeting October 8
Attendees: Ostap, PK, Sasha K, Gurleen, Matt, Pavlo, Ostap, Serhii, Boburbek Kadirkhodjaev
Topic: Oleksii spoke about module resource management
Presenter: Oleksii Petrenko
Meeting September 24
Attendees: Ostap, PK, Sasha K, Gurleen, Matt, Pavlo, Ostap, Serhii, Boburbek Kadirkhodjaev
Topic: Serhii and Boburbek present the tool developed by TJet to set up a Eureka environment locally along with remote troubleshooting
Presenter: Serhii Nosko, Boburbek
Meeting August 20, 2024
Attendees: Ostap, PK, Sasha K, Gurleen, Yogesh, Slava, Matt, Pavlo, Ostap, Serhii, Denis Kovtun
Topic: Denis Kovtun's talk about production support, issues in Rally, RRT
Presenter: Denis K
Other materials
RRT checklist for troubleshooting
Yogesh's topic Change Logs
Nightly Karate and Cypress run Change logs
Quick Note:
Please think about how to reduce the number of bugs that end up in Prod (or Bugfest). Improving logs was one suggestions. Resurrecting the RCA program to improve the code area where the issues occur was another.
Also, please monitor the daily Karate and Cypress runs on the QualityGatesCI Pipeline and address issues immediately. The Change log address is above.
Meeting August 6, 2024
Attendees: Ostap, Matt, Yogesh, PK, Sasha K, Gurleen, Kate, Sasha H, Martin, Mikita
Topic: Gitlab branching strategy
Presenter: Mikita Siadykh
Quick Note: This is coming hopefully this Fall. Looking forward!
Meeting July 23, 2024
Attendees: Ostap, Matt, Yogesh, PK, Sasha K, Serhii, Gurleen, Kate, Slava, Pavlo, Oleksii Petrenko and Tetiana Gusar
Topic: Reporting Portal and e2e test maintenance by FE engineers
Presenter: Oleksii, Tetiana
Quick Note
Reporting Portal link: https://report-portal.ci.folio.org/ui/. Please use your Github creds to log in
Leads - since now we have our Quality Gates pipeline we need to carefully monitor the e2e failure (as well as Karate failures). Please use the reporting portal to see the failures. Please make sure your FE engineers know what to do.
Some action items from the meeting (as per chatgpt )
Tatiana Gusar:
- Provide additional training for FE engineers on Cypress test maintenance
- Ensure Scrum Masters are involved in the troubleshooting process.
- Communicate with John C and Zach about monitoring Stripes e2e failures
All Team Members:
- Actively participate in the troubleshooting process.
- Utilize the report portal for tracking and resolving test failures.
Lee Braginsky/Yogesh Kumar/Oleksii:
- Monitor the engagement and progress of the team in reducing bugs.
- Ensure that all dependencies are addressed and team members are informed.
Meeting July 9, 2024
Attendees: Ostap, Matt, Yogesh, PK, Roman, Sasha K, Serhii, Gurleen, Kate, Slava,
Topic: general discussion on QualityGates, Karate failures and the priority of fixing issues
Meeting June 25, 2024
Attendees: Ostap, Matt, Yogesh, PK, Roman, Sasha K, Serhii, Gurleen, Kate, Slava,
Topic:
- Discussed various topics including setting up developing envs using BusyBee vs. Vagrant.
- We are ready to include Gatling reports into QualityGateCI pipeline
Presenter: Serhii Nosko
Quick Note
We are going to include Gatling report into QualityGatesCI pipeline. Yogesh is going to create ticket for Kitfox. Please continue identifying Karate tests to be included into Gatling report
BusyBee seems be a good reliable way to quickly create Eureka local development environment.
Meeting June 11, 2024
Attendees: Ostap, Matt, Yogesh, PK, Roman, Sasha K, Serhii, Gurleen, Sasha H, Slava,
Topic:
Presenter: all
Quick Note:
Serhii Nosko has a script to quickly and reliably set up development env using BusyBee
The Quality Gate CI Pipeline is ready. https://jenkins-aws.indexdata.com/job/folioRancher/job/folioQualityGates/job/folioCiQualityGates/
Next Time: Continue the quality topic
Meeting May 28, 2024
Attendees: Ostap, Matt, Yogesh, PK, Roman, Sasha K, Serhii, Gurleen
Topic:
- Gurleen presented her "quick wins" with Gatling report. Thank you Gurleen and Team Volaris
- Developers' maintaining e2e tests. Tjet is doing it (I think Fjet as well, but Kate wasn't in the meeting to confirm). Leads - please work with your FE developers and have them work with Ostap to come up to speed on e2e maintenance.
Presenter: all
Quick Note: We are hearing that often POs prioritize tickets related to "quality work", such as Karate tests, Gatling performance report, or e2e maintenance lower than functional features. Dear Captains, this is where you need to negotiate with your POs and explain that this type of work makes the business features more robust. I mean, at this point, nobody questions the need to implement unit tests or integration tests, right? Gatling and e2e maintenance should become as natural as unit-testing...
BTW, the Quality Gate CI Pipeline is ready. https://jenkins-aws.indexdata.com/job/folioRancher/job/folioQualityGates/job/folioCiQualityGates/
Next Time: Continue the quality topic
Meeting May 14, 2024
Attendees: Ostap, Kate, Yogesh, PK, Martin, Sasha K, Serhii, Sasha H
Topic:
- Discuss progress with Karate/Gatling tests and reports
- Developers' maintaining e2e tests
Presenter: all
Quick Note: Sasha K and Serhii stated that teams Vega and Thunderjet finally have stably green Gatling reports. Thank you! Who's next please?
Still not many teams have their FE developers maintain e2e tests. Folijet seems to be the only exception where FE developers are up to speed on Cypress. Success stories on the subject are expected next time.
Next Time: TBD
Meeting April 30, 2024
Attendees: Ostap, Kate, Yogesh Matt, PK, Martin, Sasha K, Gurleen, Sasha H
Topic:
- Discuss progress with Karate/Gatling tests and reports
- Discuss how teams are reacting to newly found bugs
Presenter: all
Quick Note: Sasha/Gurleen mentioned that Gatling is not running consistently, we need to look into it deeper. More - next time
WRT bugs, I'm hearing that in a development cycle, while newly found P1s are usually addressed immediately, the P2s are often placed on the backlog and addressed later (sometimes MUCH later - per PO's prioritization). Which means that we often complete stories with P2s. I personally don't like it, as I think this leads to more bugs before BF and during BF. Let's discuss further
A word of wisdom from Martin:
For every bug we have to ask 5 "why's". Why did the bug happen? Let's say it's coding (i.e, not due to requirements), why coding? If we didn't know the impact that the new implementation causing on other parts of the code, why didn't we know? Because we didn't have an automated test. Why didn't we have an automated test for other parts of the code? Because we didn't have time to write tests. Why didn't we have time to write tests? Because we don't have skills or because the code is not easily testable, or because we don't have time to skip features... Keep asking why will eventually get to the root cause, not just a superficial cause. Every developer should be asking this question and help the team improve. In this case if we don't have enough skills, work with management to get the skills. If the code is not testable, have a story and work with the architect to redesign the code. If no time, work with PO next time on priorities.
Next time: 1) Summarize our progress with Gatling. 2) Continue discussion of how to reduce the number of P1/P2 defects in newly developed features
Meeting April 16, 2024
Attendees: Ostap, Kate, Matt, PK, Martin, Olamide, Sasha K, Slava, Gurleen, Sasha H
Topic: Continue with Karate tests. How do we address Karate failures immediately (as opposed to schedule bug fixes)
Presenter: all
Quick Note: According to the Leads, teams almost always look into Karate failures pretty much immediately and address them right away if the fix is expected to be a quick one. However, if the fix is not expected to be quick (maybe a day or two) then it's up to POs to prioritize and schedule the fix. We want to be more assertive in addressing Karate failures. Lee is going to say a few words on this subject at Mark V's FOLIO status meeting on Thursday
Meeting April 2, 2024
Attendees: Ostap Voitsekhovskyistap, Kate, Matt, PK, Yogesh, Martin, Olamide, Sasha K, Gurleen, Serhii,
Topic: Continue with Karate tests. Using Karate tests for performance measuring.
Presenter: Olamide Kolawole
Quick Note from Olamide: let's get started!
We need each team lead to implement a scenario or more in one maven module within each team lead's ownership. You can reuse existing scenarios or create new one. Let's target next week tuesday(April 9th) to have this done so that we can have further conversations regarding its execution environment, test strategy, report collation, inclusion in continuous integration as quality gate. This first step's purpose is to gain a shared understanding. If you need any help or support please don't hesitate to reach out to me.
Meeting March 19, 2024
Attendees: Ostap Voitsekhovskyi, Kate, Matt, PK, Yogesh, Martin, Olamide, Sasha K, Gurleen, Serhii,
Topic: Continue with Karate tests. Using Karate tests for performance measuring
Presenter: Olamide Kolawole
Quick Note:
- Attain 100% pass rate. Teams will work to resolve flaky tests. Continuously erring tests can be excluded. We must be prepared to compromise.
- After reaching a stable "100%" pass rate, teams have committed to hop on test failures for immediate resolution. This can have an impact on sprint commitments.
- After reaching a stable "100%" pass rate for a flower release, a branch will be cut in the karate tests repo to represent the test suite that verifies the flower release.
Link to presentation - none
Meeting March 5, 2024
Attendees: Ostap, Yogesh, Sasha K, Gurleen, Serhii, PK, Pavlo, Kate, Matt, Lee, Sasha H, Olamide, Slava
Topic: Excellent presentation by Olamide on Karate testing strategies. This topic will continue next time as many participants had questions
Presenter: Olamide Kolawole
Next time: we will continue with the Karate testing discussion
Meeting February 20, 2024
Attendees: Ostap, Yogesh, Sasha K, Gurleen, Serhii, PK, Pavlo, Kate, Matt, Lee
Topic: Yogesh Kumar is presenting the Shift-left program and its results so far
Quick notes: Yogesh reminded us why shift-left is important for early feedback on Quality.
Presenter: Yogesh emphasized the importance of shift-left for early quality feedback. He also provided an update on our shift-left journey.
Meeting January 19, 2024
Attendees: Ostap, Yogesh, Sasha H, Sasha K, Gurleen, Serhii, PK, Matt, Lee
Topic: Ostap Voitsekhovskyi is presenting automated testing and Team Mriya
Quick notes: Ostap went through the "theory" of quality automation and why it's so important to us, as well as the technique of creating Cypress tests. I think we should continue this discussion next time and proceed to leads' questions.
Presenter: Ostap Voitsekhovskyi
Meeting December 19, 2023
Attendees Sasha K, Serhii, Kate, Pavlo, Matt, Gurleen, Lee, Sasha H (now part of the forum), Yogesh, Oleksii P
Topic: Sasha H presentation on how to use the Unified Pipeline.
Quick notes: The Unified Pipeline is a set of jenkins jobs to create rancher environments for teams. Can the leads/team members use it so the teams take care of their immediate devops needs and free up team Kitfox to focus on strategic automation initiatives?
Team leads are also encouraged to come up with additional requests for Kitfox.
Presenter: oleksandr_haimanov
Link to Presentation (documentation)
Meeting December 5, 2023
Attendees Sasha, Serhii, Kate, PK, Matt, Slava, Lee
Topic: Linked Data and Bibframe
Quick notes: Team Citation is working on adoption of the concept of Linked Data Bibframe for FOLIO.
Presenter: Punnoose Kutty Jacob Pullolickal
Next time: We will continue our conversation about LinkedData and FOLIO. Please prepare questions for Punnoose Kutty Jacob Pullolickal
Meeting November 7, 2023
Attendees Sasha K, Serhii, PK, Matt, Gurleen, Slava, Lee
Topic: DCB (Direct Consortia Borrower)
Quick notes: The DCB is K int's product, and the main challenge is to integrate this product with FOLIO. Team Volaris is releasing the initial version of DCB-FOLIO integration today! (11/7/23)
Presenter: Gurleen Kaur1
Next time: LinkedData and FOLIO presented by Punnoose Kutty Jacob Pullolickal
Meeting October 24, 2023
Attendees + Guests: Mark V, Maccabee Levin, Peter Murray, Sasha K, Kate, Serhii, PK, Matt, Gurleen, Pavlo, Slava, Lee
Topic: Controlling AWS hosting costs (subgroup of Tech Council)
Presenter: Maccabee Levine
Quick note: Starting from Q-release, new development environment requests for community projects have to go through the approval process by the Tech Council. This is done so that we can project the costs of AWS hosting. All such requests must be handled through Jira tickets assigned to Jira project COSTS
useful links:
TC AWS hosting costs Subgroup: https://folio-org.atlassian.net/wiki/display/TC/Controlling+AWS+Hosting+Costs+subgroup
Process of creating Jira tickets for requesting an justifying new environments: https://folio-org.atlassian.net/wiki/display/TC/AWS+Environment+Lifecycle
Meeting October 10, 2023
Attendees: Matt, PK, Kate, Slava, Gurleen, Sasha K, Serhii, Lee
Topic: Matt's presentation on FQM and ListApp
Presenter: Matt Weaver
Next topics (not in priority order).
- More on shift-left
- DCB overview and challenges (Gurleen: 7th Nov)
- COMPLETED: FQM overview and challenges, and the process of TCR
- LinkedData overview and challenges
- Onboarding external teams
- Review the RCA program
Meeting September 26, 2023
Attendees: Matt, PK, Kate, Gurleen, Sasha K, Serhii, Lee
Topic: Leads Forum Format pseudo retrospective: how do we make the forum more effective
Presenter: all
Overall Impression:
The forum has been useful!
Thoughts:
PK
- Like the idea of sharing best practices, both technical and process related
Sasha K
- Leave more space for interactions – i.e. never enough room for discussions after presentations.
Serhii
- Presentation prep time is well-spent. You have the opportunity to rethink the previous decisions when you’re putting things on paper
Matt
- Tech topics useful, learning a lot
- Consortia support information is very valuable for FQM
Kate
- Definitely keep it technical
- Challenges: RRT support; Rapid response team is not that “rapid”.
- RCA – how helpful is it?
- Tech debt feature in each team. Maybe make it more specifics?
Gurleen
- Gained useful knowledge from this forum.
- DCB has been challenging
Decisions:
Continue with presentation, but have more time for discussion after presentations: Going forward we’ll start at 7:45 AM EST
Next topics:
- More on shift-left
- DCB overview and challenges
- FQM overview and challenges, and the process of TCR
- LinkedData overview and challenges
- Onboarding external teams
- Review the RCA program
Please suggest more
Meeting September 5, 2023
Attendees: Matt, PK, Slava, Kate, Sasha, Serhii, Lee
Topic: ECS (Extended Consortium Support) continued. Applause and big thanks to Serhii_Nosko !
Presenter: Serhii_Nosko
Link to session recording (new)
Link to ECS doc on Wiki (same as previously)
Link to Serhii's presentation (same as previously)
Next time: Let's discuss the format of our forum: what's working, what's not working, how to improve the usefulness, etc
Potential next topics:
1) How's shift-left working (all)
2) FQM and List App (Matt Weaver )
3) DCB and Reshare (Gurleen Kaur1 )
4) Linked data and Bibframe (Punnoose Kutty Jacob Pullolickal )
5) please propose other topics
Meeting August 22, 2023
Attendees: Gurleen, Matt, PK, Slava, Kate, Sasha, Pavlo, Serhii, Lee
Topic: ECS (Extended Consortium Support)
Presenter: Serhii_Nosko
Next time: Continue with the Consortium functionality
Meeting August 8, 2023
Attendees: Gurleen, Matt, PK, Slava, Kate, Sasha, Serhii, Lee
Topic: Shift-left Testing diagram within a sprint
Presenter: Alexander Kurash
Next time: Discuss Consortium functionality
Meeting July 28, 2023
Attendees: Gurleen, Matt, Pavlo, Sasha, Serhii, Lee
Topic: "What Went Wrong...". This is a discussion of development efforts that should have gone better, and what conclusions can be drawn from those efforts
Presenter: Serhii_Nosko
Next time: We will continue discussing Serhii's prez, and we'll focus on TESTING, especially automated testing, especially automated testing of asynch architectures
Meeting July 11, 2023
Attendees: Gurleen, Matt, PK, Pavlo, Kate, Sasha, Serhii, Lee
Topic: "What Went Wrong...". This is a discussion of development efforts that should have gone better, and what conclusions can be drawn from those efforts
Presenter: Kate Senchenko