EPAM-EBSCO Team Leads' Forum
Permanent Forum Members:
Name | Team |
|---|---|
@Siarhei Charniak | Firebird |
@Pavel Filippov | Volaris |
@Kateryna Senchenko | Folijet |
@Alexander Kurash | Vega |
@Serhii_Nosko | Thunderjet |
@Pavlo Smahin | Spitfire |
@Pk Jacob Pullolickal | Citation |
@Matt Weaver | Corsair |
@Martin Tran | PTF |
@oleksandr_haimanov | Kitfox |
@Ostap Voitsekhovskyi | Mriya (AQA) |
@Yogesh Kumar | QA Manager |
@Roman_Fedynyshyn | PTF |
@Oleksii Kuzminov | Eureka |
@Craig McNally | Eureka technical PO |
Meeting Minutes
Meeting Sep 30, 2025
Attendees: PK, Matt, Serhii N, Sasha H, Pavel F, Siarhei C, Ostap, Yogesh
Presentation by PK Jacob re AI MCP Server. Can FOLIO use a NLM input?
Another link from Sasha H: https://ref.tools/
Here’s a simple architecture sketch and flow showing an MCP server between FOLIO and an AI assistant.
┌──────────────────────────────┐
│ AI Assistant │
│ (Chat UI / LLM client app) │
└──────────────┬───────────────┘
│ (natural language)
▼
┌───────────────┐
│ LLM (any) │
│ (Claude/GPT/…)│
└───────┬───────┘
│ (MCP client)
▼
┌──────────────────────┐
│ MCP Server │ ⇦ “FOLIO AI” module
│ (tool registry + │
│ auth + schemas) │
└─┬───────────┬────────┘
│ │ (tool calls: structured JSON)
┌──────▼──────┐ ┌──▼───────────┐
│ FOLIO Tools │ │ FOLIO Tools │ …additional tools
│ Read APIs │ │ Write APIs │
│ (search, │ │ (create, │
│ get by id) │ │ update, del) │
└─────┬───────┘ └──────┬────────┘
│ (HTTP/RPC) │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ FOLIO Module │ │ FOLIO Module │ …(Inventory, Users, Orders, etc.)
│ Inventory │ │ Users │
└──────────────┘ └──────────────┘
[Shared services: RBAC/permissions • Audit log • Rate limits • Secrets]
How it works (1–2–3)
User asks in plain English: “Update user 123’s email to …”
LLM plans the steps and calls MCP tools (e.g.,
folio.users.updateEmail) via the MCP client.MCP server validates auth/permissions, executes against FOLIO APIs, returns structured results; LLM replies to the user.
What lives in the MCP server (“FOLIO AI”)
Tool registry: declarative list of allowed operations with JSON schemas and examples.
Policy guardrails: RBAC check, scope rules (read/update), dry-run mode, approvals for sensitive ops.
Audit & observability: full traces of prompts, tool calls, inputs/outputs.
Provider adapter: pick/swap LLMs without changing FOLIO tools.
Meeting Sep 16, 2025
Attendees: PK, Matt, Serhii N, Sasha K, Sasha H, Yogesh, Ostap V
Topic: Use Copilot to generate Karate tests and conduct self-code review.
Presentation by @Serhii Nosko
Meeting August 19, 2025
Attendees: PK, Matt, Kate, Serhii N, Pavel F, Yogesh, Slava, Sasha K, Boburbek, Eldiar, Oleksii K, Martin T
Topics:
Create environments, local and Rancher, to run multiple consortia in one cluster
Presentation by @Serhii Nosko supported by @Boburbek Kadirkhodjaev and @Eldiiar Duishenaliev .
Notes (from Copilot):
Generated by AI. Be sure to check for accuracy.
Meeting notes:
Implementation of Multi-Consortia Support: Serhii presented the new feature enabling support for multiple consortia within a single cluster, explaining its implementation, verification, and adoption by Kitfolk, with Lee and Eldiiar participating in the discussion and demonstration.
Feature Overview and Purpose: Serhii explained that the new release introduces support for adding a second consortium, allowing teams to deploy multiple consortia within the same Rancher cluster to optimize resource usage and reduce costs, particularly for teams like FSC with large and small consortia.
Declarative Configuration and UI Bundles: Serhii described the declarative approach for configuring the number of consortia and tenants via a YAML file, and detailed how different UI bundles are mapped to different ports to support multiple consortia.
Database Schema and Tenant Mapping: Serhii demonstrated how each tenant is mapped to a specific consortium in the database, with separate schemas for each consortium, and explained the storage of consortium and central tenant IDs for each tenant.
Elasticsearch Indexes per Consortium: Serhii outlined the creation of separate Elasticsearch indexes for each central tenant, ensuring that data for each consortium is managed independently, and showed how the correct central tenant ID must be passed when enabling search functionality.
User and Shared Instance Management: Serhii demonstrated that users and shared instances can only be shared within the same consortium, not across consortia, and showed how settings and roles are isolated per consortium, with Lee confirming the expected behavior through questions.
Technical Design and Internal Workflows: Serhii provided a detailed explanation of the internal technical design, including API usage, database schema decisions, and the rationale for storing consortium and central tenant IDs, with Pk Jacob asking clarifying questions about data storage and dummy records.
User Tenant API and Data Flow: Serhii described the use of the user tenant API, which provides central tenant and consortium IDs for each tenant by extracting the tenant ID from headers and querying the appropriate database schema, ensuring correct identification for each tenant.
Consortium ID Parameterization: Serhii explained that the system was designed from the beginning to accept a consortium ID parameter, which allowed for the addition of multi-consortia support without major refactoring, as most endpoints already handled this parameter.
Dummy Records in Database: In response to Pk Jacob's question, Serhii clarified that dummy records are used in the mod users table to store central tenant and consortium IDs, facilitating integration and minimizing dependencies between modules, and supporting features like password recovery.
Central Tenant and Member Tenant Associations: Serhii detailed that the central tenant stores associations for all users, which is necessary for certain functionalities, while member tenants use dummy records to indicate their central tenant and consortium associations.
Rancher Integration and Documentation Updates: Serhii and Eldiiar discussed the integration of multi-consortia support into Rancher environments, the implementation of related jobs, and the updating of documentation, with Lee requesting documentation links and Eldiiar confirming the work.
Rancher Environment Support: Serhii explained that the new functionality is now available in Rancher environments, allowing teams to add a second consortium either during namespace creation or to existing Rancher setups, with jobs provided to facilitate this process.
Documentation and Slack Integration: Serhii mentioned that documentation has been updated to reflect the new features, and that agent bots in Slack can be used to guide users through the process of adding consortia in Rancher.
Collaboration and Implementation: Serhii credited Eldiiar for implementing the Rancher integration and updating documentation, and Eldiiar confirmed that there was nothing further to add regarding the Rancher work.
Demo and Adoption by Teams: Serhii noted that a demonstration was conducted for the FSC team, and that other teams are beginning to set up their environments using the new multi-consortia support.
Tooling Support and Acknowledgments: Serhii acknowledged Bobur for maintaining the Erika setup tool, which supports local multi-consortia deployments, and highlighted the tool's ability to run multiple consortia from a single command, with Lee and Eldiiar commenting on Bobur's role.
Erika Tool Maintenance: Serhii thanked Bobur for supporting and updating the Erika setup tool to accommodate changes in contracts and enable local deployment of multiple consortia, making it easier for teams to test and develop locally.
Follow-up tasks:
Documentation Sharing: Paste the updated Rancher documentation links into the Teams chat window for team access. (Serhii)
Documentation links:
https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/865730596/How+to+recreate+Eureka+Rancher+environment
https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/699498501/Create+Eureka+environment
https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/508690454/Namespace+useful+info+tools
https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/1123254306/Add+second+consortium
Automated performance testing during feature development
Martin raised the issue that performance degradations are often discovered late in the cycle by his team, creating significant disruption and urgent rework for development teams. To address this, we are setting a goal for FY2026 to detect performance degradation early in the development cycle. Achieving this will reduce downstream bottlenecks, prevent last-minute fire drills, and improve both product quality and delivery efficiency.
Previously, we discussed the use of the Gatling framework for performance testing, and some teams created a number of tests. Kitfox, for example, has an established Gatling pipeline:
👉 Kitfox Gatling Pipeline
Unfortunately, this practice has since been abandoned by all teams. Reviving, standardizing, and integrating such performance testing into the active development workflow will be essential to meeting our FY2026 goal.
@Viachaslau Khandramai volunteered to look into it.
Meeting July 22, 2025
Attendees: PK, Matt, Kate, Serhii N, Pavel F, Yogesh, Slava, Ostap, Sasha K
Topics:
@Ostap Voitsekhovskyi discussed Cypress test monitoring instrumentation. Leads - please make sure your team’s test, at least Smoke and Critical Path ones, run smoothly in nightly runs. Also please make sure your FE developers know how to maintain the tests and know how to use the instrumentation that AQA team have created.
Links to AQA documentation:
https://folio-org.atlassian.net/wiki/spaces/DQA/pages/722108508/Environments
https://folio-org.atlassian.net/wiki/spaces/DQA/pages/2660627/AQA+Setup+TAF+locally
https://folio-org.atlassian.net/wiki/spaces/DQA/pages/2666540/Filtering+tests+by+tags
Link to Ostap’s presentation:
Folio Team Leads' Forum-20250722_074808-Meeting Recording.mp4
Meeting June 10, 2025
Attendees: PK, Matt, Kate, Serhii N, Pavel F, Yogesh, Slava
Topics:
@Viachaslau Khandramai ‘s report on how Firebird integrates their modules, notably, Bulk Ops (and soon Data Export), with FQM. Slava notes the significant performance improvement when creating multiple streams when returning the datasets. He recommends, up to 10 streams (or threads) and up to 10,000 records in each. Slava’s presentation is here: https://open-libr-foundation.slack.com/files/U03R4UMKG0M/F090L0AGEJX/fqm_integrating.pptx
Presentation recording is here: Recap: Folio Team Leads' Forum Tuesday, June 10
We discussed as a team the importance of maintaining e2e test cases. All leads should ensure their teams are aware that these tests should not break due to changes in screens or logic. Responsibility for keeping tests up to date lies with the individual teams—not with the AQA team.
To check how your team’s test cases are running, please refer to the Allure report:
https://jenkins.ci.folio.org/job/folioScheduledTesting/job/folioQualityGates/1690/allure/You should also monitor #rancher_tests_notifications on Slack for updates.
Interesting point was brought up by @Matt Weaver who mentioned that the Gen AI agent built into IntelliJ can potentially significantly reduce the time developer spends creating unit and integration test cases, while increasing their value and quality. We’re looking forward to more info from you Matt!
Meeting May 27, 2025
Attendees: PK, Sasha K, Matt, Kate, Serhii N, Pavel F, Oleksii K, Boburbek, Yogesh
Topic: Evolution of the Eureka-cli Go tool for local troubleshooting
Serhii N presented the next incarnation of the Eureka-cli tool. Its key features:
1. Ability to deploy child applications for search, export and edge modules functionality
Add Elastic search, Kibana to list of supported containers
Add
getEdgeApiKeycommand to the tool functionalityAdd ability to use custom sidecars built locally
Add --Required option to
deployApplicationcommand to have ability to deploy only required containers to save memoryImprove Readme documentation, add different stabilizations and improvements to the tool source code
Documentation: https://github.com/folio-org/eureka-setup/tree/master/eureka-cli
Meeting April 1, April 15, 2025
Attendees: PK, Sasha H, Sasha K, Siarhei Ch, Matt, Kate, Serhii N, Craig,
Topics: pre-defined Dataset
LoC has allowed us to use their 1,000,000 sample data for testing. @Pk Jacob PullolickalK Jacob created several data sets out of the LoC sample data with different sizes. Please see this link: Test data - Sample MARC records for setting up test environments
The data can be loaded into individual envs using Data Import mechanism
Recording of PK's short presentation is here: https://ebscoind-my.sharepoint.com/personal/ykumar_corp_epnet_com/_layouts/15/stream.aspx?id=%2Fpersonal%2Fykumar%5Fcorp%5Fepnet%5Fcom%2FDocuments%2FRecordings%2FFolio%20Team%20Leads%27%20Forum%2D20240319%5F074732%2DMeeting%20Recording%2Emp4&referrer=StreamWebApp%2EWeb&referrerScenario=AddressBarCopied%2Eview%2Ef370ab13%2D24a9%2D42d6%2Da3ff%2D9ac6569c5640
Questions from @Serhii Nosko:
1. Contract with UI, is it defined in some way?
2. Did you face some issues with parallel processing, if tests running in 10 parallel threads - its not only functional testing, but also some kind of performance and stress testing?
3. Currently we have 2k tests, the plan is to have 5k tests? The plan is to automate all test cases for Bugfest?
4. If we have a new feature, when for your practice automated tests are written for it, in the next release?
5. From your practice, do we have some rounding failures, is it connected with infrastructure?
@Pk Jacob Pullolickal - this is for you
Meeting March 11, 2025
Attendees: PK, Sasha H, Sasha K, Gurleen, Matt, Yogesh, Pavlo, Slava, Kate, Serhii N, Craig, Ostap, @Oleksii Petrenko, @Eldiiar Duishenaliev
Topics: Rancher self-service, Kitfox prepared documentation
With the help of the documentation below, Development Teams should be able to take care of their own ranchers envs.
https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/840138784/Rancher+FAQ
Main points:
Kitfox Team will be reduced in size, so the development teams will need to practice "self-help" more. Kitfox will continue to improve documentation. Use the Chatbot (https://copilotstudio.microsoft.com/environments/Default-b41b72d0-4e9f-4c26-8a69-f949f367c91d/bots/cr4d9_folio/canvas?__version__=2&enableFileAttachment=true), but remember it's a POC
Kitfox is not going away so it will still provide support when needed, but, please try to minimize the need for such support
Shared envs, like etest, will continue to be managed by Kitfox
Teams should avoid deploying untested or partially tested code to shared envs. @Pavlo Smahin - if Spitfire needs an edev with a large dataset, please talk to @Matt Weaver of Corsair, as Matt's team has a rancher with large dataset
Next time - please come with your ideas how Kitfox could make the feature teams more efficient in resolving their devops issues
Meeting February 18, 2025
Attendees: PK, Sasha K, Gurleen, Matt, Yogesh, Pavlo, Slava, Kate, Serhii N
Topics:
Upgrading Java 17 to 21
Slated for SF, FOLIO-4182 Migration to Java 21 and Spring Boot 3.4
Could gain performance because of the new Virtual Threads. Also fewer containers (cost optimization)
It is possible to migrate some modules to 21 and keep others on 17
Eureka Guide
Please continue reviewing and providing comments: Eureka Developer Guide
Feature Toggles
Several teams are either doing them or already have them, namely, Tjet, Volaris, Spitfire, Vega. Citation is a special case as their FT is to run LDE outside of FOLIO and hence cannot use FOLIO tools
It appears that we do not have a single way of implementing FTs, i.e. each team is doing it their own way. We should regroup on that
It appears that the Eureka platform may have a mechanism to implement FT through its Env Vars, which would be the unified way of implementing FTs
Sensitive info
@Viachaslau Khandramai is running the show. He outlined what we mean by "sensitive". Please remember that it's only the PII, but also information containing account and invoice numbers, dollar amounts and things of this nature. Scott Macdonald: 'if it feels sensitive, treat it as such'
POs are scheduling stories for all teams in SF
Meeting January 21, 2025
Eureka Adoption
Tjet
Developing on Eureka (except Karate tests)
Use Telepresence
Karate tests still use Okapi
Single person can debug – deal with it by intrateam communication
Vega
Continue on Okapi and deploy to Eureka environment
Lots of business func that works for both
Eureka is much slower than Okapi
Citation
Develop on Eureka
Use Telepresence
Keep 1 Okapi env for Karate tests
Volaris
Develop on Eureka, use Telepresence
Keep 1 Okapi environment for Karate, DCB and Inreach
Inreach is NOT moving to Eureka???
Corsair
Still mostly use Okapi but have Eureka env with Telepresence
Primary env is still Okapi, but planning to replace to Eureka in about a week
Firebird
Development work on Eureka
Telepresence
1 Okapi env for Karate test
Folijet and Spitfire were unrepresented
Meeting January 7, 2025
Happy New Year all!
Attendees: PK, Sasha K, Gurleen, Matt, Sasha H, Ostap, @Boburbek Kadirkhodjaevlava, Yogesh, Pavlo, Craig McNally, Martin
Topics: 1) Local and remote debugging on Eureka platform 2) Sensitive info in application logs
Quick notes:
Eureka debugging
Setting up the local development env via the Go tool only works for those with at least 32 GB of RAM.
Remote debugging appears to be a viable option as well. Several notes here:
At this point only 1 developer can hit breakpoints in Rancher (all others will be blocked when a breakpoint is hit)
Sasha H said that there could be an option to use separate namespace for each tenant and hence enable multiple developers to debug. This needs to be explored further
PK suggested that if there was a way to quickly spin up a rancher env per developer and tear it down once the debugging is done, this could be another option. Cost $$ is a factor here, but labor and efficiency maybe worth more...
Lee B contacted Embassador Labs to understand the pricing model of their multi-user Telepresence option. I'll keep you posted https://www.getambassador.io/telepresence-pricing
Sensitive info in Logs
Reiterated to the leads that the sensitive info (not only the PII, but also things like dollar amounts, account numbers, etc) should NOT be at the INFO level in logs. Preferably it should not be there at all, but if we find it necessary to troubleshoot production issues, put it at the DEBUG level.
Teams TJet (mod-invoice, mod-orders) and Volaris (mod-audit, mod-email) are creating stories to remove the sensitive info.
Meeting December 10
Tjet exploring remote debugging on Rancher; building local UI; starting experimenting with Telepresence
Citation 16GB is not enough
Volaris problems building and running UI modules; trying to figure out problems with running DCB module
Folijet no progress, using deploying to Rancher
Firebird. no progress
Corsair 16 GB limitation as well
Spitfire no need for tools
Vega no progress
Meeting December 3
Attendees: PK, Sasha K, Gurleen, Matt, Ostap, Serhii, @Boburbek Kadirkhodjaevlava, Kate, Yogesh, Pavlo, Craig McNally, Martin, Yogesh
Topics: Local development on Eureka platform. Teams' readiness to develop for Eureka and on Eureka
Quick notes:
Several Teams have tried the Golang tool and there are the results:
Tjet (the tool author) can run the Acquisition app locally
Citation tried and got some container errors. PK will troubleshoot with Boburbek
Volaris was able to run some internal modules, but not the DCB (also container errors). Gurleen or her deputee will work with Boburbek
Folijet tried to deploy the Import app locally, but seemingly ran into memory issues. Kate is going to report more results shortly
Firebird. Slava intends to try the Bash tool in the next sprint
Corsair is using Rancher to deal with Eureka issues for now, but going to try the Go tool in the next sprint
Spitfire. Pavlo stated that Spitfire is mostly focusing on developing microservices independent on other modules. When they need to test integration they are doing well with Rancher, so they do not need local Eureka env.
Vega is still struggling with the Ramsons and Ramsons+ scope. Too soon to even dream about Eureka
In general, all team leads understand that in Sunflower we will be developing features exclusively on the Eureka platform, and we will touch Okapi only support production (which I'm afraid we're going to do quite a lot judging by the frenzy in which we're wrapping up Ramsons).
Please remember that the team leads are responsible for creating the most favorable development conditions for your teams. If your team chooses to continue using Rancher as Spitfire does - I respect that, however please do not complain that developing and troubleshooting on Eureka is slow and painful.
Meeting November 26
Attendees: PK, Sasha K, Sasha H, Gurleen, Matt, Ostap, Serhii, @Boburbek Kadirkhodjaevlava, Kate, Yogesh, Olamide, Pavlo, Oleksii Petrenko, Oleksii Kuzminov, Craig McNally, Steve Ellis
Topics: Local development on Eureka platform and tooling for that. In the meeting, we discussed the Go tool developed by @Boburbek Kadirkhodjaev and @Serhii Nosko
Quick notes:
We saw a very impressive presentation of the Go tool. The tool appears to be ready for prime time, but we are all anxious to hear from the users of the tool about their experience working with the tool. The Eureka team is going to record their tool demo later. Also, I scheduled another leads' meeting next week, 12/3, and I invite Eureka to present their solution then.
Another very important comment: Teams, please do not dump issues in Eureka's lap automatically when something works on Okapi but fails on Eureka. Please do some troubleshooting first. Please remember we're in it together. This is a link to the Eureka troubleshooting guide: https://folio-org.atlassian.net/wiki/spaces/FOLIJET/pages/628916278/How+to+triage+permissions+or+403+401+problems+on+Eureka+envs
Comments:
The ECS module is not loading automatically, so it needs to loaded manually (should not be a big deal, I'm hearing)
From @oleksandr_haimanov: can we figure out how to not load some standard components locally, like Kafka and postgres, to reclaim some memory space? See this tool https://www.telepresence.io/#gsc.tab=0
Team's readiness to start using the Go tool.
Teams ready to get started on the Go tool: Volaris, Thunderjet, Citation, Corsair
Teams ready to get started on the Bash Bootstrap tool: Firebird
Teams not ready to get started on Eureka work: Folijet, Vega
Teams who do not think any tool is necessary for local development on Eureka: Spitfire (@Pavlo Smahin: I'm looking forward to hearing about their way of developing applications on the Eureka platform)
Links
Meeting Recording:
Serhii/Boburbek presentation
Eureka's guide to troubleshooting issues
Meeting October 29
Attendees: PK, Sasha K, Sasha H, Gurleen, Matt, Serhii, @Boburbek Kadirkhodjaevlava, Kate, Yogesh
Topic: 1) Discuss Github Copilot tool to improve teams' productivity. Use cases considered included generating unit and Karate tests, code refactoring, unfamiliar code understanding (Python). Hoping to see a live demo next time
2) Discuss LoC support model. How do we allow developers to download LoC logs? How do we mask sensitive data pieces in the application logs?
Meeting October 8
Attendees: Ostap, PK, Sasha K, Gurleen, Matt, Pavlo, Ostap, Serhii, @Boburbek Kadirkhodjaev
Topic: Oleksii spoke about module resource management
Presenter: Oleksii Petrenko
Meeting September 24
Attendees: Ostap, PK, Sasha K, Gurleen, Matt, Pavlo, Ostap, Serhii, @Boburbek Kadirkhodjaev
Topic: Serhii and Boburbek present the tool developed by TJet to set up a Eureka environment locally along with remote troubleshooting
Presenter: @Serhii Nosko, Boburbek
Meeting August 20, 2024
Attendees: Ostap, PK, Sasha K, Gurleen, Yogesh, Slava, Matt, Pavlo, Ostap, Serhii, @Denis Kovtun
Topic: Denis Kovtun's talk about production support, issues in Rally, RRT
Presenter: Denis K
Other materials
RRT checklist for troubleshooting
Yogesh's topic Change Logs
Nightly Karate and Cypress run Change logs
Quick Note:
Please think about how to reduce the number of bugs that end up in Prod (or Bugfest). Improving logs was one suggestions. Resurrecting the RCA program to improve the code area where the issues occur was another.
Also, please monitor the daily Karate and Cypress runs on the QualityGatesCI Pipeline and address issues immediately. The Change log address is above.
Meeting August 6, 2024
Attendees: Ostap, Matt, Yogesh, PK, Sasha K, Gurleen, Kate, Sasha H, Martin, Mikita
Topic: Gitlab branching strategy
Presenter: @Mikita Siadykh