| FOLIO infrastructure / NFRs |
| Status of our initiative Tod: - LDP app folded into the Infrastructure, not certain if that has been helpful.
- Added to roadmap. How to represent to move, no resources have been allocated.
- On the roadmap to get attention for resources
Ingolf: - What is the priority? Do we need to create JIRA issues and stories ?
Brandon: - Have JIRA issues for years, institutions don't rank NFR JIRAs, they rank features or bugs.
Phil: - Technical Debt/NFR Initiative, added
Anton: - From next release on, institutions don't do institutional ranking, this is not functional with 30+ institutions. Early adopter institutions rank. NFRs most of the time solve infrastructure, solve and kill poor technical design and implementation decisions, causing a lot of defects.
- Things like Okapi or using own application and database infrastructure, causing bugs. "Not invented here" inhibiting progress.
- Anton's job adequate layer of tests across the layers. Teams addressed by product owners, PO are not professional, librarians. Build features not tests, causing bugs to slip through in feature development. Last release 70% development for bugs. Relaying on PO NFR work, test work not possible.
- Driving effort through Scrum masters, features assigned to particular release, scrum masters groom tickets, team does grooming and estimating. Lotus release better job of grooming, test features verses functional releases. Reporting every week at Thursday morning meeting.
- Reported on wiki. FOLIO K dashboard.
- UI unit, API integration, UI end-to-end. tests.
- Underlying tickets to features and test, out in the open.
- Need team capacity for different dev days, functional K automation features,
- Team decided what can be development features.
- Teams focus on feature at the cost of testing, more apparent to the tradeoff. More than 60 seconds to check-in book becomes everyone problems.
- PO have domain knowledge but not enterprise building knowledge.
- Successfully ignore to add tests, PO not priority.
- We are still able to release product every three months.
- Monstrosity with dependencies, achievement that we can release with large clients.
- We are capable of doing that, some projects do not release, painful but we can release every three months.
- Our defect rate, still relative manageable.
- Spend a lot of time to fix but not enough time for new functionality.
- Overall not enough layers of automated tests,
- Discovering defects later in the cycle expensive,
- Bug fest should only a couple dozens, but finding 100s bugs.
- Number of bugs extend bugfest, makes it difficult to deploy for bugfest.
- This group, identify NFRs, priority, pick top 3 issues
- Groom tickets, and put into release.
- Resource dev team to start working on it now.
- Create stories, link to features, recommend to TC, "groom the hell of it"
Tod: - Capacity planning assigns resources
- Largest number of teams funded by Ebsco, IndexData funds a team. One or two teams donated time. DevOps and maybe community needs.
Anton: - finding community developers very thin and shrinking
- DevOps mostly focusing on Rancher and community needs.
- Rancher is being used and people realize the value of Rancher. Rancher allows long-running set-ups to development before being committed to master. Keepers can see Rancher for the community.
- Lot of needs by Devs, buried in Rancher work to support different teams
- UIX JIRA prod tickets and underlying user stories, describe what you need to be created. Example: As a sys ops, would like Data schema for 10 million items. Stories will have sub-stories. Each module contributes to the time it takes. Identify those modules that will take the most time. PO and SysOp work.
- If you want things do, create UIX prod and related work tickets with requirements. Talk to people can size the tickets (not necessary to size)
- List of SysOps features, internally SysOps to rank top three, spend time on each one and break it down and what we would like to see happen. May convert all modules to Spring Boot framework verse Vertix for database
- Get rid of storage modules
- What is most important for infrastructure need.
- UX prod feature to link it all.
- The Data Import problems could be seen in that light.
- Grooming: UXPROD + underlying user stories e.g. it would like a schema update within 2 hours; Story will have substories (=schema updates of all the modules)
- Spring Boot Framework vs. Vertex Framework if you want to get rid of storage modules
Ingolf - Need JIRA tickets and UI ticket "grooming"
- Bring to capacity team
- Julian: "Can hardly talk about SysOps needs, because NFRs are too vague"
- We have to do much more work "grooming" tickets
Tod - Have the beginnings of the SysOps needs in the document Technical needs met when standing up FOLIO installs
- The data import problems can be seen in that light
- Anton identify things that are not in the document
- We have some things that were not talked about; Do we have an architectural model ? This needs interaction with the TC.
- Getting rid of storage modules
- Sketch out and build priorities.
- Biggest tech problem,
- Distributed "ball of mud" complicated bits of communications between modules not happening efficiently or effectively.
- Getting rid of business logic/storage distinct.
- Larger decision to fully embrace micro-services architecture,
- define multiple operations where eventual consistency is fine.
- Or move model gather modules together, many modules have large planes of interface with other modules.
- The more modules you have, the more round trips you have.
Anton: - More modules require more communication
- We store data in JSON fields, amount of processing getting data in and out of JSON field is insane.
- Schema tools for database migrations
- One thing per release
- Grooming identifying several modules business logic and storage modules to be merged together,
- APIs merge and change. Schema for the module, will change.
- Change JSON schema to a relational schema, search now driven by ES
- Update schema for releases
- Indices bigger than the dataset, improve JSON
- Uses tools that are widely adopted verses "invented here"
- JSON was a compromise for full-text search,
James - We have end user pain points: checkout time, ...
- The CheckOut and Upgrade times are great ways to communicate the risk of not tackling NFRs and seem to resonate strongly with PC. Are there other stories that we can use to help Product Owners understand the risk/impact of this work?
- Strengthen domain boundaries
- Can we pick one thing to work on: storage logic and business logic. Collapsing business logic and storage, modules addresses many of these problem
- Store data in JSON - amount of data isn't sane; complexity isn't sane.
- Tie to PO pain points, help problems focus on one thing at a time, show how one thing can fix multiple pain points. No more than three or no more than one per release.
Brandon - FOLIO cannot make a decision by committee, any of these things we are talking about take a lot time to implement.
- JIRA tickets can persist for years
- Every time we talk about this, shot down by multiple committees, the project should commit to this because NF, won't get shot down
- Doubt if one thing will just be for a release, span multiple releases
- Can't get buy-in from TC, they don't seem to be able to address any technical debt
- Focus on domain boundaries.
- It is going to cost features. It must not be shut down by any of the approx. twelve committees who work on this.
Tod: - Sounds like a good topic for next WolfCon
Philip - Mega session about technical debt at WolfCon
Jason - Tackle domain boundaries. Domain boundaries will probably have the biggest impact and benefits to the platform overall and take the most time.
James - Slow in check-out see Check Out Performance
- Merge business logic/storage modules
- Move from JSON to Relational DB fields
- UX Prod epic
Anton: - One UX Prod epic ticket, linked to multiple tickets which each team will implement.
- "Grooming" : Identify several modules that need to be merged togther.
- Change schema to relational schema. Get rid of JSON. Because of Performance.
Tod: - Thinking about check-out performance, read Mark's proposal and comments: Check Out Performance. Give us some context to work in
- Storage getting rid of JSON schema for storage
- keep JSON for communication, scrub it for storage
- This will facilitate PATCH
- Update a field in a JSON clob, need to rewrite entire CLOB, all indices that use the CLOB need to be updated
- Data import and export SRS export almost always massage data on the way out. JSON CLOB, parse the whole thing in memory to apply your export profile, insert Item and Holding data, re-serialize all into JSON
- Express things about domain boundaries would be very profitable, look at in the long view,
Ingolf - What should we do next? Write UX Prod, focus on one per release.
- Module Boundaries
- UX Prod
Tod - People should revisit the SysOps tech document and priorities
- If something changes in the Sys Ops needs, I can reflect that in the roadmap.
- Pick top three areas, come to at least one layer of steps for each area.
- Picking top three things a good strategy, if we can't get our top item, could get some purchase on the second or third areas.
Ingolf - Will look at addressing some of these during holiday. Can't meet next two Fridays?
- Should we meet in the meantime.
- Next Tuesday Integration sub-group meeting
|