Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device.
Atlassian uses cookies to improve your browsing experience, perform analytics and research, and conduct advertising. Accept all cookies to indicate that you agree to our use of cookies on your device. Atlassian cookies and tracking notice, (opens new window)
Q2 spillover work: RMB 25 roll out in progress, bugs found and being addressed. Bugfixes planned for RMB 26 (may come out as RMB 25.1.0) plus completion of two features required for perf issues around item barcode searching:
There have been several post-deadline hotfixes - haven't been great about using the release bug triage process because people were away at FOLIO face to face
I've requested another hotfix for a couple of Requests UI bugs
POs are wrapping up the Q2 feature closes and splits today
Capacity plan will be updated tomorrow so we can nail down the specific plans for (at least the beginning of) Q3 - Q3 plans may be adjusted after the SIG re-ranking
Perf issues raised after BugFest environment logs and instrumentation data were analysed, namely: MODUSERS-125 (DONE), MODUSERS-127 (DONE), MODPERMS-57/MODAT-45 (initial commits are in), MODINVSTOR-292 (likely requires a multi-table join support in RMB which will be available in RMB 26)
RMB 25 has been released. 25.0.1 out today with a fix for RMB-397. It's heavily refactored through RMB-376 (merge CQL2PG into RMB) and includes major search performance improvements including CQLPG-81 (redesign CQL-to-SQL generation) and CQLPG-96 (remove regex-based query tokenisation).
The following features has been pushed back to RMB 26:
CQLPG-99 (FK support, perf issues with location searching and item.barcode reported by analysing BugFest env metrics) which is IN PROGRESS.
CQLPG-83 (new syntax and implementation for searching arrays) which in CODE REVIEW
mod-inventory-storage has been ported to RMB 25 (MODINVSTOR-285) and the "rmb-25" branch build of mod-inventory-storage is being tested on the perf test env (FOLIO-2087). Port of mod-circulation-storage (CRICSTORE-134) is WIP.
Okapi 2.30.0 released and includes a fix for OKAPI-740 (this problem temporary broke the performance testing environment)
FOLIO K8s integration: OKAPI-729 (service discovery) and OKAPI-736 (PoC K8s API integration) will be pushed back to sprint 66.
DevOps
FOLIO-2036 (Include custom backend modules in PR builds for branch preview) has been completed
FOLIO-2008 (Publish a minimal README at Docker Hub for each back-end module) has been completed
FOLIO-2054 (Stand up cluster for CI using AWS K8s procedure and documenation) is in progress
FOLIO-2057 (SPIKE: explore AWS K8s cost models) is in progress
Core Platform team is assisting AtCult to try to get mod-/ui-marccat in shape for the Q2 release. Various issues with, grouped under FOLIO-2060.
@Cate Boerema (Deactivated)
Q2 Release:
Lots of PR reviews and release prep
PR reviews are a bottleneck - some work that was completed in time may not make release because of insufficient review time
Release milestone deadline is tomorrow (June 12)
Q2 priority bugs - can we fix after June 12?
Planning:
Holly has proposed a process by which POs review their UXPROD feature rankings with the appropriate SIGs to make sure features are understood and properly ranked - idea is that PO will frame discussion and highlight surprises. SIG members should be empowered to change rankings right in the meeting. Will test this process for Requests in RA SIG on Thursday.
We will want to "zoom out" and conduct this (or similar) exercise for all core module features, as well
To make the cap plan more useful without disrupting development, Cate, Jakub and Harry will estimate remaining unestimated features. Will tag with "SWAG" as a reminder that these were not estimated by developers (as per usual process)
What should we do about the sprint review that would normally happen next Tuesday (during the face-to-face)?
@Owen Stephens
Three Q2 2019 features in ERM that will be spilling over to Q3 2019 instead
UXPROD-1157 - we are waiting on UINV-7 to be delivered before we can complete this feature
UXPROD-1511 - required for Chalmer's Go Live and will be prioritised in Sprint 66
UXPROD-1517 - in progress but not completed. Work will continue in Sprint 66
Addressing perf issues that were raised after BugFest environment logs and instrumentation data were analysed, namely: MODPERMS-57, MODAT-45, MODUSERS-125, MODINVSTOR-292 (blocked on RMB 25, see below)
Concentrated effort to release RMB 25 this week. It's heavily refactored through RMB-376 (merge CQL2PG into RMB) and includes major search performance improvements including CQLPG-81 (redesign CQL-to-SQL generation) and CQLPG-96 (remove regex-based query tokenisation). The pending features are:
CQLPG-99 (FK support, perf issues with location searching and item.barcode reported by analysing BugFest env metrics) which is IN PROGRESS.
CQLPG-83 (new syntax and implementation for searching arrays) which in CODE REVIEW
Once RMB 25 is released the team will start porting core backend storage modules, starting with mod-inventory-storage (already ported on a branch, MODINVSTOR-285), and start testing branch builds on the perf test env. Decision to include in Q2 next week. If held back it will be slated for mid-Q3, along with other storage modules ported over.
Okapi and RMB bugfixes: RMB-384, OKAPI-740
FOLIO kubernetes integration: OKAPI-729 (service discovery) and OKAPI-736 (PoC K8s API integration) spilled over, in a waiting state due to perf and stability issues.
DevOps
FOLIO-2036 (Include custom backend modules in PR builds for branch preview)
FOLIO-2008 Publish a minimal README at Docker Hub for each back-end module, almost complete
First time the same deadline for 'platform-core' and 'platform-complete'. Long list of pending changes in mod-circulation/storage (core). Expecting dependency mismatches. Core module maintainers to specify final interface and library version numbers by the end of the week.
Just over one week remaining until feature freeze for Q2/Clover (June 5th)
Module release deadline is June 12th
Core features update:
Core functional:
Moving requests from one item to another (UXPROD-1653) is last big feature needed by Chalmers to go-live. Development starts today. We hope to get at least one of the 3 stories completed prior to feature freeze: UIREQ-269
Many other features are close to completion
There will be some spillover for features not needed by Chalmers to go live such as: Item templates (UXPROD-1604) and Making hold request loan policy settings effective on loans (UXPROD-1620)
Vega:
Time-based patron notices tracking well for release per Darcy (e.g. courtesy and overdue notices)
Expired hold report (csv) is nearly complete
There may be some spillover on Loan policy overrides (UXPROD-1609) but only a portion of that feature is needed by Chalmers to go live
Concorde:
Emma Boettcher has taken on the role of lead PO for this new team
Top priority for Concorde is to wrap up the Weekly overdue report (csv) (UXPROD-1621) which is a Chalmers go-live requirement
In addition, the team is familiarizing itself with the Circ rules editor in preparation for the work to integrate Locations into circ rules
POs are updating their feature rankings - they will use the "score" (aggregate ranking from early implementers) unless there is an obvious dependency or other likely misunderstanding in which case they will modify with comment
Tomorrow I will meet with the tech leads for the Core features to look at the top priority features and themes and distribute them across the 3 teams (Core Functional, Vega and Concorde) so we can better understand what is feasible given our capacity
Cap plan presentation for the PC is planned for Thursday
@Jakub Skoczen
Mid sprint 64
Dev:
Kubernetes integration: OKAPI-729 (service discovery) and OKAPI-736 (PoC K8s API integration)
SQL generation: RMB-376 (merge CQL2PG into RMB) DONE, CQLPG-99 (FK support, perf issues with location searching), CQLPG-83 (follow implement of array searching, perf issues for e.g identifier searches
DevOps
FOLIO-2029 (monitor bugfest environment for scalability issues)
FOLIO-2002 (clean up no longer needed tenants and bundles (PR previews))
FOLIO-2008 Publish a minimal README at Docker Hub for each back-end module
FOLIO-1992 (add Dockerfile to platform-core), FOLIO-1994 (consolidate Okapi docker containers and update documentation)
Just kicked off Sprint 64 which is the last full sprint before feature freeze for the Clover release
Vega will be picking up the work needed for the Expired holds report (csv) UIREQ-273. After much discussion, it was decided that we would create a purpose-built report for Expired holds (thin thread is simple csv export). Initially we had planned to create a new Item state (Pickup expired) which could be used to filter the existing Requests csv export but that proved too complex. Simple report will do the trick for now. Eventually we'll probably need a filterable "in-FOLIO" report aside from the csv
The Concorde developers have left Core Functional and Thunderjet and begin this sprint as their own team. Their top focus is to complete work on the Overdue loans report (csv) UIU-986 but Emma Boettcher and Sean Thomas have prepared other features and stories for Concorde, as well. Kick off meeting is today.
Core Functional is left with just 1.5 backend developers and both are off FOLIO this week (due to time off and work on other projects). This means little, if any, Move Requests from One Item to Another develoment will get done for the Clover release. Stories are ready to go, though, and this is a top priority for CF backend developers
Capacity planning team has been meeting and hopes to update the Product Council with a plan next Thursday (May 30th)
Community wants to make a push for more PO volunteers. I've started drafting a page with the job description etc. I still need to identify the proper contact person for interested parties. Might be nice to get a testimonial quote or two from happy community POs. Wiki page: Product Owners
@Jakub Skoczen
Started sprint 64
Dev:
Kubernetes integration: OKAPI-729 (service discovery) and OKAPI-736 (PoC K8s API integration)
SQL generation: RMB-376 (merge CQL2PG into RMB), CQLPG-99 (FK support, perf issues with location searching), CQLPG-83 (follow implement of array searching, perf issues for e.g identifier searches)
A list of spillover assgined to Julian – they are complete but technically unresolved and blocked on RMB-376
DevOps
FOLIO-2029 (monitor bugfest environment for scalability issues)
FOLIO-2002 (clean up no longer needed tenants and bundles (PR previews))
FOLIO-2008 Publish a minimal README at Docker Hub for each back-end module
FOLIO-1992 (add Dockerfile to platform-core), FOLIO-1994 (consolidate Okapi docker containers and update documentation)
Discussed ability to Move Requests from One Item to Another (for queue rebalancing) with the RA SIG and confirmed basic requirements. Stories will be written over the next couple of days. This is a go-live requirement for Chalmers.
Bug Fest happening this week - POs will be very busy writing new test cases and testing
PO gap analysis assignments from Holly due May 17
POs to process 211 "missing features" from gap analysis. Review feature, discuss with reporter and SIG as-needed, add to JIRA as needed. "First perusal" due this week.
Estimate unestimated features in UXPROD unless they are very low priority (there are ~100)
@Jakub Skoczen
Core: platform is in mid sprint 63
Dev (performance and backend frameworks/toolkits):
OKAPI-730 (system call timer). Scheduling. Implementation has been completed and initial bugs has been addressed.
CQLPG-83 (spike for searching array-of-objects/strings) has been updated with a new design, to be finalised in the coming days at which point an implementation task will be created.
Note: SQL query generation changes will improve performance for searching textual fields (e.g title) and complex (nested) fields (issn/isbn identifiers, contributors, location), new functionality will address search-related bug reports: UIIN-435, UIU-939, MODINVSTOR-277 and more
CQLPG-81 (redesign SQL generation) has initial implementation ready, some changes to the query generation algorithm ha
CQLPG-99 (implement foreign key support) has been clarified further and implementation is on the way
various perf issues: RMB-347 (generate UUID in a replication friendly way, completed) and bugs: RMB-353 (support change metadata without user ID/name, completed awaiting another review), CQLPG-89 (sortby id to use the id column) and CQLPG-88 (support > for chunking by ID) are spillovers
DevOps
FOLIO-1948: PoC preview capability for PRs has been completed. Description and examples has been included in the comment section of the issue.
FOLIO-2001: an optimisation task for creating and serving Stripes bundle in the CI has been created to make the PoC viable for wide use
FOLIO-1992, FOLIO-1993, FOLIO-1994 (docker containers for Okapi, platofm-core and platform-complete) has been created during the SysOps meeting last week (Wayne, Adam, Jason/TAMU, Anton, Taras) and added to the sprint
OKAPI-729 and OKAPI-728 (implementation tasks created during the same meeting) have been added to the backlog, ETA: sprint 64
FOLIO-1960: backend stopped under high load, awaits more investigation
various Ansible/CI cleaning tasks: FOLIO-1709, FOLIO-1974 (completed)
Dev (performance and backend frameworks/toolkits):
FOLIO-1953 (design spike for scheduling tasks in Okapi) concluded with OKAPI-730 (system call timer). Implementation has begun immediately in sprint 62 but faced cluster "locking" issues. Completion slated for today.
CQLPG-96 (remove regex based tokenization) has been completed and CQLPG-83 (spike for searching array-of-objects/strings) has been updated with a new design, to be finalised in the coming days at which point an implementation task will be created. Note: SQL query generation changes will improve performance for searching textual fields (e.g title) and complex (nested) fields (issn/isbn identifiers, contributors, location), new functionality will address search-related bug reports: UIIN-435, UIU-939, MODINVSTOR-277 and more
CQLPG-81 (redesign SQL generation) has been started
CQLPG-99 (implement foreign key support) has been started
various perf issues: RMB-347 (generate UUID in a replication friendly way, completed) and bugs: RMB-353 (support change metadata without user ID/name, completed awaiting another review), CQLPG-89 (sortby id to use the id column) and CQLPG-88 (support > for chunking by ID) are spillovers
DevOps
FOLIO-1948: PoC preview capability for PRs has been completed. Description and examples has been included in the comment section of the issue.
FOLIO-2001: an optimisation task for creating and serving Stripes bundle in the CI has been created to make the PoC viable for wide use
FOLIO-1992, FOLIO-1993, FOLIO-1994 (docker containers for Okapi, platofm-core and platform-complete) has been created during the SysOps meeting last week (Wayne, Adam, Jason/TAMU, Anton, Taras) and added to the sprint
OKAPI-729 and OKAPI-728 (implementation tasks created during the same meeting) have been added to the backlog, ETA: sprint 64
FOLIO-1960: backend stopped under high load, awaits more investigation
various Ansible/CI cleaning tasks: FOLIO-1709, FOLIO-1974 (completed)
Item Templates (https://folio-org.atlassian.net/browse/UXPROD-1604, https://folio-org.atlassian.net/browse/UXPROD-1664) - Chalmers has moved item templates feature from needed for go-live to can wait one quarter. Met with Charlotte, Niels and Marc and we have a proposed phasing plan. Ultimate concept is the ability to CRUD templates of a record which would involve specifying which fields should be visible and what, if any default value they should contain. Will start with really basic, semi-hardcoded version while Charlotte works with UX and SIGs for further designs
Performance in Chalmers environment: In prepping for demos at Chalmers next week, I’ve noticed ongoing performance issues. Next steps?
Planning future development in Core modules
Many features are blocked by OKAPI Scheduler (currently in progress by Core: Functional)
RA SIG is considering designing a "circ log" which would capture circ transaction data related to loans, fee/fines, requests, patron notices all in one place. The log would then be queried and displayed from various record types (e.g. patron, item, request) This may be simpler to implement than one-off logs for the various types of objects and would open the possibility of being able to search across all activity (e.g. show me all circ activity that happened within a given time frame).
@Jakub Skoczen
Core: platform is in the middle of sprint 62
Dev (performance and backend frameworks/toolkits):
FOLIO-1953 Scheduling tasks in Okapi (work required for sending patron notices and hold-shelf expiration)
CQLPG-96 (remove regex based tokenization) and CQLPG-83 (spike: searching array-of-objects/strings) – SQL query generation changes to improve performance for searching textual fields (e.g title) and complex (nested) fields (issn/isbn identifiers, contributors, location) addresses search-related bug reports: UIIN-435, UIU-939, MODINVSTOR-277 and more
various perf issues: CQLPG-89 (sortby id to use the id column), RMB-347 (generate UUID in a replication friendly way) and bugs: RMB-353 (support change metadata without user ID/name), CQLPG-88 (support > for chunking by ID)
DevOps
FOLIO-1948: PoC preview capability for PRs
FOLIO-1960: backend stopped under high load (investigation)
various Ansible/CI cleaning tasks: FOLIO-1709, FOLIO-1974
Release Management:
converted the Q1 release channel to #releases which we will use from now on to coordinate release