FOLIO Project Technical Objectives & Initiatives
Current as of February, 2023. Accepted on March 8, 2023.
Review scheduled for September, 2023.
Overview
This document reflects both current work within the FOLIO Project and TC aspirations for future technical work and direction to support the program described in FOLIO Vision, Strategic Objectives and Initiatives.
The TC will consult this document when considering and evaluating technical initiatives and priorities for the Project and use it as a guide to help make decisions about technical directions within the Project.
This document can also be used in discussion outside of the TC about sequencing technical work to support the goals of the Project. This includes discussions with the other councils, with development teams, SIGS, etc.
The TC will periodically revise this document to reflect new initiatives that come up through the community and to reflect updated technical priorities for the Project.
Vision
Support a thriving open source community in building a LSP (Library Service Platform) that is effective, cost effective, & reliable
Objectives
- Community Growth (CG)
- Enhance LSP Capabilities (LSP)
- Improved process efficiency and transparency (ET)
- Efficient operations and maintenance (OM)
- Technical Debt Management (TD)
Initiatives
The initiatives listed here both represent work in the technical aspects of the FOLIO project, whether the TC has the role of leading or endorsing the work. This includes work actively going on in different parts of the community and aspirational initiatives which the TC would like help steer the technical direction of the project.
These initiatives are at a varying degree of granularity, but generally described at a high level; refer to the supporting documents under Resources for more detail.
The time frames are intentionally ambiguous as priorities shift and available effort for particular items is not guaranteed.
Immediate Initiatives (active/ASAP)
Many of these initiatives were generated by dev teams and other technical stakeholders and are being brought to TC for review and approval.
- Enable lightweight development and deployment environments (i.e. Platform Minimal) (CG, LSP, OM)
- Has been recurring theme for SysOps and developers.
- See FOLIO should define a minimal platform and also under Basic FOLIO tenant installation
- Reduce Kafka Operational Costs, resolve tensions around increased and varied use of Kafka with the associated hosting costs. (LSP, OM)
- Help standardize community terminology around "platform", "library services platform", etc., terms are used differently in different conversations. (CG, ET)
- FOLIO Cross-Application Data Sync Solution (LSP, OM)
Near Term
- Improve Operational Documentation (ET, OM)
- Release (See Pain Points doc, under Operational documentation)
- Provisioning new tenants (See Pain Points doc, under Okapi bootstrapping and under Basic FOLIO tenant installation)
- Publish technology roadmap for 2023 (CG, LSP, ET)
- Painless Onboarding (CG)
- Create architectural blueprint (LSP, ET)
- Reduce resource footprint required to run FOLIO in the cloud and locally (CG, OM)
- Release upgrade improvements (OM)
- More complete/reliable documentation for upgrades
- Improve reindexing process/times (See Performance issues - storage)
- All modules should support read-only PostgreSQL replicas (i.e. support read/write separation) (DB_HOST_READER, RMB-348) (LSP, OM)
- Document preferred/supported/unsupported approaches to secret management (OM)
- Documentation goals: (CG, LSP, ET, OM)
- Improve documentation standards for modules, to facilitate understanding of basic purpose and operation of a module.
- Include documenting implicit/hidden dependencies such as interactions through Kafka.
- Improve API documentation to include things like supported searches (available indexes) and API side-effects, to help people new to the API understand how to use in in a practical way.
- Improve operational and installation documentation for running FOLIO and for understanding the technical architecture and philosophy.
- Improve documentation standards for modules, to facilitate understanding of basic purpose and operation of a module.
- Guidance about balance between HTTP and Kafka as inter-module communication (LSP, OM)
- Improve observability/logging for troubleshooting (OM)
- See Logging proposal.
Medium Term
- Catalog, Prioritize and plan for addressing the critical project wide tech debt items (TD)
- Consortia Support (related to tenant issues) (CG, LSP)
- Improvements to managing tenants (OM)
- Support for System, Tenant users and roles
- Improve basic FOLIO tenant installation
- Tenant Management tools - Administrative vs. Runtime
- See Pain Points doc, under Okapi bootstrapping and under Basic FOLIO tenant installation
- Conduct Hackathons, webinars and other outreach efforts to spread the word about FOLIO (CG)
- Actively recruit libraries worldwide that have a proven track record with open source development projects and have the experience and resources that would benefit FOLIO (CG)
- Promote and illustrate the benefits of FOLIO's community-owned reporting capability in partnership with the Library Data Platform (LDP) project to increase adoption of the FOLIO platform. (CG)
- Form and support User Groups, carefully ensuring fit with existing groups and SIGs (CG)
- GDPR support (CG, LSP)
- Simplify back-end module interactions (LSP, OM, TD)
- Consider how to reduce the granularity of the modules and simplify the interfaces between modules
- See Module boundaries and inter-dependencies
- Consider how modules are designed around records vs workflows
- Take up a pilot project to redraw some module boundaries
- See also Implementing the Transactional Outbox pattern
- Release packaging (OM, TD)
- See 2022 WOLFcon - Architectural Blueprint Enabler: Release Packaging
- See FOLIO modularity in practice: Seamless deployment of modules from multiple sources, Mike Taylor, WOLFcon 2022 (Slides, Recording)
- Okapi role: facilitate a conversation about the desired role and capabilities of Okapi. (LSP, OM, TD)
- What do we want and need from Okapi?
- Need to support both self-hosting and large-scale hosting.
- Okapi bootstrapping & management (OM, TD)
- Improve documentation and processes for setting up Okapi
- Improve situation for managing module discovery and deployment
- including for locally-developed modules
- ability to share modules developed in the community but outside of the project itself.
- Address issues around REST API design in FOLIO (LSP, OM, TD)
- Address issues around batch APIs in FOLIO (LSP, OM, TD)
- Performance issues around design of storage (OM, TD)
- Perceived performance issues around synchronous updates (LSP)
- Data consistency issues (see also FOLIO Cross-Application Data Sync Solution and Implementing the Transactional Outbox pattern) (OM, TD)
- Data Import complexity and performance issues (LSP, OM, TD)
Long Term
- Event driven architecture (but see also discussions of HTTP vs Kafka and Kafka hosting expenses)
- Support for non-MARC record-based data models (for instance: Dublin Core, VRAcore, PBCore, FGDC)
- Cross-tenant architecture (LSP, TD)
- Create an app store and incentivize the addition of apps to the store by individuals, institutions, and commercial partners
- Search Engine
- Reporting and Analytics
- Workflow Automation Engine
- Inter FOLIO integration
- Develop and distribute an Implementers’ Satisfaction Survey
- Measure implementers satisfaction
- Develop an Implementers SWOT Analysis
- GraphQL - Eliminates stripes connect
- Concerns that SRS is too tied to MARC21, and less open to other MARC dialects
- Stripes platform build and deploy issues
Resources
FOLIO Vision, Strategic Objectives and Initiatives