Action Items for Development

• Okapi does not have to manage deployment, but can take module descriptors.
• Possibly wind up constructing your own module descriptors, somewhat like DNS.
• Still have to work this out in practice, have thought about but not worked it out.
• EBSCO probably has most experience here.
  • Have their own CIC pipeline and register as code is committed,
  • regularly prune the list of descriptors.
  • Modify the module descriptor by appending a build or rev number, so can distinguish between builds.
  • kicking off their own Maven builds, have their own Docker images, not using much directly from the public Docker repo. Subscribed to git repos, commits kick off buids in Jenkins, can modify templates.
  • Probably not many sites hosting their own will be doing this workflow, but possible.

has been addressed in Madrid to Nassib, needs specifications.

we can go without, but there is advantage in addressing it.

Documentation belongs

a) in the README on github

b) on dockerhub

LS-Tools = A tool set at Cornell which allows technical services do library automation tasks through a common web interface.

It is essentially an ETL tool. It comprises cleanup jobs, export and import jobs. There is a specification written for each job that describes who owns it, how it is executed, what is needed, etc. The app also allows for searching of marc data with a highly granular selection capability. The app is a swiss army knife of technical services activities. It has meant a huge increase in productivity for technical services staff. Re-implementing this in Folio will be a challenge.

Some combination of the workflow engine and the marc batch loader could replace LS tools. The workflow engine is meant to be a platform to build these kinds of applications on.

• an issue getting Okapi/Folio to connect to postgres in single node deployment 
• use of a service discovery tool, rather than the current Okapi setup

• Individual microservices not typically aware of system, except self

1. Okapi is currently arbiter of microservices

2. Application shouldn’t manage its own state

3. Okapi has no sense of HA backend services

1. Can’t tell it to spin up multiple copies of service

4. Need backend module register itself with discovery service, at URL

1. Okapi would talk to discovery service like etcd or console

2. Would help with load balance abilities

• Most orchestration tools assume service discovery tool, which Okapi doesn't currently have

• 1. Proxied okapi request for circ data, what happens?

  2. How does Okapi know where that service URL is located?

  3. Okapi was designed to be agnostic to service discovery

  4. Wayne recommends current smoother path to use DNS based launch/deployment descriptors, rather than service discovery tool

  1. How much orchestration do we want Okapi to do?

  2. Difficult previously to have use-cases to give developers.

  3. What will it use other than URL for mod-circ?

• We don’t want Okapi as orchestration tool, only a proxy; too dynamic

• 1. Rather make Okapi work better w/ orchestration tools

  2. Wayne: Module registration is unavoidable in Okapi

  3. Give Okapi discovery service information ability

  4. Stephen: modules should be able to self-register to discovery service rather than posting to Okapi

  5. How do I know what I am supposed to run?

  1. Build front-end first, then back-end

  2. This is backwards

  3. However, in multi-tenant Front-end first may be more appropriate, according to Wayne
     

either being worked on or solved;

covered in Kubernetes Group.

Closed here.

At Wolfcon we worked out a plan for how to architect data migration. Implementing institutions would extract and transform their legacy data into a json representation that conformed to the json schema for the apis of the different modules, or, optionally, for bibliographic marc data, into marcxml. FOLIO developers will write a high-speed loader which would then move the json data, or parse and move the marcxml data, into the appropriate Folio modules. User loads may be an exception to this scheme because the requirements for bulk loading from on-going data feeds may be sufficiently similar to the requirements for the initial migration of data that a single loader may be able to serve both purposes. Wayne is currently exploring a suitable design for a high-speed loader. This project should have a very high priority.

Storage modules will need to accommodate bulk operations. Initial bulk loaders for migration vs. ongoing.

The data migration sub-group is currently working on an analysis of legacy data from our member institutions that is currently not represented in the published data models of the various Folio modules. We are forwarding the unrepresented data elements that we discover to the various domain SIGs to be considered for inclusion in data model revisions on which they are working. This project to enhance the data models of the various modules should have a high priority because of all of the down-stream development that depends on them, both for developers to code new functionality, but also for member institutions to review and comment on the adequacy of the institutional data that they have migrated.

Gap analysis will flow through the SIGs for the data model. This is going on with all of the modules.

https://discuss.folio.org/t/on-primary-identifiers-in-folio/1826

Functional requirements: We need human readable identifiers. Eye-readable. Can be read on the phone. It has to accept legacy IDs. New records do not have to be compatible with the legacy identifiers.

Preferred: Sequential identifiers. Sequential identifiers are related to the legacy identifiers. They leverage existing IDs by sequential numbers.

It doesn't make sense to fix it after having gone live.

• creates tenant
• sets up admin user

• reference data
• sample data

Release Management

Right now: snapshots and git commits. Is not sustainable. Need to invest resource. Similar to distribution management.

Solved in part by the Q Releases.

Other part: Ongoing work in the Kubernetes subgroup.

It's being worked on by the QA group : /wiki/spaces/DQA/pages/2656765 ; this is part of the Kubernetes subgroup.

Will develop playbooks. SysOps can pick up and follow.

Playbook: How to put together a Rancher system in AWS (is being developed).