2018-03-01 - System Operations and Management SIG Agenda and Notes

Determine note taker: Tod Olson

Review action items:

• How do we deal with FOLIO feature backlog? (2018-01-04 meeting) This action was related to the Integrations, but creeping into concern about getting SysOps concerns prioritized.
  • Wayne Schneider will review the last couple meetings, pull out items, and list, track, and push through.
  • Effectively, we will do our own product ownership until the project governance catches up.
  • These will be issues that come up around getting systems up and deployment, focus on operational needs.

Observations and Questions by TAMU, presented by Brandon Tharp

• TAMU finding a number of problems coordinating running containers across different machines.
• We think the purpose of this group is to raise these sorts of issues and push for their resolution.
• On pulling all available modules into Okapi, Wayne Schneider expects that in production we would not work in an uncurated world, but that any site would curate the list of which modules and versions one would run.
• Right now, difficult to build Stripes and run modules on different machines.
  • Okapi does not have to manage deployment, but can take module descriptors.
  • Possibly wind up constructing your own module descriptors, somewhat like DNS.
  • Still have to work this out in practice, have thought about but not worked it out.
  • EBSCO probably has most experience here.
    • Have their own CIC pipeline and register as code is committed,
    • regularly prune the list of descriptors.
    • Modify the module descriptor by appending a build or rev number, so can distinguish between builds.
    • kicking off their own Maven builds, have their own Docker images, not using much directly from the public Docker repo. Subscribed to git repos, commits kick off buids in Jenkins, can modify templates.
    • Probably not many sites hosting thier own will be doing this workflow, but possible.
• TAMU now has a very basic working system, but may not be ready
  • PoC: try to start by splitting out into separate containers, then stand up a real Okapi, Postgres, etc.
  • hard part is figuring out based on the front end modules, what versions of the back-end modules you need to run.
  • Existing sample deploy script works with single-container or host deploy, but hard to split across containers

  • One idea: FOLIO module descriptor registry, when you do an Okapi pull, post a URL to your local copy and that URL is actually an Okapi instance. There might be some way to expose the install endpoint, when you do ```install&simulate=true```, so you can get the dependencies.

  • Is there a way we can expose an endpoint? Wayne Schneider will talk to the Okapi developers, this seems like a reasonable use case, and a necessary feature for an appstore-like experience.
• Issue aobut enforcing dependencies, from Craig McNally. In order to register a module, the dependencies must already be in place. That's a lot of overhead for someone coming in new.
  • Pull feature, for pulling all modules, was an attempt to deal with this
  • Problem is that it clutters up the system.
• Back to TAMU experiences:
  
  • First part: what versions am I supposed to run?
  • Then need to find those versions in GitHub or in the Docker hub.
  • Then hazy where to go from there. See question 8.
  • Particular questions about how to spin up a module with a back-end database, often hard to know the right way to get the initial data in (temporary way is to get Vagrant image and dump the data). Big pain point.
  • Related, Wayne Schneider is working on RAML for better documentation of run-time switches, etc.
  • Lots of problems consistently building Stripes. Dependencies change, etc.
    • How to specify what goes in a Stripes bundle?
    • By building a Stripes bundle with a package.json file, use yarn to pull in all dependencies.
    • Stripes changes a lot, get weird build errors during dev process. Will there be a stable version of Stripes to pull from?
    • Wayne Schneider Stripes has been moving too fast to do effective release management, but the plan is to publish stables releases of stripes-code and various modules. Moving towards that for June.
    • Will still have to build the web pack, because that has information specific to your installation.
    • Plan right now is to stabilze the package releases so that you can use yarn or mpn to manage the whole process. That's supposed to make manageing Stripes easier, but does not have experience with that yet.
    • Likely mechanism is for project to publish a cannonical Stripes package that points to correct versions of modules that are supposed to go with v1.
    • Think we will always be building from top-down.

other questions / comments