/
2019-04-05 - System Operations and Management SIG Agenda and Notes

2019-04-05 - System Operations and Management SIG Agenda and Notes

Owned by Ingolf Kuss
Last updated: Apr 05, 2019

Date

Attendees

Goals

  • Miscellaneous

Discussion items

TimeItemWhoNotes
5WelcomeIngolf
  • Welcome; are there new members ?
    • Jo Drexl works at the Leibniz Supercomputing Centre of the Bavarian Academy of Sciences and Humanities. Jo points out that Docker has security issues. Jo uses 2000 VM-Ware machines for FOLIO deployment at his institution and they want to go to 4000. They don't intend to use Docker Containers. The VM Ware cluster will do the load balancing, it will spawn machines. The orchestration will be done by Ansible.
    • Jack Hill works at Duke University, where he is a system administrator. Jack also sees security issues using Docker.

  • Who is willing to be the note taker ? Ingolf volunteers.

SysOps SIG F2F Meeting at June FOLIO Working MeetingIngolf

Working Meeting is from June 17-19 in Crystal City

Jesse Koennecke (the co-chair of the Product Council) confirmed (shortly after the meeting) that the SysOps SIG will be definitely included in the meeting schedule ! And we will get more details next week. Registration for the meeting is not yet out.

  Recommended ToolchainIngolf, all

Continue the discussion we started last week.

  • Other tools than Kubernetes
  • Other orchestration tools being considered

Meeting Notes

Discussion around using Docker containers for deployment arises.

Anton: FOLIO will be shipped in Docker. The .jar files will be available also.

Wayne points out that FOLIO will move to Java 11 (in which release, Wayne Schneider ?)

Jason mentions that there is a difference between orchestration management and configuration management . We should be clear about where the difference is. Orchestration management could be done with Ansible and Salt. But Ansible is rather a configuration management tool (all agree).

Anton says that a firewall can be statically configured down to the load balancer. But there is a gray area. We shouldn't introduce an irreducible complexity about the Kubernetes-Rancher solution.

Rancher uses Charts / Helm Charts as a path to update. Helm charts are not specific to Rancher. They are an update path from one release to another. They can apply to any Kubernetes infrastructure ! The Charts are the "footprint" of continous deployment. The old chart is replaced by a new chart on the frontend modules. At the same time, you have to exchange the backend modules. This procedure comes with a certain risk. The more you accept downtimes, the more you can minimize that risk. Rancher has an expansion to Helm Charts.

One way to orchestrate Docker is through Docker Compose. Docker Swarm can also handle that for you. U Colorado (@Boulder ?) has done much work on that. (IK: Let's contact mark.stacy to give us the status on this).

Jo Drexl asks if we have built subgroups that work on deployment with the different operating systems (Windows, Linux flavors; bare metal machine or VM).

Anton replies that there is an engineering process which works for Linux, Windows, bare metal and virtual services. There should be some recipes, whether to swap services by hand, Salt or by Rancher. We should be trying to get something common between all of us.

Ingolf points out that we should definitely agree on something which is common for all. For also from the developers (Jakub) Ingolf heard that they expect that the SysOps SIG will agree on some kind of recommended deployment procedure. If that will not be the case it will be hard for the core developers to support something that is needed for production deployment. So far, the development has (naturally) focused on deployment for development purposes. Jakub and the other core developers will follow closely the issues and hooks that the SysOps SIG will bring up in connection with production deployment. Jakub also said that he considers Rancher as an adequate solution for the TAMU deployment scenario, but he wouldn't consider it the method of choice for all other institutions. - We should focus on more basic principles, which are shared by several orchestration or deployment tools. - Ingolf thinks that Anton just gave a great example of this with mentioning the Helm Charts. - Jakub said (in the ERM Production Release Strategy Meeting today) that he is looking for requirements concerning the use of etcd in production deployment (it was said that etcd provides persistency, while Okapi doesn't actually persist; postgres is used for persisting).

Brandon sums up where our issues in prodcution deployment, that we have encountered so far, lie:

  1. Dependency resolution within FOLIO
  2. Service Discovery - How do we let Okapi know where all the things are
  3. Tenant initialization
  4. The Upgrade Process - How to go from one release to another

Jo: (We need to) give Okapi and the other modules the possibility to .... while ... is on hold ...  (!!! @Jo, can you complete this sentence ? I somehow missed the complete context...)

We don't expect that we will be able to (and need to) go from one version to any other version (persumably).

Wayne about Edge Modules :

How are Edge Modules deployed ? What do they look like ? Edge Modules are not employed through Okapi. Wayne will add a documentation. NCIP, RTAC and OAI-PMH are examples of Edge Modules. Pointing outside access to FOLIO via standard APIs. SIP2 is also developed as an Edge Module.



Notes from ERM Production Release Strategy Meeting



FOLIO-1729 - Getting issue details... STATUS

  • Integration with etcd; what are the requirements; are there issues ?

Action items

  •