2018-01-25 - System Operations and Management SIG Notes

Date

25 Jan 2018

Attendees

• Ingolf Kuss

• Craig Boman

• Anne L. Highsmith
• Chris Creswell
• Chris Manly
• Dale Arntson
• Lina Lakhia
• spampell

• Tod Olson

• Craig McNally

Goals

Discussion items

Notes

Notes:

1. Chris wants to reach out to Katalin and Cate (who are in Madrid for the conference) : Ongoing

2. Deferred: notes on breakout session in Madrid

3. Data migration: sub group?

1. May require its own charge as a subgroup of Systems SIG

2. Operational efforts take much discuss on its own

3. Membership? Timing?

1. Three zoom accounts have some simultaneous uses

2. Look for Doodle poll for possible from Chris Manly

4. Charge? Deliverables?

1. Assuming data migration works as APIs, subgroup may define adequacy and best practices for migrations

2. Record layouts of particular records?

1. This may be a responsibility as well

3. How to get data out in an acceptable format?

4. Which record types will users want to migrate?

5. First step may be to define what questions for migration

1. To define whether migration is possible or starting over is necessary

6. How many modules are API defined for migration?

1. Smaller at the moment

2. Schedule for module/API creation

5. Ingolf Kuss: will create wiki page with charge notes for migration subgroup

6. Membership of subgroup may be comprised of membership from other SIGs, or at least have some liaisons to the different SIGs

1. Could send out call for data migration memberships to other SIGs

2. Membership could prioritize groups who are currently migrating

4. Installation and documentation for deployment?

1. Wayne have some notes, which are developer oriented

2. Chris M. has started to deploy test instance on AWS without Docker

3. We should encourage each other to keep and share documentation as we attempt setup

4. Stripes: does it go with Okapi?

1. Set of static documents to serve via web server

5. Vagrant Security:

1. Relying on internal security of vagrant box

2. Weak public/private key; lacks proper firewall (not setup for public exposure)

6. Https/ssl (appears to be http currently):

1. We are not sure https is the baseline for developers, which we will need

7. AWS:

1. Would be great to have new rather than persistent demo instance

2. Avoid stepping on each others’ toes

3. We don’t know what the resources requirements are to setup in AWS for pricing (load requirements unsure)

4. Need to load in 70,000 users and 8 million bibs to see how much RAM we need (storage and CPU should be less of an issue when pricing AWS)

5. Data transfer chargers may be an issue

5. Specifications for migration tools

1. Python may be a strong migration tool

2. ETLs to OLE: didn’t seem to be sufficiently flexible (need to do data cleanup)

1. Shell scripting and java has been used

2. Perl loading directly to mysql database has been used as well to OLE

3. APIs: some concerns that APIs may be too slow

1. Developers are committed to providing support for sufficient APIs in Folio

4. More experimentation is needed using different potential migration tools

6. Next meeting topics:

1. Review Madrid developers meeting (could be sufficient agenda)

2. New members joining Systems SIG next week (according to Wayne)

Action items

• Chris Manly : Create Doodle poll for subgroup meetings
• Ingolf Kuss : Collect migration subgroup charges on a new wiki page (in the SysOps space)