Skip to end of banner
Go to start of banner

2018-01-25 - System Operations and Management SIG Notes

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Date

Attendees

  • Ingolf Kuss

  • Craig Boman

  • Anne Highsmith
  • Chris Creswell
  • Chris Manly
  • Dale Arntson
  • Lina Lakhia
  • Stephen Pampell
  • Tod Olson

Goals

Discussion items

TimeItemWhoNotes
5WelcomeIngolf

Assign today's note taker

20Data migrationIngolf
  • do we want to build a subgroup that is solely concerned with questions of data migration (and meets outside of the regular SIG meetings) ?
  • what aspects of data migration will the SIG be responsible for ?
  • what should be delivered by the SIG ?
30Installation & DeploymentIngolf + group
  • status and plans of institutional installations (for testing and production)
    • plans for data migration
  • existing deployment documentation
  • Specifications for migration tools that have not already been identified for development (from SIG charge)
5Topics for next meetingIngolf

Notes

Notes:

  1. Chris wants to reach out to Madeleine (who is in Madrid for the conference) : Ongoing

  2. Deferred: notes on breakout session in Madrid

  3. Data migration: sub group?

    1. May require its own charge as a subgroup of Systems SIG

    2. Operational efforts take much discuss on its own

    3. Membership? Timing?

      1. Three zoom accounts have some simultaneous uses

      2. Look for Doodle poll for possible from Chris Manly

    4. Charge? Deliverables?

      1. Assuming data migration works as APIs, subgroup may define adequacy and best practices for migrations

      2. Record layouts of particular records?

        1. This may be a responsibility as well

      3. How to get data out in an acceptable format?

      4. Which record types will users want to migrate?

      5. First step may be to define what questions for migration

        1. To define whether migration is possible or starting over is necessary

      6. How many modules are API defined for migration?

        1. Smaller at the moment

        2. Schedule for module/API creation

    5. Ingolf Kuss: will create wiki page with charge notes for migration subgroup

    6. Membership of subgroup may be comprised of membership from other SIGs, or at least have some liaisons to the different SIGs

      1. Could send out call for data migration memberships to other SIGs

      2. Membership could prioritize groups who are currently migrating

  4. Installation and documentation for deployment?

    1. Wayne have some notes, which are developer oriented

    2. Chris M. has started to deploy test instance on AWS without Docker

    3. We should encourage each other to keep and share documentation as we attempt setup

    4. Stripes: does it go with Okapi?

      1. Set of static documents to serve via web server

    5. Vagrant Security:

      1. Relying on internal security of vagrant box

      2. Weak public/private key; lacks proper firewall (not setup for public exposure)

    6. Https/ssl (appears to be http currently):

      1. We are not sure https is the baseline for developers, which we will need

    7. AWS:

      1. Would be great to have new rather than persistent demo instance

      2. Avoid stepping on each others’ toes

      3. We don’t know what the resources requirements are to setup in AWS for pricing (load requirements unsure)

      4. Need to load in 70,000 users and 8 million bibs to see how much RAM we need (storage and CPU should be less of an issue when pricing AWS)

      5. Data transfer chargers may be an issue

  5. Specifications for migration tools

    1. Python may be a strong migration tool

    2. ETLs to OLE: didn’t seem to be sufficiently flexible (need to do data cleanup)

      1. Shell scripting and java has been used

      2. Perl loading directly to mysql database has been used as well to OLE

    3. APIs: some concerns that APIs may be too slow

      1. Developers are committed to providing support for sufficient APIs in Folio

    4. More experimentation is needed using different potential migration tools

  6. Next meeting topics:

    1. Review Madrid developers meeting (could be sufficient agenda)

    2. New members joining Systems SIG next week (according to Wayne)

Action items

  •  
  • No labels