[UXPROD-949] Rewrite Cornell's Web based LS-Tools Created: 14/Jun/18 Updated: 18/Apr/23 Resolved: 18/Apr/23 |
|
| Status: | Closed |
| Project: | UX Product |
| Components: | None |
| Affects versions: | None |
| Fix versions: | Q4 2019 |
| Type: | Epic | Priority: | P3 |
| Reporter: | Hkaplanian | Assignee: | Unassigned |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | sysops_mgt | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||||||||||
| Issue links: |
|
||||||||||||||||||||||||||||||||
| Epic Name: | Rewrite Cornell's Web based LS-Tools (batch ETL tool) | ||||||||||||||||||||||||||||||||
| Score: | 14.5 | ||||||||||||||||||||||||||||||||
| Start date (migrated): | |||||||||||||||||||||||||||||||||
| End date: | |||||||||||||||||||||||||||||||||
| Epic Color: | ghx-label-7 | ||||||||||||||||||||||||||||||||
| Description |
|
LS-Tools = A tool set at Cornell which allows technical services do library automation tasks through a common web interface. |
| Comments |
| Comment by Chris Manly [ 09/Jul/18 ] |
|
Here is my attempt to capture what LSTools is in a way that will facilitate figuring out what would be needed to support it in FOLIO: LSTools is a platform for automated library technical services, centered around batch processing of MARC metadata and feeding it into Cornell's installation of Voyager. It was not originally designed as such, but instead grew organically out of a collection of scripts and programs that did library automation tasks. The programmer responsible for those scripts realized that by putting a web interface in front of the automation jobs, they could be invoked by the technical services staff directly. Over time, patterns emerged in the types of jobs being run, and the programmer consolidated disparate scripts into fewer, more capable programs that could be adjusted via parameters for specific situations. In one way, LSTools can be viewed as a highly specialized ETL tool, in that it extracts, transforms, and loads data. It does not, however, play the role that ETL tools generally do in transforming data to feed a data warehouse. The most effective way to replicate the functionality of LSTools within FOLIO would be to identify and replicate its generalized capabilities as part of the platform, and then glue them together with an app that provides an optimized UI for managing those capabilities. Those capabilities are: 1. Advanced querying of metadata records. A key component of FOLIO is a custom program called 'Harvest' which uses flat-file extracts of binary MARC data from Voyager to do detailed searches of metadata. It can select records on multiple criteria (including the absence of certain fields, specific subfields, etc.). It can then output a list of record IDs, or specified fields from the records as tabular data. Replicating Harvest directly would not be desirable (and probably not even technically possible, as a specification for the original program does not exist. Previous attempts to re-implement the C program in perl did not succeed). A modern implementation could likely be accomplished with a solr or elasticsearch index of appropriate configuration. Some of these capabilities may already exist within FOLIO or be on the roadmap. The envisioned Workflow engine might provide the ability to drive jobs based on a schedule or a triggering event, and/or provide escalation response to a failed job. A general-purpose metadata loader could likely serve as the interface for loading records in this context as well. To the extent that these capabilities overlap with existing FOLIO tools, we should make use of existing functionality. |
| Comment by Ann-Marie Breaux (Inactive) [ 12/Jul/18 ] |
|
In Batch Loader subgroup mtg today, we were talking about LS Tools, Bulk Edit, and Open Refine. Chris Manly I linked LS Tools and Bulk Edit since whatever development happens on these apps/services seems like it should be done in the context of each other. Christie Thomas Also need to be taking scripts into account. And will need to think about how this may fit with Workflows app. |
| Comment by Philip Robinson [ 28/Mar/19 ] |
|
Screenshot of main screen in LSTools |
| Comment by Philip Robinson [ 28/Mar/19 ] |
|
LSTools "Business Analysis" screen describing the system's main functions |
| Comment by Philip Robinson [ 29/Mar/19 ] |
|
LSTools priorities spreadsheet from Cornell |
| Comment by Philip Robinson [ 29/Mar/19 ] |
|
LSTools / FOLIO critical needs statement from Cornell |
| Comment by Erin Nettifee [ 17/Apr/23 ] |
|
Thomas Trutt should this be closed? |
| Comment by Erin Nettifee [ 18/Apr/23 ] |
|
Per Thomas at Cornell, this can be closed. |