All work
- ETL: Update Translate_Bibliographic DAG to invoke new Java functionMODLD-456
- ETL: Enhance Airflow pipeline to support AuthoritiesMODLD-454
- ETL: Create (or Enhance) Java function for MARC to Graph translation in Translate_Authority stepMODLD-453
- ETL: Update ETL to insert records into "instance_metadata" tableMODLD-449
- ETL : Create Java function for MARC to Graph translation in Translate_Bibliograph stepMODLD-447
- Check Spring Native as a core of the Translate Module for the ETLMODLD-384Punnoose Kutty Jacob Pullolickal
- ETL: Check Spring as a core for the Translate Module of the ETLMODLD-358
- ETL: Check Quarqus as a core for Translate Module of the ETLMODLD-357Punnoose Kutty Jacob Pullolickal
- Implement Folio mod-linked-data graph rebalancingMODLD-246David Feeney
- Java Library - Carrier type URI point to an incorrect linkMODLD-233Resolved issue: MODLD-233Punnoose Kutty Jacob Pullolickal
- Save Inventory identifier in Linked Data Storage in the FOLIO flowsMODLD-169Resolved issue: MODLD-169Pavel Bobylev
- ETL - Carrier type URI prefix seems to be wrongMODLD-165David Feeney
- ETL: Improve performance of pipeline-marc pipelineMODLD-163David Feeney
- PLACEHOLDER: Identify topics on schema architecture that are needed for documentationMODLD-147
- Save SRS identifiers in Linked Data Storage in the FOLIO flowsMODLD-141Resolved issue: MODLD-141Pavel Bobylev
- Spike: Save ETL output in Linked Data Schema without using temporary DB schemasMODLD-137Resolved issue: MODLD-137David Feeney
- Allow selecting target DB in the ETLMODLD-136Resolved issue: MODLD-136
- Remove unnecessary nodes creation in Instance mappingMODLD-120Pavel Bobylev
- ETL: Save Inventory and SRS identifiers in Linked Data Storage in the ETLMODLD-109David Feeney
- Add all missing BIBFRAME 2.0 Instance fields into ETL transformation to BIBFRAME LiteMODLD-98Resolved issue: MODLD-98Gloria Gonzalez
- Copy ETL CLI and Migrations modules into Linked Data repositoriesMODLD-97Resolved issue: MODLD-97Victor Bodrov
- Index Linked Data via Kafka in Search Index (temporary solution)MODLD-95Resolved issue: MODLD-95Punnoose Kutty Jacob Pullolickal
- Convert Item records from MARC to BIBFRAMEMODLD-92Resolved issue: MODLD-92
- Convert Work records from MARC to BIBFRAMEMODLD-91Resolved issue: MODLD-91Pavel Bobylev
- Convert Instance records from MARC to BIBFRAME in ETLMODLD-90Resolved issue: MODLD-90Pavel Bobylev
- ETL: Copy ETL debugfs module into Linked Data repositoriesMODLD-85
- Copy ETL TrashFS, PruneFS, Echoer, Mover modules into Linked Data repositoriesMODLD-84Resolved issue: MODLD-84Victor Bodrov
- Copy ETL Combiner, Statusfs, Workspace modules into Linked Data repositoriesMODLD-83Resolved issue: MODLD-83Punnoose Kutty Jacob Pullolickal
- Copy ETL Preprocessor Module source code into Linked Data repositoriesMODLD-80Resolved issue: MODLD-80Victor Bodrov
- Copy ETL LocalStack into Linked Data repositoriesMODLD-79Resolved issue: MODLD-79Victor Bodrov
- Copy ETL DAG files into Linked Data repositoriesMODLD-78Resolved issue: MODLD-78Victor Bodrov
- Copy ETL Translation Module source code into Linked Data repositoriesMODLD-77Resolved issue: MODLD-77Victor Bodrov
- Prepare the ETL for the demo sessionMODLD-70Resolved issue: MODLD-70Victor Bodrov
- SPIKE: Add a step to convert LC data for further processing in the ETLMODLD-67Resolved issue: MODLD-67Victor Bodrov
- ETL: Pass non-BIBFRAME fields through ETL to Linked Data StorageMODLD-65
- SPIKE: Update the ETL process to save files in the File StorageMODLD-61Resolved issue: MODLD-61
- SPIKE: Run the ETL to move data to Linked Data DatabaseMODLD-58Resolved issue: MODLD-58Victor Bodrov
37 of 37
ETL: Update Translate_Bibliographic DAG to invoke new Java function
Description
Environment
None
Potential Workaround
None
Checklist
hide
Created August 2, 2024 at 6:37 PM
Updated January 17, 2025 at 12:48 AM
Activity
Show:
Overview
Purpose of this story is to update the Translate_Bibliograph Step of ETL pipeline to invoke the new Java function created by .
Scope
Create Dockerfile for exposing the java function as AWS Batch job definition
Work with Platform team () to create new AWS Batch Job definition. Create a separate Rally ticket for this.
Specify the new docker image name in this line of the Python code.
At present, the params passed from DAG to Job definition is specific to the Python translate module. Update this line of the code as follows so that params required by Java Translate function is passed.