Page Comparison

...

mod-data-export-worker is intended to receive events from mod-data-export-spring,andexecute its jobs. The module is built based on Spring Batch Framework, and jobs are configured by a set of steps. The execution of a job happens in 3-stages: retrieve data, process data, and write data to the temporary file. Uploading files to some vendor-specific storage (AWS S3 bucket) is preconfigured already by the listener and happens when the file is completely written.

Image Added
What should be done in this module:
      - create a Reader extending base functionality (CsvItemReader.java, see CirculationLogCsvItemReader.java as an example). The reader should retrieve packages/titles using REST clients, taking search parameters from the incoming Kafka event (from job parameters);
      - create a Processor (implementing ItemProcessor). The processor has to take only selected fields for export from the incoming packages/titles. The list of fields for export comes from job parameters; [Use 2 processors]
      - create a Writer extending base functionality (we can just use CsvWriter.java if nothing special is needed);
      - create a Configuration to build a job and set Reader, Writer, and Processor (see CirculationLogJobConfig.java);