marc4j overview
- Oleksii Kuzminov
- Kruthi Vuppala
- Pavel Korolenok
MARC4J provides an easy to use API for working with MARC (binary), MARCXML, MARC JSON in Java. MARC stands for MAchine Readable Cataloging and is a widely used exchange format for bibliographic data. MARCXML provides a loss-less conversion between MARC (MARC21 but also other formats like UNIMARC) and XML.
Features
The MARC4J library includes:
- An easy to use interface that can handle large record sets.
- Readers and writers for both MARC and MARCXML.
- A build-in pipeline model to pre- or post-process MARCXML using XSLT stylesheets.
- A MARC record object model (like DOM for XML) for in-memory editing of MARC records.
- Support for data conversions from MARC-8 ANSEL, ISO5426 or ISO6937 to UCS/Unicode and back.
- A forgiving reader which can handle and recover from a number of structural or encoding errors in records.
- Implementation independent XML support through JAXP and SAX2, a high performance XML interface.
- Support for conversions between MARC and MARCXML.
- Tight integration with the JAXP, DOM and SAX2 interfaces.
- Easy to integrate with other XML interfaces like DOM, XOM, JDOM or DOM4J.
- Command-line utilities for MARC and MARCXML conversions.
- Javadoc documentation.
MARC4J provides readers and writers for MARC and MARCXML. A org.marc4j.MarcReader implementation parses input data and provides an iterator over a collection of org.marc4j.marc.Record objects. The record object model is also suitable for in-memory editing of MARC records, just as DOM is used for XML editing purposes. Using a org.marc4j.MarcWriter implementation it is possible to create MARC or MARCXML. Once MARC data has been converted to XML, you can further process the result with XSLT, for example to convert MARC to MODS .
Although MARC4J is primarily designed for Java development, you can use the command-line utilities org.marc4j.util.MarcXmlDriver and org.marc4j.util.XmlMarcDriver to convert between MARC and MARCXML. It is also possible to pre- or postprocess the result using XSLT, for example to convert directly from MODS to MARC or from MARC to MODS.
Parsing records
For parse records and interact with them you need to use two main Interfaces:
- MarcReader
- Record
Readers
For reading records this lib provide simple interface org.marc4j.MarcReader with many implementations for all types and formats of records.
Records
The main entity that describes all types of records is org.marc4j.marc.Record. It represents a MARC document as a plain java object with fields. Each MARC record after parsing has such fields:
- Leader leader;
- List<ControlField> controlFields
- List<DataField> dataFields
- List<MarcError> errors
Writers
Marc4j lib testing experience
Types
For testing this library, we used all types of MARC records. These records can be found here.
Types of records were tested:
- MARC Authority
- MARC Bibliographic
- MARC Holdings
- Mixed MARC Bib+Hld
Marc4j successfully parsed all types and converted them into the list of internal Record entities. For mixed records it works fine, but we can not select from the parsed list the necessary records by its type. It is possible only by analyzing record's fields. For the first iteration of Batch Loader, different types of MARC records should be in separate files.
Encoding and formats
During testing marc4j lib we used records with these formats:
- XML
- Binary *.mrc files
Binary records were encoded:
- UTF-8
- MARC 8
During parsing records, marc4j decode all data from records into UTF-8 format. It works nice for all types and encodings and it understands non-Latin symbols. After parsing, the data in the list of records is stored in UTF-8 encoding.
Examples
Source record
LEADER 00508cjm a22001813 4500
001 10062588
005 20171013073237.0
007 sd fsngnnmmneu
008 170825s2017 xx nn n zxx d
024 7 $a00190295755553$2gtin-14
024 1 $a190295755553
035 $a(OCoLC)1002130878
035 $a10062588
040 $aBTCTA$beng$cBTCTA
100 1 $aRossi, Daniele
245 00$aSaint-Saens: Organ Symphony and Carnival of The Animals
260 $bWea Corp$c2017.
948 2 $a20171013$bm$dbatch$elts$xdeloclcprefix
JSON representation of marc record entity
{
"leader": "00508cjm a22001813 4500",
"fields": [
{
"001": "10062588"
},
{
"005": "20171013073237.0"
},
{
"007": "sd fsngnnmmneu"
},
{
"008": "170825s2017 xx nn n zxx d"
},
{
"024": {
"subfields": [
{
"a": "00190295755553"
},
{
"2": "gtin-14"
}
],
"ind1": "7",
"ind2": " "
}
},
{
"024": {
"subfields": [
{
"a": "190295755553"
}
],
"ind1": "1",
"ind2": " "
}
},
{
"035": {
"subfields": [
{
"a": "(OCoLC)1002130878"
}
],
"ind1": " ",
"ind2": " "
}
},
{
"035": {
"subfields": [
{
"a": "10062588"
}
],
"ind1": " ",
"ind2": " "
}
},
{
"040": {
"subfields": [
{
"a": "BTCTA"
},
{
"b": "eng"
},
{
"c": "BTCTA"
}
],
"ind1": " ",
"ind2": " "
}
},
{
"100": {
"subfields": [
{
"a": "Rossi, Daniele"
}
],
"ind1": "1",
"ind2": " "
}
},
{
"245": {
"subfields": [
{
"a": "Saint-Saens: Organ Symphony and Carnival of The Animals"
}
],
"ind1": "0",
"ind2": "0"
}
},
{
"260": {
"subfields": [
{
"b": "Wea Corp"
},
{
"c": "2017."
}
],
"ind1": " ",
"ind2": " "
}
},
{
"948": {
"subfields": [
{
"a": "20171013"
},
{
"b": "m"
},
{
"d": "batch"
},
{
"e": "lts"
},
{
"x": "deloclcprefix"
}
],
"ind1": "2",
"ind2": " "
}
}
]
}Code snippet
to parse MARC file and print all information to console
public class App {
public static void main(String args[]) throws Exception {
// path to .mrc file
String path = "";
// Input Stream from file
InputStream in = new FileInputStream(path);
MarcReader reader = new MarcStreamReader(in);
while (reader.hasNext()) {
Record record = reader.next();
Leader leader = record.getLeader();
List<ControlField> controlFields = record.getControlFields();
List<DataField> dataFields = record.getDataFields();
System.out.println();
System.out.println("LEADER: " + leader);
System.out.println("Control fields: ");
controlFields.forEach(System.out::println);
System.out.println("Data fields: ");
dataFields.forEach(System.out::println);
System.out.println();
}
}
}
Transform MARCJson to MARCXML.
Code snippet
convert MARCJson to MARCXML and print to console:InputStream fileInputStream = new FileInputStream(INPUT_JSON_FILE); MarcReader marcJsonReader = new MarcJsonReader(fileInputStream); StringWriter writer = new StringWriter(); MarcXmlWriter marcXmlWriter = new MarcXmlWriter(System.out, true); while (marcJsonReader.hasNext()) { Record record = marcJsonReader.next(); marcXmlWriter.write(record); } marcXmlWriter.close(); fileInputStream.close();
If you need other xml format output, ou can post-process the result using a Source object pointing to a stylesheet resource and a Result object to hold the transformation result tree.The example below converts MARCJson to MARC XML and transforms the result tree to Dublin Core using the stylesheet provided by The Library of Congress:
From Json to Dublin Core dataString stylesheetUrl = "https://www.loc.gov/standards/marcxml/xslt/MARC21slim2OAIDC.xsl"; Source stylesheet = new StreamSource(stylesheetUrl); Result result = new StreamResult(System.out); InputStream input = new FileInputStream(INPUT_JSON_FILE); MarcReader reader = new MarcJsonReader(input); MarcXmlWriter writer = new MarcXmlWriter(result, stylesheet); writer.setConverter(new AnselToUnicode()); while (reader.hasNext()) { Record record = (Record) reader.next(); writer.write(record); } writer.close();
Testing Marc4j writer
- Reading a Marc Json and printing in MARC format. Observation was that if encoding was not used, the MARC generated was invalid
import java.io.FileInputStream;
import java.io.InputStream;
import org.marc4j.*;
import org.marc4j.marc.Record;
public class TestMarc4jWriter {
public static void main(String args[]) throws Exception {
# The json file used here is referenced above
InputStream input = new FileInputStream("MarcFileinjsonFormat");
MarcReader reader = new MarcJsonReader(input);
MarcWriter writer = new MarcStreamWriter(System.out, "UTF8");
while (reader.hasNext()) {
Record record = reader.next();
writer.write(record);
}
writer.close();
}
}
OUTPUT
01741nam a2200373 cb4500001001100000003000700011005001700018008004100035020003000076020003700106020005000143035002400193035002100217035002600238040002000264041000800284245008900292264003700381300003700418336002600455337004600481338002500527490009600552500002500648650007700673650006300750650007800813700007700891700003900968830012301007856012101130900005301251954006301304101073931XDE-60120180416162657.0180111s2018\\\\sz\\\\\\\\\\\\000\0\eng\d\\a331964399193-319-64399-1\\a97833196439919978-3-319-64399-1\\a9783319644004 (electronic)9978-3-319-64400-4\\a(OCoLC)ocn992783736\\a(OCoLC)992783736\\a(DE-599)GBV101073931X\\bgercGBVCPerda0\aeng00aFutures, biometrics and neuroscience researchcLuiz Moutinho, Mladen Sokele, editors31aChambPalgrave Macmillanc[2018]\\axxix, 224 SeitenbIllustrationen\\aTextbtxt2rdacontent\\aohne Hilfsmittel zu benutzenbn2rdamedia\\aBandbnc2rdacarrier0\aInnovative research methodologies in managementv / Luiz Moutinho, Mladen Sokele ; Volume 2\\aEnthält 9 Beiträge\781.1\xaBetriebswirtschaftslehre0(DE-601)0913513910(DE-STW)12041-52stw\781.2\xaManagement0(DE-601)0913761730(DE-STW)12085-62stw\781.3\xaWissenschaftliche Methode0(DE-601)0914014450(DE-STW)16727-02stw1\aMoutinho, LuizeHerausgeberIn4edt0(DE-601)5094509540(DE-588)1314502041\aSokele, MladeneHerausgeberIn4edt\0aInnovative research methodologies in managementb / Luiz Moutinho, Mladen SokelevVolume 292.2018w(DE-601)101138029342yInhaltsverzeichnisuhttp://www.gbv.de/dms/zbw/101073931X.pdfmV:DE-601;B:DE-206qapplication/pdf3Inhaltsverzeichnis\\aGBVbZBW Kiel <206>d!H:! A18-1775xLzLCs206/1\\0ZBW Kiel <206>a26b1740761685c01fH:dA18-1775eux206/1
2. Defining fields using Record
import org.marc4j.MarcStreamWriter;
import org.marc4j.MarcWriter;
import org.marc4j.marc.MarcFactory;
import org.marc4j.marc.Record;
public class TestMarc4jWriter2 {
public static void main(String args[]){
MarcWriter writer = new MarcStreamWriter(System.out, "UTF8");
MarcFactory factory = MarcFactory.newInstance();
Record record = factory.newRecord("00714cam a2200205 a 4500");
record.addVariableField(factory.newControlField("001", "12883376"));
record.addVariableField(factory.newControlField("005", "20030616111422.0"));
record.addVariableField(factory.newControlField("008", "020805s2002 nyu j 000 1 eng "));
record.addVariableField(factory.newDataField("020", ' ', ' ', "a", "0786808772"));
record.addVariableField(factory.newDataField("020", ' ', ' ', "a", "0786816155 (pbk.)"));
record.addVariableField(factory.newDataField("040", ' ', ' ', "a", "DLC", "c", "DLC", "d", "DLC"));
record.addVariableField(factory.newDataField("100", '1', ' ', "a", "Chabon, Michael."));
record.addVariableField(factory.newDataField("245", '1', '0', "a", "Summerland /", "c", "Michael Chabon."));
record.addVariableField(factory.newDataField("250", ' ', ' ', "a", "1st ed."));
record.addVariableField(factory.newDataField("260", ' ', ' ', "a", "New York :", "b", "Miramax Books/Hyperion Books for Children,", "c", "c2002."));
record.addVariableField(factory.newDataField("300", ' ', ' ', "a", "500 p. ;", "c", "22 cm."));
record.addVariableField(factory.newDataField("520", ' ', ' ', "a", "Ethan Feld, the worst baseball player in the history of the game, finds himself recruited by a 100-year-old scout to help a band of fairies triumph over an ancient enemy."));
record.addVariableField(factory.newDataField("650", ' ', '1', "a", "Fantasy."));
record.addVariableField(factory.newDataField("650", ' ', '1', "a", "Baseball", "v", "Fiction."));
record.addVariableField(factory.newDataField("650", ' ', '1', "a", "Magic", "v", "Fiction."));
writer.write(record);
writer.close();
}
}
00714cam a2200205 a 45000010009000000050017000090080041000260200015000670200022000820400018001041000021001222450034001432500012001772600067001893000021002565200175002776500013004526500023004656500020004881288337620030616111422.0020805s2002 nyu j 000 1 eng a0786808772 a0786816155 (pbk.) aDLCcDLCdDLC1 aChabon, Michael.10aSummerland /cMichael Chabon. a1st ed. aNew York :bMiramax Books/Hyperion Books for Children,cc2002. a500 p. ;c22 cm. aEthan Feld, the worst baseball player in the history of the game, finds himself recruited by a 100-year-old scout to help a band of fairies triumph over an ancient enemy. 1aFantasy. 1aBaseballvFiction. 1aMagicvFiction.
Validating a MARC file locally
An opensource tool MARCEDIT can be used for validating MARC files, and also for conversion
Reference: https://marcedit.reeset.net/
REFERENCES:
https://github.com/marc4j/marc4j
Formateed Tutorial: http://projects.freelibrary.info/freelib-marc4j/tutorial.html