<!-- 
RSS generated by JIRA (1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d) at Fri Feb 09 00:37:08 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary add field=key&field=summary to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>FOLIO Jira</title>
    <link>https://folio-org.atlassian.net</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>1001.0.0-SNAPSHOT</version>
        <build-number>100246</build-number>
        <build-date>07-02-2024</build-date>
    </build-info>

<item>
            <title>[UXPROD-4082] Long term solution for applying mapping rules change to authority records that exist in database. </title>
                <link>https://folio-org.atlassian.net/browse/UXPROD-4082</link>
                <project id="10000" key="UXPROD">UX Product</project>
                    <description>&lt;p&gt;&lt;b&gt;Problem:&lt;/b&gt; Tremendous difficulty to&#160; upgrade a large set of MARC authority records when mapping rules change (per release) have been applied.&#160; With the Nolana release, a short term solution was applied &lt;a href=&quot;https://folio-org.atlassian.net/wiki/display/DD/ARCH-36+Provide+a+way+to+update+MARC+authority+records+when+mapping+rules+have+changed&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://folio-org.atlassian.net/wiki/display/DD/ARCH-36+Provide+a+way+to+update+MARC+authority+records+when+mapping+rules+have+changed&lt;/a&gt; but has proven unreliable and not scalable.&#160;&#160;&lt;/p&gt;

&lt;p&gt;Until this issue is addressed, many libraries will not be able to use the MARC authority app and authority control effectively. Note Library of Congress has over 10 million authority records.&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;The proposed solution should&lt;/b&gt;&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;be discussed with Folijet or developed in collaboration with this team because ideally it should be the solution (serve as the pattern) for bib/holdings/item records of any format.&#160;&lt;/li&gt;
	&lt;li&gt;consider concurrency level change to minimize performance and/or reliability degradation&lt;/li&gt;
	&lt;li&gt;support the following environment setups
	&lt;ul&gt;
		&lt;li&gt;Self-hosted?&lt;/li&gt;
		&lt;li&gt;Single tenant + single cluster&#160;&#160;&lt;/li&gt;
		&lt;li&gt;Enhanced consortia support&#160;&lt;/li&gt;
		&lt;li&gt;Multi-tenant + single cluster &amp;gt; if one or more tenants is processing a very large dataset or entire data set then other tenants should continue to operate without significant latency or delays. &lt;span class=&quot;error&quot;&gt;&amp;#91;Since stand-alone solution &amp;gt; shared database. Performance degradation will occur but unsure of impact because it is a database update. No use of folio modules in this operation. Will need to measure via PTF testing.&amp;#93;&lt;/span&gt;&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;not impact significantly deteriorate a.) Check-in Check out (CICO), b.) data import (for example can a library still run data import jobs for orders? bibs? holdings? items? c.) access to Inventory and MARC authority workflows&#160;&#160;&lt;/li&gt;
	&lt;li&gt;&lt;b&gt;Stats&lt;/b&gt;&#160;
	&lt;ul&gt;
		&lt;li&gt;&#160;Action: Mapping rules change
		&lt;ul&gt;
			&lt;li&gt;
			&lt;ul&gt;
				&lt;li&gt;500,000 records are updated []&lt;/li&gt;
				&lt;li&gt;1 million records are updated []&lt;/li&gt;
				&lt;li&gt;5 million records are updated []&lt;/li&gt;
				&lt;li&gt;10 million records are updated []&lt;/li&gt;
				&lt;li&gt;20 million records can updated []&lt;/li&gt;
			&lt;/ul&gt;
			&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;Ability to have these migrations executed in the background with minimal impact to FOLIO usage.&#160;&lt;/li&gt;
	&lt;li&gt;Implement
	&lt;ul&gt;
		&lt;li&gt;slicing solution similar to data import implementation 
    &lt;span class=&quot;jira-issue-macro resolved&quot; data-jira-key=&quot;UXPROD-4337&quot; &gt;
                &lt;a href=&quot;https://folio-org.atlassian.net/browse/UXPROD-4337&quot; class=&quot;jira-issue-macro-key issue-link&quot;  title=&quot;Reliably process large files in DI by automatically splitting and processing source files&quot; &gt;
            &lt;img class=&quot;icon&quot; src=&quot;https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10322?size=medium&quot; /&gt;
            UXPROD-4337
        &lt;/a&gt;
                                                    &lt;span class=&quot;aui-lozenge aui-lozenge-subtle aui-lozenge-success jira-macro-single-issue-export-pdf&quot;&gt;Closed&lt;/span&gt;
            &lt;/span&gt;
&lt;/li&gt;
		&lt;li&gt;Report on progress and status&#160;&lt;/li&gt;
		&lt;li&gt;Users should have a simple solution to get/receive files that contain records with errors (The records that were not processed successfully).&lt;/li&gt;
		&lt;li&gt;Log and report errors &amp;gt; Provide a response that includes details on records that failed to create or update in SRS and Inventory.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;


&lt;ul&gt;
	&lt;li&gt;
	&lt;ul&gt;
		&lt;li&gt;Support UTF-8 encoding.&lt;/li&gt;
		&lt;li&gt;Support optimistic locking mechanism.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;Very little manual intervention. Should be painless for hosting.&#160;&lt;/li&gt;
	&lt;li&gt;Applies to MARC 21 format&#160;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&lt;b&gt;View migration numbers for several libraries&lt;/b&gt; : &lt;a href=&quot;https://docs.google.com/spreadsheets/d/10GiFrfZee8aY8PcE0JJxf-lWtMkddFWnOYo_tiKYXrs/edit#gid=0&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/spreadsheets/d/10GiFrfZee8aY8PcE0JJxf-lWtMkddFWnOYo_tiKYXrs/edit#gid=0&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Definition of done&lt;/b&gt;&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;Load testing by development team&#160;&#160;Scenario: Existing customer that already uses MARC authority app: Mapping rules change&#160;&lt;/li&gt;
	&lt;li&gt;Verify handling of errors&#160;&lt;/li&gt;
	&lt;li&gt;PTF testing Scenario: Existing customer that already uses MARC authority app: Mapping rules change&#160;&lt;/li&gt;
	&lt;li&gt;Production-like testing: Scenario: Existing customer that already uses MARC authority app: Mapping rules change&#160;&#160;&lt;/li&gt;
	&lt;li&gt;Release notes and documentation&#160;&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="13764">UXPROD-4082</key>
            <summary>Long term solution for applying mapping rules change to authority records that exist in database. </summary>
                <type id="10002" iconUrl="https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10322?size=medium">New Feature</type>
                            <parent id="11927">UXPROD-2944</parent>
                                    <priority id="10000" iconUrl="https://dev.folio.org/assets/jira-priority/jira-p1.svg">P1</priority>
                        <status id="3" iconUrl="https://folio-org.atlassian.net/images/icons/statuses/inprogress.png" description="This issue is being actively worked on at the moment by the assignee.">In Progress</status>
                    <statusCategory id="4" key="indeterminate" colorName="yellow"/>
                                    <resolution id="-1">Unresolved</resolution>
                                                        <assignee accountid="62a96ae7192edb006f9f1bf9">Khalilah Gambrell</assignee>
                                                                <reporter accountid="62a96ae7192edb006f9f1bf9">Khalilah Gambrell</reporter>
                                    <labels>
                            <label>LC-priority2</label>
                            <label>NFR</label>
                            <label>SolutionArchitecture</label>
                            <label>arlef-di</label>
                            <label>authority</label>
                            <label>back-end</label>
                            <label>cataloging</label>
                            <label>data-import</label>
                            <label>di-swat</label>
                            <label>loc</label>
                            <label>marc-authority</label>
                            <label>metadatamanagement</label>
                    </labels>
                <created>Mon, 20 Feb 2023 17:49:05 +0000</created>
                <updated>Mon, 5 Feb 2024 11:15:48 +0000</updated>
                                                            <fixVersion>Quesnelia (R1 2024)</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>8</watches>
                                                                <comments>
                                                            <comment id="18281" author="62a96ae7192edb006f9f1bf9" created="Tue, 28 Mar 2023 13:10:38 +0000"  >&lt;p&gt;Hey &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ade4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; class=&quot;user-hover&quot; rel=&quot;557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; data-account-id=&quot;557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; accountid=&quot;557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; rel=&quot;noreferrer&quot;&gt;Taras Spashchenko&lt;/a&gt; and &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5eb44b5a96bbcb0b8585c98d&quot; class=&quot;user-hover&quot; rel=&quot;5eb44b5a96bbcb0b8585c98d&quot; data-account-id=&quot;5eb44b5a96bbcb0b8585c98d&quot; accountid=&quot;5eb44b5a96bbcb0b8585c98d&quot; rel=&quot;noreferrer&quot;&gt;Pavlo Smahin&lt;/a&gt;&#160;&lt;/p&gt;

&lt;p&gt;Here is a note from University of Missouri related to this feature&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Has anyone been working with authority control vendors, in terms of sending/reloading records, and have you used scripts to handle it in Data Import? The University of Missouri system has been trying to figure out how to navigate this based on DI functionality in Nolana and an efficient way to reload ~4-4.5M bib records isn&apos;t really clear to us. We&apos;re less concerned with the potential of quarterly updates but figuring out how to do the initial sync is causing us some headaches.&lt;/em&gt;&lt;/p&gt;</comment>
                                                            <comment id="18282" author="62857cd58133bb006f6ddd0a" created="Tue, 28 Mar 2023 13:59:25 +0000"  >&lt;p&gt;This is also something that Wellesley College is interested in. We are sending our bibliographic records to our authority control vendor in late May/early June for updating and creating our new base file with FOLIO identifiers. This will be the first time we have done authority control post-migration to FOLIO in June 2022. We need to be able to import and update our bibliographic records with the updates when they are returned to us. We are expecting over 674,000 bibliographic records to be returned. We will also get the authority files that go with those records. We are concerned about how to get these records back into FOLIO in a timely and accurate manner considering the limitations of Data Import.&lt;/p&gt;</comment>
                                                            <comment id="18283" author="557058:5ceb2fe3-7fa0-4074-babb-a5941cdcf312" created="Tue, 28 Mar 2023 16:40:22 +0000"  >&lt;p&gt;Regular quarterly updates from an authority vendor can also be very large. Last year Library of Congress changed the URLs in subfield $0 in authorized fields. This required all bib records in everyone&apos;s databases to be replaced.&lt;/p&gt;</comment>
                                                            <comment id="18284" author="557058:5ceb2fe3-7fa0-4074-babb-a5941cdcf312" created="Tue, 28 Mar 2023 17:59:56 +0000"  >&lt;p&gt;Authority vendors also sometimes send files of authority records that need to be deleted. There needs to be a mechanism to load a file of MARC authority records, and delete them in FOLIO.&lt;/p&gt;</comment>
                                                            <comment id="18285" author="557058:5ceb2fe3-7fa0-4074-babb-a5941cdcf312" created="Mon, 15 May 2023 22:40:35 +0000"  >&lt;p&gt;To be clear, this would need a lot of the features of DI. It would need field protection for example. In the case of getting new bib records from an authority vendor, I would want to replace the fields that the vendor updates without replacing other fields that might have local changes I don&apos;t want to overlay. Different vendors may update different fields and each vendor has many options for what fields they change and what they don&apos;t. It would really have to be very flexible. It&apos;s not just a total record replacement.&lt;/p&gt;</comment>
                                                            <comment id="18286" author="5af5e627525ba96b58654f12" created="Wed, 17 May 2023 03:52:46 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3A5ceb2fe3-7fa0-4074-babb-a5941cdcf312&quot; class=&quot;user-hover&quot; rel=&quot;557058:5ceb2fe3-7fa0-4074-babb-a5941cdcf312&quot; data-account-id=&quot;557058:5ceb2fe3-7fa0-4074-babb-a5941cdcf312&quot; accountid=&quot;557058:5ceb2fe3-7fa0-4074-babb-a5941cdcf312&quot; rel=&quot;noreferrer&quot;&gt;Lloyd Chittenden&lt;/a&gt; for the locally-changed fields that need to be protected, how would those be identified? Would they have a certain subfield present, e.g. $4localcode? Would it be particular MARC fields that are protected? It would be really helpful to have a sample MARC Authority file with some of those locally-changed fields. Would it be possible to attach that to this Jira? Thank you!&lt;/p&gt;</comment>
                                                            <comment id="18287" author="557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52" created="Wed, 31 May 2023 08:33:33 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=62a96ae7192edb006f9f1bf9&quot; class=&quot;user-hover&quot; rel=&quot;62a96ae7192edb006f9f1bf9&quot; data-account-id=&quot;62a96ae7192edb006f9f1bf9&quot; accountid=&quot;62a96ae7192edb006f9f1bf9&quot; rel=&quot;noreferrer&quot;&gt;Khalilah Gambrell&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;I am reassigning this feature back to you. The design &lt;a href=&quot;https://folio-org.atlassian.net/browse/ARCH-46&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://folio-org.atlassian.net/browse/ARCH-46&lt;/a&gt; has been completed.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10001">
                    <name>Cloners</name>
                                                                <inwardlinks description="is cloned by">
                                        <issuelink>
            <issuekey id="12962">UXPROD-4533</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10008">
                    <name>Defines</name>
                                                                <inwardlinks description="is defined by ">
                                        <issuelink>
            <issuekey id="79271">MODMARCMIG-4</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="79268">MODMARCMIG-11</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10006">
                    <name>Gantt End to Start</name>
                                            <outwardlinks description="has to be done before">
                                        <issuelink>
            <issuekey id="12754">UXPROD-3969</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10003">
                    <name>Relates</name>
                                            <outwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="79267">MODMARCMIG-9</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="79270">MODMARCMIG-5</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="79269">MODMARCMIG-6</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="34819">MODELINKS-173</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="79274">MODMARCMIG-7</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="79272">MODMARCMIG-8</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10007">
                    <name>Requires</name>
                                            <outwardlinks description="requires">
                                        <issuelink>
            <issuekey id="34850">MODELINKS-84</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="34857">MODELINKS-92</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="56556">MODINVSTOR-1057</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="56561">MODINVSTOR-1068</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Back End Estimate</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10125"><![CDATA[XXXL: 30-45 days]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10054" key="com.atlassian.jira.plugin.system.customfieldtypes:userpicker">
                        <customfieldname>Back End Estimator</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>62a96ae7192edb006f9f1bf9</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10055" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Back-End Confidence factor</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10137"><![CDATA[100%]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummarycf">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10057" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Development Team</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10180"><![CDATA[Spitfire]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10014" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue key="$xmlutils.escape($text)">MARC Authority - Phase 1 Features</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10050" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Front End Estimate</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10094"><![CDATA[Out of scope]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10051" key="com.atlassian.jira.plugin.system.customfieldtypes:userpicker">
                        <customfieldname>Front End Estimator</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>62a96ae7192edb006f9f1bf9</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10063" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>PO Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10019" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0|i06cl4:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10046" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Release</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10076"><![CDATA[Poppy (R2 2023)]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10020" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10024" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>[CHART] Date of First Response</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 28 Mar 2023 13:59:25 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                </customfields>
    </item>
</channel>
</rss>