<!-- 
RSS generated by JIRA (1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d) at Fri Feb 09 00:18:45 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary add field=key&field=summary to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>FOLIO Jira</title>
    <link>https://folio-org.atlassian.net</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>1001.0.0-SNAPSHOT</version>
        <build-number>100246</build-number>
        <build-date>07-02-2024</build-date>
    </build-info>

<item>
            <title>[UXPROD-1815] support for upgrading schemas without complete reload of data (DB migrations) - PoC + devops guide</title>
                <link>https://folio-org.atlassian.net/browse/UXPROD-1815</link>
                <project id="10000" key="UXPROD">UX Product</project>
                    <description>&lt;p&gt;(Note: Q3 work include a PoC and initial devops migration guide, rollout and additional Platform functionality is postponed to Q4)&lt;/p&gt;

&lt;p&gt;Currently in FOLIO, if there is a change to the schema used by a storage module, there is no upgrade path for the data already in the module. As we now have test deployments and are trying to move towards production deployments, this will be critical for ongoing testing and operations.&lt;/p&gt;

&lt;p&gt;This lack has been identified as Technical Debt by the Technical Council.&lt;/p&gt;

&lt;p&gt;Some solution for schema upgrades and migrating data will be needed for all storage modules. The solution could include multiple components. For example, maybe one part of the solution is to include data updates the the Definition of Done for each development team, and maybe there&apos;s something that could be added to the core to make it easier to create these migrations. That&apos;s just for illustration. The important thing is that FOLIO needs a way to address address schema updates and corresponding data changes that can be consistent across modules.&lt;/p&gt;

&lt;p&gt;SysOps identified these sub-issues:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Ability to roll back an upgrade&lt;/li&gt;
	&lt;li&gt;Consider the wider scope of how upgrades will occur in FOLIO systems&lt;/li&gt;
	&lt;li&gt;Downtime considerations, can we limit or have no downtime for an application?&lt;/li&gt;
	&lt;li&gt;May need additional work in modules/Okapi/RMB&lt;/li&gt;
	&lt;li&gt;Would want schema upgrade included in Definition of Done&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&lt;b&gt;FOLIO June meeting TC tech debt slides:&lt;/b&gt;&lt;br/&gt;
&lt;a href=&quot;https://docs.google.com/presentation/d/1Cz5-xhvMdCdm7SXYLhIQE8KV9YIsQW1faNUIWmMANx8&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/presentation/d/1Cz5-xhvMdCdm7SXYLhIQE8KV9YIsQW1faNUIWmMANx8&lt;/a&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="11064">UXPROD-1815</key>
            <summary>support for upgrading schemas without complete reload of data (DB migrations) - PoC + devops guide</summary>
                <type id="10002" iconUrl="https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10322?size=medium">New Feature</type>
                            <parent id="11697">UXPROD-1814</parent>
                                    <priority id="10002" iconUrl="https://dev.folio.org/assets/jira-priority/jira-p3.svg">P3</priority>
                        <status id="6" iconUrl="https://folio-org.atlassian.net/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="green"/>
                                    <resolution id="10003">Done</resolution>
                                                        <assignee accountid="557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52">Taras Spashchenko</assignee>
                                                                <reporter accountid="557058:b787c0f1-34df-41c2-8585-c4aed346caec">Tod Olson</reporter>
                                    <labels>
                            <label>back-end</label>
                            <label>cap-mvp</label>
                            <label>platform-backlog</label>
                            <label>po-mvp</label>
                            <label>q3-2019</label>
                    </labels>
                <created>Wed, 29 May 2019 19:54:59 +0000</created>
                <updated>Wed, 16 Sep 2020 21:13:27 +0000</updated>
                            <resolved>Fri, 11 Oct 2019 19:24:50 +0000</resolved>
                                                    <fixVersion>Q3 2019</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>12</watches>
                                                                <comments>
                                                            <comment id="12502" author="557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d" created="Wed, 5 Jun 2019 12:58:50 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab787c0f1-34df-41c2-8585-c4aed346caec&quot; class=&quot;user-hover&quot; rel=&quot;557058:b787c0f1-34df-41c2-8585-c4aed346caec&quot; data-account-id=&quot;557058:b787c0f1-34df-41c2-8585-c4aed346caec&quot; accountid=&quot;557058:b787c0f1-34df-41c2-8585-c4aed346caec&quot; rel=&quot;noreferrer&quot;&gt;Tod Olson&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In general, I think the migrations can be grouped, roughly, into two groups:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;unattended automatic schema and data migrations &amp;#8211; migrations performed by a FOLIO module fully automatically during the module upgrade procedure. Unattended migrations do not require operator&apos;s intervention unless an error condition arises.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Example for mod-inventory-storage: a new field called &quot;notes&quot; is added to the Instance entity schema, the field is empty by default. Automatic migrations updates the DB schema (creates an index) and stored data. The migration is performed during the module upgrade callback.&lt;/p&gt;

&lt;p&gt;Problems (some) that needs to be addressed &lt;br/&gt;
  &amp;#8211; data consistency and integrity. The migration should be performed within a transaction to allow for rollbacks in case of errors. &lt;br/&gt;
  &amp;#8211; Error reporting and resiliency. Is it acceptable to report errors in response to the lifecycle callbacks or is there a requirement for queueing error reports?&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;migrations that require operator&apos;s attention &amp;#8211; migration that can&apos;t be performed fully automatically because they require operators intervention (validation/evaluation) or can&apos;t be performed in isolation by the module (require access to information or data that the module does not posses)&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Example for mod-inventory-storage: a new field called &quot;alternativeTitle&quot; is added to the Instance schema. The field is marked mandatory and has no default value. The field value should be populated by the data import module from MARC21 &quot;246$a&quot; &quot;Varying Form of Title&quot; field. The migration can&apos;t be performed automatically by the module without access to additional information (original MARC record).&lt;/p&gt;

&lt;p&gt;Strategy 1: module performs automatic migration of schema (creates an index) and data. The value of the new field is set to a distinguishable placeholder value e.g &quot;NEEDS IMPORT&quot;. The placeholder value is overridden by performing a manual data import, using FOLIO&apos;s data import functionality.&lt;/p&gt;

&lt;p&gt;Strategy 2: module performs automatic migration of schema (creates an index) and requests data re-import from FOLIO data import subsystem. Module upgrade is complete when data is fully re-imported. This strategy increases complexity of migration as introduces dependency on external modules (additional error conditions). The task also becomes blocked on availability of the data import subsystem which is work in progress. &lt;/p&gt;

&lt;p&gt;The above is my own (very rough) take on scenarios and strategies. I am sure there is more and this is something for the SIG to establish.&lt;/p&gt;

&lt;p&gt;In terms of executing on the user stories: we will likely need to reserve some capacity in the Platform team to provide an example in a selected module(s) and extend existing automatic integration support (if needed. We also need to create tasks for all other teams to implement migrations targeting specific modules and versions. Finally, the DoDs should be also extended to provide automatic migrations (probably for release artifacts).&lt;/p&gt;</comment>
                                                            <comment id="12505" author="557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d" created="Tue, 9 Jul 2019 14:31:27 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ade4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; class=&quot;user-hover&quot; rel=&quot;557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; data-account-id=&quot;557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; accountid=&quot;557058:de4eac47-6d4e-4b50-9c3c-4a82ed705e52&quot; rel=&quot;noreferrer&quot;&gt;Taras Spashchenko&lt;/a&gt; please, touch base with me when you get back from vaca so we can plan some SPIKEs for the schema upgrade work. This issue so far is the high-level descripiton of the task.&lt;/p&gt;</comment>
                                                            <comment id="12507" author="5af5ed55244bc90a106063c7" created="Mon, 29 Jul 2019 17:33:49 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; can you please give this a PO rank?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10000">
                    <name>Blocks</name>
                                                                <inwardlinks description="is blocked by">
                                        <issuelink>
            <issuekey id="81330">FOLIO-2169</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10001">
                    <name>Cloners</name>
                                                                <inwardlinks description="is cloned by">
                                        <issuelink>
            <issuekey id="11213">UXPROD-2120</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10002">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="10488">UXPROD-753</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10003">
                    <name>Relates</name>
                                            <outwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="38350">DEBT-2</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="80131">FOLIO-464</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="80776">FOLIO-1519</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="80104">FOLIO-406</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="80338">FOLIO-838</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="78570">MODVEND-81</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Back End Estimate</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10124"><![CDATA[XXL < 30 days]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10054" key="com.atlassian.jira.plugin.system.customfieldtypes:userpicker">
                        <customfieldname>Back End Estimator</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10000" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummarycf">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10057" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Development Team</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10144"><![CDATA[Core: Platform]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10014" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue key="$xmlutils.escape($text)">Platform, DevOps and Release Management</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Estimation Notes and Assumptions</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>XXL is the largest available. This item is likely larger.</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10063" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>PO Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10019" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0|hzzs27:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10071" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: 5Colleges (Full Jul 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10223"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10068" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Chicago (MVP Sum 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10208"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10070" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Duke (Full Sum 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10218"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10074" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: GBV (MVP Sum 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10238"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10078" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Lehigh (MVP Summer 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10258"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10080" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: Leipzig (ERM Aut 2019)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10269"><![CDATA[R2]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10089" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: TAMU (MVP Jan 2021)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10314"><![CDATA[R1]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10091" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Rank: U of AL (MVP Oct 2020)</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10325"><![CDATA[R2]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10020" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10024" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>[CHART] Date of First Response</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 5 Jun 2019 12:58:50 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10025" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>[CHART] Time in Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>