<!-- 
RSS generated by JIRA (1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d) at Thu Feb 08 23:22:03 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary add field=key&field=summary to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>FOLIO Jira</title>
    <link>https://folio-org.atlassian.net</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>1001.0.0-SNAPSHOT</version>
        <build-number>100246</build-number>
        <build-date>07-02-2024</build-date>
    </build-info>

<item>
            <title>[MODINVSTOR-513] hitcount estimation degeneration when upgrading MIS 19.1.2 (fameflower) -&gt; MIS 19.2.0-SNAPSHOT (RMB30)</title>
                <link>https://folio-org.atlassian.net/browse/MODINVSTOR-513</link>
                <project id="10139" key="MODINVSTOR">mod-inventory-storage</project>
                    <description>&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=70121%3A84bca0b8-4380-4d87-8a90-53e9133584f6&quot; class=&quot;user-hover&quot; rel=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; data-account-id=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; accountid=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; rel=&quot;noreferrer&quot;&gt;Mikhail Fokanov&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5c10cd488ce9b546efc4d9c4&quot; class=&quot;user-hover&quot; rel=&quot;5c10cd488ce9b546efc4d9c4&quot; data-account-id=&quot;5c10cd488ce9b546efc4d9c4&quot; accountid=&quot;5c10cd488ce9b546efc4d9c4&quot; rel=&quot;noreferrer&quot;&gt;Oleksii Popov&lt;/a&gt; this ticket is created to track the work Mikhail is doing on bugfest with &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=70121%3A329e0f11-cc86-459e-a311-80d4eb134a7f&quot; class=&quot;user-hover&quot; rel=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; data-account-id=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; accountid=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; rel=&quot;noreferrer&quot;&gt;Sobha Duvvuri&lt;/a&gt; &lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5af5ed55244bc90a106063c7&quot; class=&quot;user-hover&quot; rel=&quot;5af5ed55244bc90a106063c7&quot; data-account-id=&quot;5af5ed55244bc90a106063c7&quot; accountid=&quot;5af5ed55244bc90a106063c7&quot; rel=&quot;noreferrer&quot;&gt;Cate Boerema&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Af80403de-e149-421e-8750-af45c853402f&quot; class=&quot;user-hover&quot; rel=&quot;557058:f80403de-e149-421e-8750-af45c853402f&quot; data-account-id=&quot;557058:f80403de-e149-421e-8750-af45c853402f&quot; accountid=&quot;557058:f80403de-e149-421e-8750-af45c853402f&quot; rel=&quot;noreferrer&quot;&gt;Charlotte Whitt&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=63e2a2771b13d42998e4e706&quot; class=&quot;user-hover&quot; rel=&quot;63e2a2771b13d42998e4e706&quot; data-account-id=&quot;63e2a2771b13d42998e4e706&quot; accountid=&quot;63e2a2771b13d42998e4e706&quot; rel=&quot;noreferrer&quot;&gt;Marc Johnson&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Problem&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;We are seeing imprecise hitcounts reported after upgrading Bugfest from MIS used for Fameflower to the latest MIS snapshot using RMB 30 (which included the harmonisation of hit count between Instance records sorted by title). Most hitcounts are reported as ~37000.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Analyis from Mikhail&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;I found out that the instance table wan&apos;t analysed due to some error during the deployment of the new version of inventory.&lt;br/&gt;
Manually running of:&lt;br/&gt;
ANALYZE  fs09000000_mod_inventory_storage.instance;&lt;br/&gt;
fixes the problem.&lt;br/&gt;
But VACUUM with ANALYZE was failing (possibly due to timeout).&lt;br/&gt;
But just ANALYZE query is pretty fast, so it completed.&lt;br/&gt;
So, as far as I can see, the root cause is that because of changes which was implemented in scope of 
    &lt;span class=&quot;jira-issue-macro resolved&quot; data-jira-key=&quot;RMB-583&quot; &gt;
                &lt;a href=&quot;https://folio-org.atlassian.net/browse/RMB-583&quot; class=&quot;jira-issue-macro-key issue-link&quot;  title=&quot;Move f_unaccent and concat* from public to schema, fix search_path&quot; &gt;
            &lt;img class=&quot;icon&quot; src=&quot;https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium&quot; /&gt;
            RMB-583
        &lt;/a&gt;
                                                    &lt;span class=&quot;aui-lozenge aui-lozenge-subtle aui-lozenge-success jira-macro-single-issue-export-pdf&quot;&gt;Closed&lt;/span&gt;
            &lt;/span&gt;
 during the deployment there is a migration which involves REINDEX and ANALYZE but due to timeout of this procedures the database wasn&apos;t completely migrated, but stuck in the kind of inconsistent state and therefore counters are not accurate.&lt;/p&gt;

&lt;p&gt;Documentation:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Speadsheet: &lt;a href=&quot;https://docs.google.com/spreadsheets/d/1ergJ7jDHdLbD_noWsL-ZdtVzB2NiiIPaCcUeO4ZFLZw/edit#gid=0&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/spreadsheets/d/1ergJ7jDHdLbD_noWsL-ZdtVzB2NiiIPaCcUeO4ZFLZw/edit#gid=0&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
</description>
                <environment></environment>
        <key id="56097">MODINVSTOR-513</key>
            <summary>hitcount estimation degeneration when upgrading MIS 19.1.2 (fameflower) -&gt; MIS 19.2.0-SNAPSHOT (RMB30)</summary>
                <type id="10001" iconUrl="https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium">Bug</type>
                                            <priority id="10001" iconUrl="https://dev.folio.org/assets/jira-priority/jira-p2.svg">P2</priority>
                        <status id="6" iconUrl="https://folio-org.atlassian.net/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="green"/>
                                    <resolution id="10003">Done</resolution>
                                                        <assignee accountid="70121:84bca0b8-4380-4d87-8a90-53e9133584f6">Mikhail Fokanov</assignee>
                                                                <reporter accountid="557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d">Jakub Skoczen</reporter>
                                    <labels>
                            <label>platform-backlog</label>
                    </labels>
                <created>Tue, 2 Jun 2020 10:31:22 +0000</created>
                <updated>Mon, 15 Jun 2020 12:18:45 +0000</updated>
                            <resolved>Mon, 8 Jun 2020 16:21:59 +0000</resolved>
                                                    <fixVersion>19.2.0</fixVersion>
                                        <due></due>
                            <votes>0</votes>
                                    <watches>7</watches>
                                                                <comments>
                                                            <comment id="130608" author="5ee89462f7aa140abd82d11d" created="Tue, 2 Jun 2020 15:52:07 +0000"  >&lt;p&gt;RMB has full logging enabled when running the upgrade SQL scripts:&lt;br/&gt;
&lt;a href=&quot;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java#L3505-L3516&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java#L3505-L3516&lt;/a&gt;&lt;br/&gt;
RMB first uses log.info to print &quot;trying to execute: &quot; and the SQL query to run.&lt;br/&gt;
Then it tries to run the query.&lt;br/&gt;
On success RMB uses log.info to print &quot;Successfully executed: &quot; and the successfull SQL query again.&lt;br/&gt;
On failure RMB uses log.error to print the exception message and the exception stacktrace.&lt;br/&gt;
I don&apos;t see any possibility how to increase logging.&lt;/p&gt;</comment>
                                                            <comment id="130610" author="5ee89462f7aa140abd82d11d" created="Tue, 2 Jun 2020 16:06:12 +0000"  >&lt;p&gt;Creating a mod-inventory-storage jar with RMB 30 but the 
    &lt;span class=&quot;jira-issue-macro resolved&quot; data-jira-key=&quot;RMB-583&quot; &gt;
                &lt;a href=&quot;https://folio-org.atlassian.net/browse/RMB-583&quot; class=&quot;jira-issue-macro-key issue-link&quot;  title=&quot;Move f_unaccent and concat* from public to schema, fix search_path&quot; &gt;
            &lt;img class=&quot;icon&quot; src=&quot;https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium&quot; /&gt;
            RMB-583
        &lt;/a&gt;
                                                    &lt;span class=&quot;aui-lozenge aui-lozenge-subtle aui-lozenge-success jira-macro-single-issue-export-pdf&quot;&gt;Closed&lt;/span&gt;
            &lt;/span&gt;
 commit reverted is difficult and requires much work because several other later commits depend on it.&lt;br/&gt;
My suggestion to try to reproduce the hitcount issue is this:&lt;br/&gt;
Start with the mod-inventory-storage 19.1.2 installation.&lt;br/&gt;
&lt;tt&gt;DROP TABLE fs09000000_mod_inventory_storage.rmb_internal_index;&lt;/tt&gt;&lt;br/&gt;
This will remove the information about the existing indexes and RMB will recreate all of them on next upgrade.&lt;br/&gt;
Now run an upgrade from 19.1.2 to 19.1.2, this is an upgrade to the same version. It will recreate all indexes and may reproduce the issue.&lt;/p&gt;</comment>
                                                            <comment id="130611" author="557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d" created="Wed, 3 Jun 2020 09:07:10 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt; Are you proposing that we try to reproduce the issue with 19.1.2 (fameflower version) ONLY? Why do you expect that the problem is already present in that version?&lt;/p&gt;</comment>
                                                            <comment id="130613" author="5ee89462f7aa140abd82d11d" created="Wed, 3 Jun 2020 15:54:18 +0000"  >&lt;p&gt;The problem may be the result of long running index recreations that get interrupted. If this is the case the 19.1.2 to 19.1.2 &quot;upgrade&quot; will also show the problem.&lt;/p&gt;</comment>
                                                            <comment id="130615" author="5ee89462f7aa140abd82d11d" created="Thu, 4 Jun 2020 10:11:36 +0000"  >&lt;p&gt;Tenant upgrade uses the jdbc driver. It does not use the vertx driver and is therefore not affected by the switch to the vertx-pg-client.&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;TenantAPI uses runSqlFile: &lt;a href=&quot;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/impl/TenantAPI.java#L311&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/impl/TenantAPI.java#L311&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;runSqlFile uses execute(String[] sql, ...): &lt;a href=&quot;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java#L3499-L3510&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java#L3499-L3510&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;This uses the JDBC getStandaloneConnection: &lt;a href=&quot;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java#L3413-L3428&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/folio-org/raml-module-builder/blob/v30.0.0/domain-models-runtime/src/main/java/org/folio/rest/persist/PostgresClient.java#L3413-L3428&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</comment>
                                                            <comment id="130616" author="5ee89462f7aa140abd82d11d" created="Thu, 4 Jun 2020 11:27:08 +0000"  >&lt;p&gt;We can reproduce the problem and it is not caused by interrupted reindexing.&lt;br/&gt;
PostgreSQL has an autovacuum daemon that is enabled by default and automatically runs VACUUM and ANALYZE:&lt;br/&gt;
&lt;a href=&quot;https://www.postgresql.org/docs/current/routine-vacuuming.html&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.postgresql.org/docs/current/routine-vacuuming.html&lt;/a&gt;&lt;br/&gt;
There are two configuration parameters (see &lt;a href=&quot;https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-ANALYZE-THRESHOLD&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-ANALYZE-THRESHOLD&lt;/a&gt; ):&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;autovacuum_analyze_threshold: Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table. The default is 50 tuples.&lt;/li&gt;
	&lt;li&gt;autovacuum_analyze_scale_factor: Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE. The default is 0.1 (10% of table size).&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;My guess is that recreating the index deletes the statistics but no ANALYZE is executed because there is no tuple change.&lt;br/&gt;
Therefore I propose to add ANALYZE to the upgrade script so that it runs after the index recreation.&lt;/p&gt;</comment>
                                                            <comment id="130617" author="70121:84bca0b8-4380-4d87-8a90-53e9133584f6" created="Thu, 4 Jun 2020 11:54:13 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt;, could you create a branch with this fix in order to deploy and test it?&lt;/p&gt;</comment>
                                                            <comment id="130618" author="5ee89462f7aa140abd82d11d" created="Thu, 4 Jun 2020 12:00:54 +0000"  >&lt;p&gt;Reproducing this with the performance dataset.&lt;br/&gt;
Show the correct counts:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
postgres=# select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;source&apos;&lt;/span&gt;);
  1259
postgres=# select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;master&apos;&lt;/span&gt;);
  1863
postgres=# select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;last&apos;&lt;/span&gt;);
  4206
postgres=# select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;be&apos;&lt;/span&gt;);
 23596
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Drop and recreate the index:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
postgres=# drop index instance_title_idx_ft;
DROP INDEX
postgres=# create index instance_title_idx_ft on instance using gin ((to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;))));
CREATE INDEX
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Now EXPLAIN returns the same estimation 13806 for all search strings:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;source&apos;&lt;/span&gt;);
 Aggregate  (cost=50122.97..50122.98 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=147.00..50088.45 rows=13806 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;source&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..143.55 rows=13806 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;source&apos;&lt;/span&gt;&apos;&apos;::tsquery)
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;master&apos;&lt;/span&gt;);
 Aggregate  (cost=50122.97..50122.98 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=147.00..50088.45 rows=13806 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;master&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..143.55 rows=13806 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;master&apos;&lt;/span&gt;&apos;&apos;::tsquery)
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;last&apos;&lt;/span&gt;);
 Aggregate  (cost=50122.97..50122.98 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=147.00..50088.45 rows=13806 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;last&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..143.55 rows=13806 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;last&apos;&lt;/span&gt;&apos;&apos;::tsquery)
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;be&apos;&lt;/span&gt;);
 Aggregate  (cost=50122.97..50122.98 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=147.00..50088.45 rows=13806 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;be&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..143.55 rows=13806 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;be&apos;&lt;/span&gt;&apos;&apos;::tsquery)
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Run ANALYZE to collect statistics:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
postgres=# analyze instance;
ANALYZE
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;Now we get good estimations (1288, 1288, 3220, 22541):&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;source&apos;&lt;/span&gt;);
 Aggregate  (cost=5287.40..5287.41 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=29.98..5284.18 rows=1288 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;source&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..29.66 rows=1288 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;source&apos;&lt;/span&gt;&apos;&apos;::tsquery)
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;master&apos;&lt;/span&gt;);
 Aggregate  (cost=5287.40..5287.41 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=29.98..5284.18 rows=1288 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;master&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..29.66 rows=1288 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;master&apos;&lt;/span&gt;&apos;&apos;::tsquery)
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;last&apos;&lt;/span&gt;);
 Aggregate  (cost=12824.56..12824.57 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=48.96..12816.51 rows=3220 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;last&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..48.15 rows=3220 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;last&apos;&lt;/span&gt;&apos;&apos;::tsquery)
postgres=# explain select count(*) from instance where to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, f_unaccent(jsonb-&amp;gt;&amp;gt;&lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;)) @@ to_tsquery(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;, &lt;span class=&quot;code-quote&quot;&gt;&apos;be&apos;&lt;/span&gt;);
 Aggregate  (cost=77481.57..77481.58 rows=1 width=8)
   -&amp;gt;  Bitmap Heap Scan on instance  (cost=226.69..77425.21 rows=22541 width=0)
         Recheck Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;be&apos;&lt;/span&gt;&apos;&apos;::tsquery)
         -&amp;gt;  Bitmap Index Scan on instance_title_idx_ft  (cost=0.00..221.05 rows=22541 width=0)
               Index Cond: (to_tsvector(&lt;span class=&quot;code-quote&quot;&gt;&apos;simple&apos;&lt;/span&gt;::regconfig, (jsonb -&amp;gt;&amp;gt; &lt;span class=&quot;code-quote&quot;&gt;&apos;title&apos;&lt;/span&gt;::text)) @@ &lt;span class=&quot;code-quote&quot;&gt;&apos;&apos;&apos;be&apos;&lt;/span&gt;&apos;&apos;::tsquery)
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;</comment>
                                                            <comment id="130619" author="5ee89462f7aa140abd82d11d" created="Fri, 5 Jun 2020 11:02:31 +0000"  >&lt;p&gt;Upgrading mod-inventory-storage to RMB 30.0.2 with the fix 
    &lt;span class=&quot;jira-issue-macro resolved&quot; data-jira-key=&quot;RMB-637&quot; &gt;
                &lt;a href=&quot;https://folio-org.atlassian.net/browse/RMB-637&quot; class=&quot;jira-issue-macro-key issue-link&quot;  title=&quot;Run ANALYZE after index (re-)creation&quot; &gt;
            &lt;img class=&quot;icon&quot; src=&quot;https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium&quot; /&gt;
            RMB-637
        &lt;/a&gt;
                                                    &lt;span class=&quot;aui-lozenge aui-lozenge-subtle aui-lozenge-success jira-macro-single-issue-export-pdf&quot;&gt;Closed&lt;/span&gt;
            &lt;/span&gt;
 &quot;Run ANALYZE after index (re-)creation&quot; solves this issue.&lt;/p&gt;</comment>
                                                            <comment id="130620" author="63e2a2771b13d42998e4e706" created="Fri, 5 Jun 2020 11:42:35 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5f8314dfbdef80006f6f572d&quot; class=&quot;user-hover&quot; rel=&quot;5f8314dfbdef80006f6f572d&quot; data-account-id=&quot;5f8314dfbdef80006f6f572d&quot; accountid=&quot;5f8314dfbdef80006f6f572d&quot; rel=&quot;noreferrer&quot;&gt;Adam Dickmeiss&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=70121%3A84bca0b8-4380-4d87-8a90-53e9133584f6&quot; class=&quot;user-hover&quot; rel=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; data-account-id=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; accountid=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; rel=&quot;noreferrer&quot;&gt;Mikhail Fokanov&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We&apos;ve had challenges in the past (with triggers) where some environments are using database users with limited permissions.&lt;/p&gt;

&lt;p&gt;What is the best way for us to communicate that any module using RAML Module Builder 30.0.2 will need to have a user than can analyse statistics?&lt;/p&gt;

&lt;p&gt;Does this issue make upgrading to RAML Module Builder 30.0.2 required? As I understand it 30.0.0 will drop and recreate the indexes, which without re-analysis of the statistics, will result in incorrect counts?&lt;/p&gt;

&lt;p&gt;And do we need to redo the upgrade re-testing?&lt;/p&gt;</comment>
                                                            <comment id="130621" author="5ee89462f7aa140abd82d11d" created="Fri, 5 Jun 2020 12:09:51 +0000"  >&lt;p&gt;Using a database user for upgrading that does not have ANALYZE permissions doesn&apos;t fail the upgrade, there will be a warning only:&lt;/p&gt;
&lt;div class=&quot;code panel&quot; style=&quot;border-width: 1px;&quot;&gt;&lt;div class=&quot;codeContent panelContent&quot;&gt;
&lt;pre class=&quot;code-java&quot;&gt;
postgres=&amp;gt; \c postgres diku_mod_inventory_storage
You are now connected to database &lt;span class=&quot;code-quote&quot;&gt;&quot;postgres&quot;&lt;/span&gt; as user &lt;span class=&quot;code-quote&quot;&gt;&quot;diku_mod_inventory_storage&quot;&lt;/span&gt;.
postgres=&amp;gt; analyze instance;
WARNING:  skipping &lt;span class=&quot;code-quote&quot;&gt;&quot;instance&quot;&lt;/span&gt; --- only table or database owner can analyze it
ANALYZE
&lt;/pre&gt;
&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The &lt;a href=&quot;https://www.postgresql.org/docs/10/sql-analyze.html&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANALYZE documentation&lt;/a&gt; says:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;To analyze a table, one must ordinarily be the table&apos;s owner or a superuser. However, database owners are allowed to analyze all tables in their databases [...] ANALYZE will skip over any tables that the calling user does not have permission to analyze.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The &lt;a href=&quot;https://www.postgresql.org/docs/10/ddl-priv.html&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;Privileges documentation&lt;/a&gt; says:&lt;/p&gt;
&lt;blockquote&gt;&lt;p&gt;The owner is normally the role that executed the creation statement.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;FOLIO uses a bootstrap user that runs each installation and each upgrade. If they don&apos;t change this bootstrap user then the bootstrap user is still the owner of the table and therefore can run ANALYZE. Why would they change the bootstrap user?&lt;/p&gt;</comment>
                                                            <comment id="130622" author="5ee89462f7aa140abd82d11d" created="Fri, 5 Jun 2020 12:20:09 +0000"  >&lt;blockquote&gt;&lt;p&gt;Does this issue make upgrading to RAML Module Builder 30.0.2 required? As I understand it 30.0.0 will drop and recreate the indexes, which without re-analysis of the statistics, will result in incorrect counts?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;RMB 30 will drop and recreate some indexes. This also drops statistics resulting in bad estimated counts. PostgreSQL will automatically run ANALYZE to calculate new statistics after 50 changes (insert, update, delete) have been made to the underlying data.&lt;/p&gt;

&lt;p&gt;This issue is about this time frame. People test the upgraded system without changing data.&lt;/p&gt;

&lt;p&gt;RMB 30.0.2 runs ANALYZE at the end of the upgrade process and avoids this time frame with incorrect estimation and therefore solved this issue.&lt;/p&gt;</comment>
                                                            <comment id="130623" author="63e2a2771b13d42998e4e706" created="Fri, 5 Jun 2020 14:55:20 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;FOLIO uses a bootstrap user that runs each installation and each upgrade. If they don&apos;t change this bootstrap user then the bootstrap user is still the owner of the table and therefore can run ANALYZE. Why would they change the bootstrap user?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Thank you. I wasn&apos;t entirely sure which user ran the upgrades. I asked the question because an upgrade script was written previously that disabled triggers, this worked successfully in the hosted environments but not in others, making me a little cautions of these things.&lt;/p&gt;</comment>
                                                            <comment id="130624" author="63e2a2771b13d42998e4e706" created="Fri, 5 Jun 2020 14:57:36 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt; Thanks&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;This issue is about this time frame. People test the upgraded system without changing data.&lt;/p&gt;&lt;/blockquote&gt;

&lt;blockquote&gt;&lt;p&gt;RMB 30.0.2 runs ANALYZE at the end of the upgrade process and avoids this time frame with incorrect estimation and therefore solved this issue.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Then I think the question might be to &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; and &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5af5ed55244bc90a106063c7&quot; class=&quot;user-hover&quot; rel=&quot;5af5ed55244bc90a106063c7&quot; data-account-id=&quot;5af5ed55244bc90a106063c7&quot; accountid=&quot;5af5ed55244bc90a106063c7&quot; rel=&quot;noreferrer&quot;&gt;Cate Boerema&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Are there circumstances where it is acceptable for the record counts to be possible significantly incorrect for a while after an upgrade and eventually improve?&lt;/p&gt;

&lt;p&gt;If not, then I think that suggests that all modules need to upgrade to RAML Module Builder 30.0.2 in order to prevent this.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt; please correct me if I haven&apos;t followed correctly.&lt;/p&gt;</comment>
                                                            <comment id="130625" author="70121:329e0f11-cc86-459e-a311-80d4eb134a7f" created="Fri, 5 Jun 2020 15:08:21 +0000"  >&lt;p&gt;I had similar question too &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt; - this fix seems to me that all modules should upgrade to RMB 30.0.2. &lt;br/&gt;
Does the volume of data impact the timing of index re-creation and engine running autovacuum in any way or just the number of inserts, updates or deletes made? Should all modules gain advantage of the added ANALYZE step?&lt;br/&gt;
Also, it took 3+ hours for MODINVSTOR to upgrade from 19.1.2 to 19.2.0-snapshot. Anyway in which this could be reduced? Thanks!&lt;/p&gt;</comment>
                                                            <comment id="130626" author="5ee89462f7aa140abd82d11d" created="Fri, 5 Jun 2020 22:04:13 +0000"  >&lt;blockquote&gt;&lt;p&gt;Does the volume of data impact the timing of index re-creation and engine running autovacuum in any way or just the number of inserts, updates or deletes made?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;There are two configuration parameters (see &lt;a href=&quot;https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-ANALYZE-THRESHOLD&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.postgresql.org/docs/current/runtime-config-autovacuum.html#GUC-AUTOVACUUM-ANALYZE-THRESHOLD&lt;/a&gt; ):&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;autovacuum_analyze_threshold: Specifies the minimum number of inserted, updated or deleted tuples needed to trigger an ANALYZE in any one table. The default is 50 tuples.&lt;/li&gt;
	&lt;li&gt;autovacuum_analyze_scale_factor: Specifies a fraction of the table size to add to autovacuum_analyze_threshold when deciding whether to trigger an ANALYZE. The default is 0.1 (10% of table size).&lt;/li&gt;
&lt;/ul&gt;
</comment>
                                                            <comment id="130627" author="5ee89462f7aa140abd82d11d" created="Fri, 5 Jun 2020 22:22:11 +0000"  >&lt;blockquote&gt;&lt;p&gt;Should all modules gain advantage of the added ANALYZE step?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Yes. We cannot assume that there will be enough data changes in a reasonable time.&lt;/p&gt;

&lt;p&gt;Jakub posted in Slack #releases yesterday:&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;All modules using RMB &lt;b&gt;MUST&lt;/b&gt; at minimum upgrade to version 30.0.0 but it is &lt;b&gt;HIGHLY RECOMMENDED&lt;/b&gt; to upgrade to the latest bugfix release available.&lt;/p&gt;&lt;/blockquote&gt;</comment>
                                                            <comment id="130628" author="63e2a2771b13d42998e4e706" created="Mon, 8 Jun 2020 12:35:47 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;Yes. We cannot assume that there will be enough data changes in a reasonable time.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I think I&apos;m a little confused. This statement suggests that you believe waiting till the data changes occur is unacceptable. Is that a reasonable interpretation?&lt;/p&gt;

&lt;p&gt;If so, that suggests it is &lt;b&gt;required / expected&lt;/b&gt; that storage modules upgrade to RAML Module Builder 30.0.2 or later. However, you referred to &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; advice (quoted below) that it is only recommended.&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;All modules using RMB MUST at minimum upgrade to version 30.0.0 but it is HIGHLY RECOMMENDED to upgrade to the latest bugfix release available.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Please can either of you help me understand what the expectations are.&lt;/p&gt;</comment>
                                                            <comment id="130629" author="63e2a2771b13d42998e4e706" created="Mon, 8 Jun 2020 12:38:24 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=70121%3A329e0f11-cc86-459e-a311-80d4eb134a7f&quot; class=&quot;user-hover&quot; rel=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; data-account-id=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; accountid=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; rel=&quot;noreferrer&quot;&gt;Sobha Duvvuri&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;Also, it took 3+ hours for MODINVSTOR to upgrade from 19.1.2 to 19.2.0-snapshot. Anyway in which this could be reduced?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I don&apos;t think it is likely this can be reduced at this point. This upgrade process is dropping and re-creating many large indexes and re-analysing the statistics, these are very time consuming operations.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=70121%3A84bca0b8-4380-4d87-8a90-53e9133584f6&quot; class=&quot;user-hover&quot; rel=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; data-account-id=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; accountid=&quot;70121:84bca0b8-4380-4d87-8a90-53e9133584f6&quot; rel=&quot;noreferrer&quot;&gt;Mikhail Fokanov&lt;/a&gt; or &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=557058%3Ab8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; class=&quot;user-hover&quot; rel=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; data-account-id=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; accountid=&quot;557058:b8e64633-1f7c-402d-9caf-9959a5ba5d0d&quot; rel=&quot;noreferrer&quot;&gt;Jakub Skoczen&lt;/a&gt; are best placed to confirm this.&lt;/p&gt;</comment>
                                                            <comment id="130630" author="70121:84bca0b8-4380-4d87-8a90-53e9133584f6" created="Mon, 8 Jun 2020 12:43:11 +0000"  >&lt;p&gt;&lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=70121%3A329e0f11-cc86-459e-a311-80d4eb134a7f&quot; class=&quot;user-hover&quot; rel=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; data-account-id=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; accountid=&quot;70121:329e0f11-cc86-459e-a311-80d4eb134a7f&quot; rel=&quot;noreferrer&quot;&gt;Sobha Duvvuri&lt;/a&gt;, &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=63e2a2771b13d42998e4e706&quot; class=&quot;user-hover&quot; rel=&quot;63e2a2771b13d42998e4e706&quot; data-account-id=&quot;63e2a2771b13d42998e4e706&quot; accountid=&quot;63e2a2771b13d42998e4e706&quot; rel=&quot;noreferrer&quot;&gt;Marc Johnson&lt;/a&gt;  this index recreation is not related to hit counts. Hit counts implementation doesn&apos;t affect it. Indexes are affected by &lt;a href=&quot;https://folio-org.atlassian.net/browse/RMB-583&quot; class=&quot;external-link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://folio-org.atlassian.net/browse/RMB-583&lt;/a&gt;. So, maybe &lt;a href=&quot;https://folio-org.atlassian.net/secure/ViewProfile.jspa?accountId=5ee89462f7aa140abd82d11d&quot; class=&quot;user-hover&quot; rel=&quot;5ee89462f7aa140abd82d11d&quot; data-account-id=&quot;5ee89462f7aa140abd82d11d&quot; accountid=&quot;5ee89462f7aa140abd82d11d&quot; rel=&quot;noreferrer&quot;&gt;Julian Ladisch&lt;/a&gt; can answer the question.&lt;/p&gt;</comment>
                                                            <comment id="130631" author="5ee89462f7aa140abd82d11d" created="Mon, 8 Jun 2020 12:57:11 +0000"  >&lt;p&gt;&amp;gt;&amp;gt; Should all modules gain advantage of the added ANALYZE step?&lt;br/&gt;
&amp;gt; Yes.&lt;/p&gt;

&lt;p&gt;Should = highly recommended. There may be cases where this is not needed. For example if the table is so small that one cannot notice any performance difference. Or when upgrading changes all records which automatically starts the autovacuum daemon that runs ANALYZE. &lt;/p&gt;</comment>
                                                            <comment id="130632" author="5ee89462f7aa140abd82d11d" created="Mon, 8 Jun 2020 13:53:56 +0000"  >&lt;p&gt;Please use a separate issue (
    &lt;span class=&quot;jira-issue-macro resolved&quot; data-jira-key=&quot;MODINVSTOR-514&quot; &gt;
                &lt;a href=&quot;https://folio-org.atlassian.net/browse/MODINVSTOR-514&quot; class=&quot;jira-issue-macro-key issue-link&quot;  title=&quot;Migrating from Fameflower to Goldenrod takes hours&quot; &gt;
            &lt;img class=&quot;icon&quot; src=&quot;https://folio-org.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium&quot; /&gt;
            MODINVSTOR-514
        &lt;/a&gt;
                                                    &lt;span class=&quot;aui-lozenge aui-lozenge-subtle aui-lozenge-success jira-macro-single-issue-export-pdf&quot;&gt;Closed&lt;/span&gt;
            &lt;/span&gt;
) to discuss the long upgrading time.&lt;/p&gt;</comment>
                                                            <comment id="130633" author="63e2a2771b13d42998e4e706" created="Mon, 8 Jun 2020 16:21:59 +0000"  >&lt;p&gt;Closing this as due to be imminently released. Please re-open if necessary.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10000">
                    <name>Blocks</name>
                                                                <inwardlinks description="is blocked by">
                                        <issuelink>
            <issuekey id="57256">RMB-637</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10006">
                    <name>Gantt End to Start</name>
                                            <outwardlinks description="has to be done before">
                                        <issuelink>
            <issuekey id="56095">MODINVSTOR-504</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10003">
                    <name>Relates</name>
                                            <outwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="57283">RMB-591</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="56098">MODINVSTOR-511</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="56099">MODINVSTOR-514</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="57257">RMB-638</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="56785">RMB-644</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="relates to">
                                        <issuelink>
            <issuekey id="57199">RMB-583</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="57253">RMB-633</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10000" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummarycf">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10057" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Development Team</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10144"><![CDATA[Core: Platform]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10019" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0|i018bz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10046" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Release</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10022"><![CDATA[Q2 2020]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10020" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1108">CP: sprint 90</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10044" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10024" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>[CHART] Date of First Response</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 2 Jun 2020 15:52:07 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10025" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>[CHART] Time in Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>