[FOLIO-1102] import.pl in bib-data-import causes read timeout with latest test-data-loader Created: 02/Mar/18  Updated: 12/Nov/18  Resolved: 06/Mar/18

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Bug Priority: P3
Reporter: Wayne Schneider Assignee: shale99
Resolution: Done Votes: 0
Labels: ci, sprint33
Remaining Estimate: Not Specified
Time Spent: 2 hours
Original estimate: Not Specified

Sprint:

 Description   

MARC files can be posted using curl, but the import.pl script generates this error in the test-data-loader log:

01 Mar 2018 13:39:48:767 INFO  LoaderAPI  REQUEST ID 00b75196-c335-4e7e-97cd-4ac23313aa5c
01 Mar 2018 13:42:10:092 ERROR LoaderAPI  Read timed out
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.folio.rest.impl.LoaderAPI.post(LoaderAPI.java:785)
        at org.folio.rest.impl.LoaderAPI.managePushToDB(LoaderAPI.java:744)
        at org.folio.rest.impl.LoaderAPI.lambda$1(LoaderAPI.java:429)
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:271)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
01 Mar 2018 13:42:10:095 ERROR LoaderAPI  Read timed out
java.lang.Exception: Read timed out
        at org.folio.rest.impl.LoaderAPI.lambda$1(LoaderAPI.java:431)
        at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$1(ContextImpl.java:271)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
01 Mar 2018 13:42:10:099 INFO  LogUtil  org.folio.rest.RestVerticle start  invoking public void org.folio.rest.impl.LoaderAPI.postLoadMarcData(java.lang.String,int,java.io.InputStream,java.util.Map,io.vertx.core.Handler,io.vertx.core.Context) throws java.lang.Exception
01 Mar 2018 13:42:10:101 INFO  LogUtil  127.0.0.1:46268 POST /load/marc-data storageURL=http://localhost:9139 HTTP_1_1 500 54 141798 tid=diku Internal Server Error

Even with the error, the records load successfully.



 Comments   
Comment by Wayne Schneider [ 02/Mar/18 ]

If the file takes less than 125 seconds to load, it seems fine. I get the error when the file takes ~140 seconds. Working on more precision.

Comment by Wayne Schneider [ 02/Mar/18 ]

Increasing the SO_TIMEOUT and CONNECTION_TIMEOUT in test-data-loader/src/main/java/org/folio/rest/impl/LoaderAPI.java took care of the problem. Pushed to a branch.

Comment by Wayne Schneider [ 02/Mar/18 ]

Submitted PR, assigning to shale99 for review. If that works for you, feel free to close out this issue.

Generated at Thu Feb 08 23:10:51 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.