[FOLIO-695] Implement centralized log console for FOLIO CI systems Created: 26/Jun/17  Updated: 12/Nov/18  Resolved: 29/Mar/18

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: John Malconian Assignee: John Malconian
Resolution: Done Votes: 0
Labels: ci, cicd, for-next-sprint, sprint17, sprint18, sprint19
Remaining Estimate: Not Specified
Time Spent: 1 week, 7 hours
Original estimate: Not Specified

Issue links:
Blocks
is blocked by FOLIO-849 Logging formats for backend FOLIO mod... Closed
Sprint:

 Description   

The CI systems, folio-stable and folio-testing, and possibly others should send aggregated their log files to a centralized log system and made accessible via a console that core FOLIO developers have access to.



 Comments   
Comment by John Malconian [ 26/Jun/17 ]

I'd like to try to implement the ideas here:

https://aws.amazon.com/blogs/database/ek-is-the-new-elk-simplify-log-analytics-by-transforming-data-natively-in-amazon-elasticsearch-service/

Comment by John Malconian [ 26/Jun/17 ]

Add apt repository for filebeats to FOLIO test systems. Directions here:

https://www.elastic.co/guide/en/beats/filebeat/current/setup-repositories.html

Comment by Jakub Skoczen [ 26/Jul/17 ]

We have the logs accessible via URL, John will paste them here.

Comment by John Malconian [ 26/Jul/17 ]

We are now collecting Okapi logs for folio-testing, folio-staging, and folio-stable via an ELK set up at AWS using
Filebeat as the log shipper to a central Logstash instance for filtering and then to an AWS Elasticsearch instance. In order to access the Elasticsearch instance, I've set up a special proxy so that no AWS credentials are required to access the Elasticsearch/Kibana interface. The proxy is running at:

http://folioci-logs.aws.indexdata.com

There are some basic Auth controls on it (u/p) which I can disseminate some other way.

Comment by Jakub Skoczen [ 02/Aug/17 ]

Question remains how do we document the above for the core team eyes only?

Comment by John Malconian [ 26/Sep/17 ]

Need to add backend module logs to this. Pausing this issue until we can achieve some consistency with logging in individual modules - See FOLIO-849 Closed .

Comment by John Malconian [ 29/Mar/18 ]

Marking this as complete for now since Okapi deployed modules log to the okapi log.

Generated at Thu Feb 08 23:07:41 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.