PoC for hosted development environment for FOLIO teams
(FOLIO-2601)
|
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None | Parent: | PoC for hosted development environment for FOLIO teams |
| Type: | Task | Priority: | P2 |
| Reporter: | Jakub Skoczen | Assignee: | John Malconian |
| Resolution: | Done | Votes: | 0 |
| Labels: | dev-environment | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Sprint: | DevOps: sprint 92, DevOps: sprint 90 |
| Development Team: | FOLIO DevOps |
| Epic Link: | PoC for hosted development environment for FOLIO teams |
| Description |
|
Implement EFK log stack on FOLIO cluster. The log stack should be able to ship logs from any container to an Elasticsearch instance that can be accessible via a Kibana dashboard. Github OAUTH should be enabled to allow access to the dashboard from any member of github.com/folio-org. The entire stack should be run on the K8s cluster. |
| Comments |
| Comment by John Malconian [ 16/Jul/20 ] |
|
I've created an EFK log aggregation stack at: https://logs.ci.folio.org. Authentication is via OAUTH2 and restricted to github.com/folio-org members. Navigate to 'Discovery' from the home page to get to the main logging dashboard. |
| Comment by Jakub Skoczen [ 17/Jul/20 ] |
|
Back in progress to implement index retention. |
| Comment by John Malconian [ 22/Jul/20 ] |
|
Increased disk capacity of Elasticsearch nodes in order to hold at least a few days worth of logs and configured the following ES Lifecycle Policy for filebeat (logs) indexes. PUT _ilm/policy/filebeat
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_age": "1d",
"max_size": "50gb"
},
"set_priority": {
"priority": 100
}
}
},
"warm": {
"actions": {
"set_priority": {
"priority": 50
}
}
},
"delete": {
"min_age": "3d",
"actions": {
"delete": {}
}
}
}
}
}
This may need further refinement, but the policy should rotate log indexes daily (or if the index reaches 50GB first). Rotated indexes will be deleted after three days. This should be ok for a dev environment. My best guess at this point is that we will generate ~50GB daily. |