Take heap dump
Summary
This document contains step-by-step flow how to effectively and fast take a heap dump from folio modules.
Two approaches:
- Manual - this step takes advantage of the heapdump script that is deployed in every docker container.
- Faster, less steps than before
- No need to do multiple SCPs from different instances as heap dump will go strait to S3 bucket
- Automated: fse-collect-heapdump job
Flow
- Check EC2 instance where particular docker container is located by clicking on service task in ECS service:
2. Connect to EC2 instance via SSH with command like
ssh -i "folio-int.pem" ec2-user@{IP}
3. Do Docker ps and grab id of needed container;
4. SSH inside container by command
docker exec -it {dockerID} /bin/bash
5. Inside entry point (default entry folder) run
jmap -dump:format=b,file=mod-oai-pmh.hprof PID
a. Name of .hprof file matters. It should be name of service without "-b" postfix. (also you can check how to give a name to this file you can open heapdump.sh script and check one of the last lines it contains proper name of the file);
b. To get PID (process ID of the java process) run top command and check java PID.
6. After #5 command will generate .hprof file run
bash heapdump.sh
7. This command will zip heap dump and sent it to appropriate S3 bucket in heap folder
FSE Jenkins Job
Alternatively one can use the FSE Jenkins job, fse-collect-heapdump, to collect heap dumps of one or multiple modules at a time. Access will be needed to work with this job.