Creating heap dump for modules

Heap dumps are primarily used for analyzing memory usage, identifying memory leaks, and understanding the overall memory footprint of modules. First, we need to write/edit access for the Rancher corresponding environment, in this case, access to edit deployments.

 

  1. First of all, we need to create PVC (Persistent Volume Claim), open Rancher and choose the necessary Cluster and Namespace, open Storage(1), PersistentVolumeClaim(2), Create(3)

    Screenshot_1.png


    Name PVC and remember it (example. search-dump), choose storage class gp2, Request Storage 10GB should be enough, and create the new PVC.

    Screenshot_2.png

     

  2. Open Rancher, Choose Cluster (1), Namespace (2), Workloads (3), Deployments (4) and filter the necessary module for which you want to get heap dump.

  3. After filtering the required module we should edit the configuration
    IMPORTANT: Choose Edit YAML

  4. In the configuration, first, we should find JAVA_OPTIONS and add these
    two parameters -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/dump ,
    it should look like this

    Before

    After

  5. The next step is to add the new PVC and mount it to the instance, scroll down to the end of the configuration file, and add these two configurations in the correct places as shown in the screenshot
    IMPORTANT: claimName should be the same as you named created PVC!!!

    volumeMounts: - mountPath: /tmp/dump name: dump
    volumes: - name: dump persistentVolumeClaim: claimName: search-dump

     

  6. By the last step, in the same config, we need to add init containers with a security context, add this snipped in the correct place, an example in the screenshot

    initContainers: - command: - /bin/sh - '-ec' - chmod 777 /tmp/dump - chown -R 1000:1000 /tmp/dump image: docker.io/bitnami/os-shell:11-debian-11 imagePullPolicy: IfNotPresent name: init-chmod-data resources: {} securityContext: allowPrivilegeEscalation: true capabilities: {} privileged: false readOnlyRootFilesystem: false runAsUser: 0 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /tmp/dump name: dump

That's all, save the configuration and wait until the new pod starts, now all the necessary information will be saved in volume. The final step is to copy the dump from PVC to S3, in most cases, you will not have the necessary permission, so please ask someone from the KitFox team, and we will help you.
In future we’re planning to automate this process.