View Issue Details

IDProjectCategoryView StatusLast Update
0000080Rocky-Linux-8Generalpublic2021-12-11 21:57
ReporterMohammad Ali Sarbanha Assigned ToRelease Engineering  
PrioritynormalSeverityminorReproducibilityhave not tried
Status assignedResolutionopen 
Summary0000080: VM 'used' memory is growing? Probably memory leakage
DescriptionThis image shows the time when the VM has consumed entire memroy and crashes. HDD activity dropped and CPU utilization has become almost 80~90%

I use Rocky8.5 Linux and I am experiencing growing the size of used memory when I start transferring gigantic files by SCP!

Here is the scenario, I have an ESXi in datacenter A and a VM in datacenter B. In order to access ESXi I had to make IPSec tunnel. The tunnel is made between the VM and the remote network gateway with strongswan. I have a 350GB file that I want to download to the remote VM. The only tool that I have is SCP at the moment. other tools like Rsync or FTP neither available nor fits the current topology.

When I start transferring data using SCP, used memory starts increasing:

    [root@gateway]# free -h
                  total used free shared buff/cache available
    Mem: 15Gi 1.4Gi 11Gi 16Mi 2.5Gi 13Gi
    Swap: 4.0Gi 0B 4.0Gi

At the same time buff/cache is getting increased which is normal I think. When I run `sync; echo 3 > /proc/sys/vm/drop_caches` buff/cache shrinks:

    [root@gateway]# free -h
                  total used free shared buff/cache available
    Mem: 15Gi 1.4Gi 13Gi 16Mi 119Mi 13Gi
    Swap: 4.0Gi 0B 4.0Gi

But the used memory remains at the same size. I am experiencing the VM crash after downloading 60~70GB of the file.

I have increased the RAM from 4GB to 16GB to see if it helps, but the result was that it takes longer to crash

Initially thought that it was SSH or SCP memory leakage, so I changed my method by setting up NFS on the VM and mount it on remote ESXi machine, then started regular file copy. Still, growing used memory observed.

I used 'ps waux' to see if I can find any process overusing the memory. Interestingly, processes barely reach 0.3% MEM

In order to see if stopping file copy reduces the allocated memory, I stopped the copy process and left the machine for few hours. Allocated memory wasn't released back to available memory.

The hardware on which the VM is hosted is Dell PowerEdgeâ„¢ R6515, The hypervisor used is VMWare vSphere ESXi 7.x

When the VM crashes, it doesn't allow to send reboot signal through keyboard. Only cold boot works.
TagsNo tags attached.

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change