Symptoms
I receive kernel: TCP: out of memory -- consider tuning tcp_mem in the messages log, what can I do about this?
Description
If the TCP protocol uses more memory than the third value of the tcp_mem, the system will perform an out of memory checks, and If the memory required exceeds the maximum limit, it will throw an OOM error. The min/pressure/max values are automatically set in /proc/sys/net/ipv4/tcp_mem during boot based on available RAM size, however, It can be configured in /etc/sysctl.conf.
Workaround
You will want to adjust the kernel parameters for tcp_mem. Below is a snippet from the TCP manual.
tcp_mem (since Linux 2.4)
This is a vector of 3 integers: [low, pressure, high]. These
bounds, measured in units of the system page size, are used by
TCP to track its memory usage. The defaults are calculated at
boot time from the amount of available memory. (TCP can only
use low memory for this, which is limited to around 900
megabytes on 32-bit systems. 64-bit systems do not suffer
this limitation.)
low TCP doesn't regulate its memory allocation when the
number of pages it has allocated globally is below this
number.
pressure
When the amount of memory allocated by TCP exceeds this
number of pages, TCP moderates its memory consumption.
This memory pressure state is exited once the number of
pages allocated falls below the low mark.
high The maximum number of pages, globally, that TCP will
allocate. This value overrides any other limits
imposed by the kernel.
To verify the issue, check your messages log
# grep 'kernel: TCP: out of memory -- consider tuning tcp_mem' /var/log/messages|tail -2
Aug 18 08:33:14 jp58 kernel: TCP: out of memory -- consider tuning tcp_mem
Aug 18 08:33:14 jp58 kernel: TCP: out of memory -- consider tuning tcp_mem
To check the current limit, using cat read the following file and look at the 3rd value; 57350 in this case.
cPs# cat /proc/net/sockstat
sockets: used 8061
TCP: inuse 4823 orphan 481 tw 2159 alloc 4844 mem 57350 <--
UDP: inuse 13 mem 17
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
Then, we want to verify the 3rd value in the system protocol tcp_mem file. This should be big enough to accommodate the current values set.
$ cat /proc/sys/net/ipv4/tcp_mem
4096 4096 4096
If this needs to be changed, you can make permanent modifications by adding the TCP parameters to the /etc/sysctl.conf file and reloading the configuration, or restarting.
sysctl -p
As an example, here are some values that can be set. You may need to adjust these based on your system.
net.core.wmem_max = 16777216 net.core.wmem_default = 131072 net.core.rmem_max = 16777216 net.core.rmem_default = 131072 net.ipv4.tcp_rmem = 4096 131072 16777216 net.ipv4.tcp_wmem = 4096 131072 16777216 net.ipv4.tcp_mem = 4096 131072 16777216 net.core.netdev_max_backlog = 30000 net.ipv4.ipfrag_high_threshold = 8388608