How to diagnose high server loads

Question

How do you diagnose high server loads?

Answer

You will first want to determine if the high server load is related to memory, CPU, or I/O; you can do this with the sar command. Before you understand the sar output, you should know what is considered high memory, CPU load, and high I/O. The following links will help do so:

What load is considered high?

What is considered high I/O?

Understanding Memory Usage On Linux Operating Systems

Determining the cause

You can use the sar utility to determine if the load is due to memory, high CPU load, or high I/O. The sar command will provide you with historical and up-to-date information on the system statistics.  

How to diagnose high loads with "sar" command

In the sar output, you will see a steal column; if this is high, you will want to review the article:

What is cpu steal and when should I be concerned about it?

If you experience high memory or CPU usage, you can use the ps command to list all processes and sort them by usage. The list sorted by usage can help you determine which processes are the culprit quickly.

How to diagnose high loads with the "ps" command

From there, you can use lsof to view the files the process is using.

How to utilize the lsof command

And then strace the process to see what the process is doing.

Basic guide for using strace to diagnose software problems on Linux

To see a live preview of the system's processes, you can use the top command. 

How to diagnose high loads with "top" command

If the issue is determined to be high I/O, you can check what processes use the most I/O using iotop, iostat, and iowait. iotop will let you review the I/O usage live, similar to the top command.

How to diagnose high I/O and high load with iotop

How to diagnose high loads with the "iostat" command

What is IOWait relative to the sar command and when should I be concerned?

Common causes of high load

• PHP

PHP High load is usually due to high traffic, but can sometimes also be related to php-fpm needing to be tuned.

PHP-FPM Performance Tuning Basics

• Apache

High loads caused by httpd, you will want to review the Apache scoreboard first.

Understanding the Apache scoreboard

How can I tell if Apache is experiencing a DDoS attack?

How do I troubleshoot Apache MaxRequestWorkers errors?

• MySQL

The high load caused by MySQL can be due to a low query cache, or possibly none.

High load averages and MySQL's query cache

For more information, you should reach out to your System Administrator or hire a qualified System Administrator if you do not have one.

System Administration Services