Websites slow due to I/O

Symptoms

Websites may seem sporadically slow despite no errors/failures with services and without elevated CPU or Memory usage. The only significant performance metric that seems elevated is Disk I/O.

Description 

Identifying high Disk I/O is commonly accomplished using the sar command. The "%iowait" column designates Disk I/O. Expected values are less than 1.00 or 2.00. Occasional spikes in Disk I/O are expected during planned disk-heavy operations (such as nightly backups). Sustained spikes or spikes during other operating hours would be considered unusual.

Workaround

Using the iostat command, you can get an idea if you are experiencing iowait:

sar 1 5 

This command runs sar five times in one-second intervals.

If you see numbers in the iowait column that is elevated and persistent, your websites may be slow due to high iowait.

Other articles that are helpful for I/O wait troubleshooting are:

What is considered high I/O?

How to diagnose high I/O and high load with iotop

How to diagnose high loads with the "iostat" command