Introduction
We also have the following guide which explains another possible technique for diagnosing high IO with the iotop command:
How to diagnose high IO and high load with iotop
One issue that you may notice with the above technique is that some processes could be performing high throughput I/O in very short bursts. When this occurs, those processes could be causing the majority of the I/O usage, but only appear on the screen for a very small amount of time, making it nearly impossible to detect.
The technique outlined in this resource makes use of the following options:
man iotop | grep -A1 "\-a,\|-o,\|-P,"
-o, --only
Only show processes or threads actually doing I/O, instead of showing all processes or threads. This can be dynamically toggled by pressing o.
--
-P, --processes
Only show processes. Normally iotop shows all threads.
--
-a, --accumulated
Show accumulated I/O instead of bandwidth. In this mode, iotop shows the amount of I/O processes have done since iotop started.
Using those options allows you to monitor processes more effectively if they're making short but high throughput bursts of activity.
Before beginning with the following procedure, you should review the following article so that you are familiar with what %IOWait is, which is critical to making use of this technique:
What is IOWait in relation to the sar command, and when should I be concerned?
Another thing that you should keep in mind when diagnosing high IO resource usage issues is the possibility of disk or filesystem corruption as being the cause. In those cases, you may find that there are no processes doing large amounts of IO, yet the IO usage is reported as being very high. You can learn how to diagnose disk or filesystem corruption issues with the following resource:
Identifying Disk Corruption and Filesystem Errors
Also please keep in mind that the investigation, diagnoses, and resolution of problems related to resource usage are tasks that are best handled by a systems administrator with the skills, training, and expertise required to do so for you. cPanel support is able to provide support for the use of the software that we create and provide, along with resolving problems that stem from defects in the software that we create. cPanel support is not able to perform systems administration services for you such as diagnosing resource usage problems. This guide is provided as a courtesy only.
Procedure
1. Login to the server via SSH or Terminal as the root user
2. Use the following command to review the recent %IOWait on the server
sar
3. If you see that there is a recent and sustained pattern of IOWait above 0, or especially above %5, there is a good chance that whatever is causing the elevated IO resource usage is currently active on the server.
4. If you believe that the IO usage is currently happening, issue the following command
iotop -aoP
5. Allow this command to run from anywhere between 30 seconds to a couple of minutes and actively monitor the output as it runs. You should be examining the output for patterns where there are large amounts of data being read or written by a process in a short amount of time. As a general rule, processes that are doing the most IO will float to the top of the list, although it is most helpful to pay attention to the actual amounts of data being read and written. Usually, a telltale sign of a process that is causing high IO resource usage is one that is reading or writing Megabytes or Gigabytes worth of data within a few seconds. Another thing to keep in mind when reviewing the output of this command is that filesystems with a journal such as EXT 3 and 4 may have a journal process that is consuming large amounts of IO. While this journaling process consuming large amounts of IO is a problem, it is generally a symptom of another process that is also doing lots of IO. You may want to focus on optimizing the IO usage of the secondary process before looking to reduce the IO of the filesystem's journaling process. If you do want to reduce the IO usage of this journaling process, you should review the following resource which goes over Filesystem Performance Tuning:
Comments
0 comments
Article is closed for comments.