Disk 100% full, but still multiple GB left on device

DennisMidjord

• October 29, 2019 04:07

Hi, We have this odd problem where df reports the disk as 100% full, but we could still have 20GB free on that disk. When the disk is 100% full, we can no longer write anything to the disk. Any idea why this happens?

• 

  cPanelLauren

  • October 29, 2019 22:04

  Hi @DennisMidjord Can you show me an example of this? as well as the output of the following: cd / du -h --max-depth=1
  

  0
• 

  DennisMidjord

  • October 30, 2019 20:32

  Hi @cPanelLauren I don't have an example right now - we usually extend the disk when we see there's an issue. The 'df -h' command will just report 100% full on /home, even though it has multiple GB left.

  0
• 

  cPanelLauren

  • October 30, 2019 21:05

  Hi @DennisMidjord There was a point I was going to make with that. This is a common concern and the best example is shown If you use df and du you'll a lot of times encounter a situation where they show differing amounts. This is due to the way that they each calculate disk space and would account for why you are seeing available disk space when df is telling you there's nothing available. df measures disk space in 512-byte disk sectors and on some file systems like zFS df is sometimes inaccurate in providing enough information to determine if the file system is running out of space. df is measuring File System usage The great comparison is du which a lot of times can take a lot longer because it shows File Space usage Another thing to keep in mind as well is that df adds deleted items still held in memory to the calculation as they are by all purposes still present until they're no longer held in memory. You can see this usage by running something like: lsof |grep '(deleted)'
  

  0
• 

  DennisMidjord

  • October 30, 2019 21:52

  du -h shows this: 614G ./home df -h shows this: /dev/mapper/cloudlinux-home 676G 614G 29G 96% /home

  0
• 

  cPanelLauren

  • October 30, 2019 22:38

  du -h shows this: 614G ./home df -h shows this: /dev/mapper/cloudlinux-home 676G 614G 29G 96% /home

  
  So that difference there is what I was referring to. The difference between File Space usage and File System usage. DF Is calculating File System usage in blocks as well as counting deleted items held in memory where DU isn't. Sometimes this will lead to potentially showing more disk space used.

  0
• 

  DennisMidjord

  • October 30, 2019 22:39

  But it showed the same amount used? :D Both commands showed 614GB used.

  0
• 

  cPanelLauren

  • October 30, 2019 22:41

  Oh! I misread it without the bar present: Filesystem Size Used Avail Use% Mounted on
  In this case then, it does show you have available space, but I'm assuming this isn't an example where it's reporting the server is out of space?

  0
• 

  DennisMidjord

  • October 30, 2019 22:42

  Correct - but it did last week. I didn't save any information from back then, unfortunately.

  0
• 

  DennisMidjord

  • November 19, 2019 12:39

  Hello, @cPanelLauren We now have the issue again. Could you please take a look at ticket 13798409? I've added a temporary login for FTP so you can check the issue.

  0
• 

  DennisMidjord

  • November 19, 2019 13:58

  Hi, I think I figured out why. PureFTP has a setting called MaxDiskUsage set to 99 (%). On large partitions, this can be many gigabytes. I'm gonna set it to 100.

  0
• 

  cPanelLauren

  • November 19, 2019 15:53

  Hi @DennisMidjord That's interesting, were you only seeing the usage in FTP?

  0
• 

  DennisMidjord

  • November 19, 2019 16:20

  No - we were only seeing the error message with FTP (apparently). I had understood that it was an issue with writing anything, but it seems like it was just FTP. Writing anything to the disk from cPanel or uploading a database didn't cause any issues which led me to investigate whether it was actually just the FTP server.

  0
• 

  cPanelLauren

  • November 19, 2019 16:24

  Hi @DennisMidjord Ahhh ok, that makes sense. Nice catch there!

  0

Please sign in to leave a comment.