Disk Full

You've got an alert that a disk partition is full or rapidly filling and you can't find the culprit. You've checked all the usual places and nothing stands out.
One programming technique that can lead to this type of situation involves opening a file on the filesystem (typically in /tmp), then unlinking it (deleting) without closing the file handle. This gives you a file handle that:

• bypasses quotas
• is nearly undetectable
• confuses the heck out of admins
• gets cleaned up properly when the process exits normally or dies

Problems with this approach:

• it bypasses quotas
• it's nearly undetectable
• it confuses the heck out of admins
• a runaway file writing process can fill up the filesystem

How do you tell if you have this type of situation? You can look in every processes' directory tree under /proc for file descriptors that have been deleted...they look similar to:
Probably the fastest means of uncovering these type of files is with the 'lsof +L1' command. Here's sample output:
As you can see, there is a 2.5 gig file in /tmp. Stopping the appropriate process will clear them up or, if you can't find the right one, reboot (sometimes just taking the system to runlevel 1 is sufficient.)
Note: this is NOT a technique developed by system crackers nor an indication that a system has been compromised although it can be said that crackers may use this technique to hide their activities.