du and dfspace reporting

Hi,

I am using SCO UNIX version 6.0.0 release 5. I am using du and df space to see the used space in the / partition. I am using du -k option to get count in 1024 k so that it directly makes kb. In dfspace I subtracted the used mb from total size mb which should be the used space and then convert to GB. But the output from du -k (converted to gb ) donot match this output of dfspace (converted to gb)

Here it goes :

> du -k  =  38339676 (kb) = 38.3 gb
 
> dfspace 
 
788718.80 MB of 851248.99 MB available 

so 851.2 gb - 788.7 gb = 61gb approx. used space in /

I suppose , these should match, but they didn't .

Any help is appreciated.

regards,

dexter

Hi,

< http://www.unix.com/unix-dummies-que...isk-space.html >

The links explains that df will use the block size to give the usage details. My system block size is 1024bytes and for that the outputs are not matching. I already know that block size is something which affects the usage reported, but my question was after implementing that logic. I used DU with block size of 1024 and df space is also using 1024 b block size (or let me know how to check df's used block size)

My query still remains...

regards,

Dexter

---------- Post updated at 10:41 AM ---------- Previous update was at 10:34 AM ----------

And BTW, I talked about "dfspace" and not "df". Reporting by dfspace is in MB . I donot know the underlying princile of how it reports the usage.

Dexter

You should post both

du -k

and

df -k

precise output.

dfspace

is apparently a just df wrapper.
Is the discrepancy still there after a reboot ?

On my partition /

du -k | tail -1 gives

38253272 (last line is the summantion )

df -k -v gives

/          /dev/root            871678974  63965476 807713498     8%

i.e it shows used blocks (1024) = 63965476 whereas du shows 38253272

I know that du is primarily for directory and file contents and df for partition but 38 gb and 63 gb are a big difference . Is 25 gb being taken by filesystem contents ???

regrads,

dexter

From the man page for du. (SCO 6.0.0)

@dextergenious: again, is this discrepancy still there after a reboot ?

Is your file system clean ? (fsck)

I don't know how to explain this any better than the thread that I supplied a link to as the first reply on this thread.

Can someone help me word this differently to dextergenious please?

I'm lost for the words (for once)!!

(The difference is due to allocated blocks (as per inode) vs actual file sizes (bytes) and the "wasted" space in between.)

How do you say that guys? I thought the link was self-explanatory but we need to make it more clear.

The point is the OP poster doesn't complain about a discrepancy between the actual file size and what du is reporting, which is what the quote is about, but about the discrepancy between df and du.

df is telling 63 GB are used, this is global file system view of free space.
du is telling 38 GB are used, this is a hierarchical, file by file computation.

The du manual page just tell the actual data file size is smaller than 38 GB but gives no clue about the 25 GB difference.

There are many reasons for having such a difference, and actually df always tells more space is used than du but the difference seems here too large so I'm trying to exclude the most usual ones with my still unanswered questions.

@dextergenious: another question: are there other file systems mounted on / and where are they ?

@jlliagre......thanks for the help here.

My understanding is (correct me if I'm wrong) that df (what we all look at as sysadmins to see how full filesystems are) regards all disk sectors (blocks) allocated as unavailable (which they are).

One file may contain, say, 4 bytes, but occupy the minimum 512 bytes (1 block), therefore the discrepancy can be huge if there are a lot of small files. The larger the files (eg. big Oracle dB) will mean occupied blocks and actual file bytes used look closer.

The df tells you filesystem occupied blocks but du is basically adding up file sizes which will be different.

Anyone disagree with that? Useful discussion this!!

I'm afraid I have to fully disagree. du isn't adding up file sizes but adding up file disk usage (as its name implies) i.e. a 4 bytes file will use at least one block.

What du isn't adding up is size it can't see, like removed but still used files, file system specific areas (e.g. journal, superblocks, ...), files hidden by an overlay mount, files unreachable by a non root user due to permissions, blocks lost due to file system corruption, and so on ...

1 Like

Yea, right. Got it! That makes perfect sense.

I don't know if this is still true, but many years ago df would report less free space available for non-root users than for root users and some filesystem code would return ENOSPC errors when an application wasn't running as root even when about 10% of the space on the filesystem was still unallocated. (This was intended to allow a sys admin to have a little wiggle room to try to clean up an expanding filesystem that was almost almost full before the system became a paperweight.)

Reference post #1. Which refers only to the root filesystem.
Just in case this is the old issue with du following links.
Please post:

du -k /
du -kx /
df -k /

How many files etc. do you have in the root filesystem?

find // -xdev -print | wc -l

Hello guys,

Thanks for joining my discussion. hicksd8 didn't get my question exactly. jlliagre got me right.

jlliagre

< du isn't adding up file sizes but adding up file disk usage (as its name implies) i.e. a 4 bytes file will use at least one block.
What du isn't adding up is size it can't see, like removed but still used files, file system specific areas (e.g. journal, superblocks, ...), files hidden by an overlay mount, files unreachable by a non root user due to permissions, blocks lost due to file system corruption, and so on ...
>

Both du and df add any file between 512 bytes as 1 block . So the point is not of "the space of actual file size and the lefet over space in a block " . With both taking them as 1 block and counting the blocks based on that, my question was what exactly tyhe data du is not showing (relating to a partition) which df is adding (around 25 gb). Please note that I DONOT have any left over file opened which is occuping space . This I checked with

lsof | grep tmp

System seems to show this difference without having any such left over process or other discrepency.

I didn't rebooted the server yet and that is teh last option I am left with.

Thanks and regards,

Dexter

If one of your non root file systems (eg: /var, /opt, /export, ...) is mounted on a previously non empty directory, this directory size is unaccounted by du .

I only have /stand partition on / partition . This is of very small size , almost negligible to whole / size. I think I shall report status again after reboot, which may be tommorow.

Thanks and regards,

Dexter

What matters here is not /stand file system size but the size of what might be present under the /stand directory when no file system is mounted there.

My stand is only in mb's .

# # du  -k | tail -1
11540   .                                                     <- 
# df -k -v |
Mount Dir  Filesystem              blocks      used      free   %used
/          /dev/root            871678974  64133868 807545106     8%
/stand     /dev/boot                40959      6413     34546    16%
 
                                                        ^

So , the stand is 11.5 mb by du and 6.4 mb by df -v .

Also see :

# dfspace
/         : Disk space: 788618.26 MB of 851248.99 MB available (92.64%).
/stand    : Disk space:    33.73 MB of    39.99 MB available (84.34%).
 
# mount
/ on /dev/root read/write on Mon Sep 10 10:04:57 2012
/stand on /dev/boot read/write on Mon Sep 10 10:07:36 2012
/proc on /proc read/write on Mon Sep 10 10:09:39 2012
/dev/fd on /dev/fd read/write on Mon Sep 10 10:09:39 2012
/dev/_tcp on /dev/_tcp read/write on Mon Sep 10 10:09:39 2012
/system/processor on /processorfs read/write on Mon Sep 10 10:10:18 2012

regards,

Dexter

Not that it is necessarily the root cause of your problem but I'm afraid you are still missing what I mean. You need to unmount /stand to know if this directory is empty or not in the first place.