du and df do not match on NFS share

UNIX for Advanced & Expert Users
1

Here is the scenario...

NFS share that is accessed every few minutes by approx 70 systems (AIX 5.3/6.1). Filesystem space is being eaten up rapidly according to df however du numbers really never change. lsof and fuser cannot see any unlinked files on either the NFS server or remote clients. Restarting nfsd does not resolve the issue. However, attempting to umount the underlying JFS2 filesystem fails (device busy) obviously but magically frees up the "used" disk space...

Any thoughts on this?

2

Guessing that you're seeing a cache consistency problem. A common "feature" of using NFS. With NFS you are seeing an abstracted view of data (clients for example are not guaranteed to see the same view.. and what's physically present on the storage doesn't have to match up with any given client/server view). But we call it a filesystem and use the same kind of utilities on it that we use on "normal" filesystems.

There might be something more going on... I'm just taking a very wild guess.

3

There is some extravagant scripts running on all 70(ish) LPARs from that share which I think is likely somehow creating hidden unlinked files or a whack of cache as you suggest. I didn't write them so I am digging through and correcting potential culprits in hopes of stumbling upon the issue. Nothing jumps out at me but there is many lines of code to look through.

4

  • the problem you are facing is most likely being caused by a blocksize mismatch ;

  • every filesystem has its own blocksize (defined or calculated upon creation) ;

  • you will have to unmount you nfs , and before remounting it, discover its native blocksize (on the exporting host), and use the proper mount options to respect this value ;

  • like , for example :

rsize=8192,wsize=8192
  • you can add these to your fstab(Linux) or filesystems(AIX) file, or pass them along any other "mount options" you might have on command-line ;

HTH

good luck, and success !
_________________________________
alexandre botao
<< botao {at} unix {dot} sh >>
"comets never dodge"