Can't delete directory

Hi,

I have a problem on Solaris 10 regarding deletion of files

when I try to recursively delete the directory /opt/Tivoli as root, I get the following error message:

rm: Unable to remove directory Tivoli/lcf/dat/1/no-del/bin/solaris2/TME: File exists
rm: Unable to remove directory Tivoli/lcf/dat/1/no-del/bin/solaris2: File exists
rm: Unable to remove directory Tivoli/lcf/dat/1/no-del/bin: File exists
rm: Unable to remove directory Tivoli/lcf/dat/1/no-del: File exists
rm: Unable to remove directory Tivoli/lcf/dat/1: File exists
rm: Unable to remove directory Tivoli/lcf/dat: File exists
rm: Unable to remove directory Tivoli/lcf: File exists
rm: Unable to remove directory Tivoli: File exists

ls -ldR:
.:
total 2
230329 drwxr-xr-x 3 root other 512 Mar 8 16:47 lcf

./lcf:
total 2
230335 drwxr-xr-x 3 root other 512 Feb 28 13:45 dat

./lcf/dat:
total 2
230336 drwxr-xr-x 3 root other 512 Mar 8 16:47 1

./lcf/dat/1:
total 2
230364 drwsrwxrwt 3 root other 512 Mar 8 16:41 no-del

./lcf/dat/1/no-del:
total 2
230522 drwsrwxrwt 3 root root 512 Mar 7 15:31 bin

./lcf/dat/1/no-del/bin:
total 2
230523 drwsrwxrwt 3 root root 512 Mar 8 16:31 solaris2

./lcf/dat/1/no-del/bin/solaris2:
total 2
234624 drwsrwxrwt 3 root root 512 Mar 8 16:31 TME

./lcf/dat/1/no-del/bin/solaris2/TME:

I can mv it in the same fs , when I move it to /tmp for example, the destination directory is created, but the source dir still remains

fuser -c . in /opt works
fuser -c . in /opt/Tivoli gives the following error msg: .: fuser: Invalid argument

removing /opt/Tivoli/lcf/dat/1/no-del/bin/solaris2/TME with rmdir result in

rmdir: directory "/opt/Tivoli/lcf/dat/1/no-del/bin/solaris2/TME": Directory not empty, but it is empty

output of mount:

/opt on /dev/md/dsk/d41 read/write/setuid/devices/intr/largefiles/logging/xattr/onerror=panic/dev=1540029 on Tue Jan 30 06:16:36 2007

I can't unmount the fs for a check, any ideas?

thanks in advance

funksen

What type of filesystem is this? Use "df -n" to find out. Cd to that TME directory and do:
ls -ld .
ls -la
Cd to some other directory in that filesystem and do:
mkdir TME2
cd TME2
ls -ld .
ls -la

I don't believe that /opt is mounted by default in single user mode. Can you reboot to single user mode to fsck that filesystem?

thanks for the answer perderabo

for the damaged dir:

# ls -ld .
drwsrwxrwt 3 root root 512 Mar 8 16:31 .

# ls -la
total 4
drwsrwxrwt 3 root root 512 Mar 8 16:31 .
drwsrwxrwt 3 root root 512 Mar 8 16:31 ..

new dir TME2:

# pwd
/opt/TME2

# ls -ld .
drwxr-xr-x 2 root root 512 Mar 9 13:34 .

# ls -la
total 4
drwxr-xr-x 2 root root 512 Mar 9 13:34 .
drwxr-xr-x 17 root other 512 Mar 9 13:34 ..

df -n:

/opt : ufs

seems like there is a third hardlink that points to that directory

I can't reboot the system nor unmount /opt cause it's a productive oracle machine, and oracle lies on /opt/oracle

cheers
funksen

That link count of 3 is crazy and is probably the cause of the problem. There may be another directory hardlinked to it. You can do an "ls -lid on the directory to get the inode number. Then a "find /opt -inum..." should find all hard links. Most likely the link count is wrong and you will not find a third hard link. There are some extreme measures that might fix this, but you risk a panic if you try them. Since you cannot reboot, my advice is to empty /opt/Tivoli as fully as you can. Then
mv /opt/Tivoli /opt/damagedTivoli
chmod 0 /opt/damagedTivoli
mkdir /opt/Tivoli
and then take it from there. This is the safest of several unsafe courses of action. Operating with a filesystem that is known to be damaged is a little risky.

I think you are right, playing with the damaged fs is too risky

perhaps I can get our customer to give us a short downtime, for now I rename and recreate the directory

on monday I try to find the third hardlinked dir, but as you said it's very unlikely to find something

thanks for your help

cheers
funksen