Permission denied, but user is owner and has group ownership too

UNIX for Advanced & Expert Users
1

Folks,
I have a problem with a particular file, that seems to have some kind of lock on it, that takes around 1 hour approx to timeout. I have used lsof and nothing has an open file handle on it, yet I cannot open it. My user/group owns the file and I can create edit/delete files in the same directory as the same user. Running on SUSE 10, with weblogic 10.

     I have attached an strace below, and there seems to be an issue with accessing the file.   I used the command "strace cat ./\_WLS_MANAGED01000000.DAT"  and have a sample of the output below.

      If I wait for over an hour, the "lock" seems to disapear, so I discounted permissions in the path.  I am unable to find any tool\(I have a request to install dtrace, but that will take a few days\), that can find what is holding a "lock" or otherwise.   Any help appreciated.

I found the following documentation on what could be potentially causing it, and have been able to discount all of them, though, there is something wrong with that theory, as one of them must be causing an issue.....

          Any help or pointers greatly appreciated....


 [EACCES]       Search permission is denied for  a  component
                of the path prefix.

 [EACCES]       The required permissions \(for reading  and/or
                writing\) are denied for the named file.

 [EACCES]       O_CREAT  is  specified,  the  file  does  not
                exist, and the directory in which it is to be
                created does not permit writing.

 [EACCES]       A device to be opened for writing  is  physi-
                cally write protected.

= 0
open("./_WLS_MANAGED01000000.DAT", O_RDONLY) = -1 EACCES (Permission denied)
write(2, "cat: ", 5cat: ) = 5
write(2, "./_WLS_MANAGED01000000.DAT", 28./_WLS_MANAGED01000000.DAT) = 28

2

Your are in a directory, cd .. to go up one level and then show the output of

ls -ld [directory_name]

Are there any mountpoints in your path? Is this an NSF mount?

3

ls -ld
drwxr-xr-x 4 scott scott 4096 2009-07-24 13:26

It is not an NFS mount.

There is a mount piont in the path.

mount point is /opt/mysrv
cd /opt
ls -ld
drwxr-xr-x 13 root root 4096 2009-07-10 21:53
ls -al
drwxrwxr-x 7 bea mysrv 4096 2009-07-22 15:39 mysrv

Directory is
/opt/mysrv/weblogic10/user_projects/domains/mydomain/servers/managed01/data/store
ls -al
drwxr-xr-x 2 scott scott 4096 2009-08-19 19:24 default

             thanks for any help.....

                          Sc0tt.....
4

access comes & goes.
Did you check ctime? as in did something chmod a file or directory?

If ls -lcd [each directory in the file path] is recent (like during when you had problems)
it means something is running chmod on a path component.

You do not get EACCES from open O_RDONLY when a mandatory file lock is active. fcntl will return EAGAIN or EACCES when testing a lock. Your trace output shows clearly that open is failing. This is for standard linux filesystems.

This isn't a qfs filesystem for some bizarre reason is it? What filesystem is it? was it mounted -o mand? check /etc/mtab.

Use inotifywatch to monitor changes in file/directory metadata.

5

Directory and file locking is a feature of BEA-Weblogic software.
Maybe you need to speak to Oracle about what you can and cannot do in these directories.

BEA Product Documentation

6

That - I did not know. We don't use it. Thanks.

7

Hi,
Yes I have been speaking with the weblogic folks, I had two avenues of thought, one was that is was a unix issue, and the other that it was a weblogic iasue.

Webogic offerred a workaround of sorts. The file is a weblogic file store, and we have moved the file store "Synchronous write policy" from "Direct Write" to "Cache-Flush". However there is a performance hit in doing so, and we effectively bypass the "direct I/O" in weblogic.

    I am still working on it, and all the comments have helped, in at least eliminating what it is not.  I can kill the jvm and still the file has some kind of a lock on it.  But I am wondering what tools can I use to find what has the lock and how to stop it.  Maybe Weblogic will come back with a better answer, which I will post here, but till then, I am still going down the unix route....