Using perl 5.8.0, Linux 2.4.20-30.9, RedHat 9.0.
We have many .cgi's that allow privileged users to upload files to the server through a web browser. We've had these .cgi's for years and have never had any problems with them. Recently the files being uploaded are sometimes being given 600 permissions instead of 644 permissions, which they always used to be given. When you try to access the uploaded file set to 600 with a web browser you get a 403 Forbidden error. I was playing with it last night and uploaded the same file using the same .cgi about 10 times and every time it was given 600 permissions. Then, without changing anything, the next couple of times the file was uploaded with 644 permissions, then it went back to the 600 permissions. I'm really lost. Any help would be great.
after you upload the file have perl change the perms on the file.
dont rely on the umask.
but it sounds like some other process is messing around with the files.
We're working on changing the perl files to change the permissions manually. We just have so many that I was hoping there would be an easier way.
I agree it sounds like another process is screwing up the files, but I have no idea what process it would be or how to find it. If you have any insight on that I'd appreciate it. Thanks for your help.
unfortunetly i never used perl to upload files. i always use FTP.
1) when files are uploaded are they always uploaded to the same directory no matter what user signs in?
2) is there any corrolation to the user sign in to the unix account on the machine?
3) are all the users umask the same?
First, I really appreciate you helping me out with this. I've been on other forums and nobody has tried to help as much as you have.
1) The files that are being uploaded can go to hundreds of different folders. We have multiple sites using the uploading scripts, and within those sites users can choose to upload the files in various places.
2) The users on our sites have no correlation to our unix users. On the sites we're having the most problems with, there's an average of about 50 users, with all usernames defined by the user.
3) I didn't even know about umask until your last post. The only thing I know how to do with it now is type it in and get 0022 in response, which I think means all uploaded files will be given 644 permissions. I did man umask but I couldn't even figure out where to find the umask info. It listed all the bash builtins and I got confused.
We did figure out that we only have this problem at certain times. The only times we know of so far is sometime around 11:00 pm MST and between 5:00 pm - 6:00 pm MST. Between 5:00 and 6:00 it's very rare that the file gets uploaded with the correct permissions. We must have done 30 uploads in that time span and only 3 or 4 were uploaded correctly. We were running top at the same time, but we couldn't find any odd-looking processes running.
Thanks again for your help.
First, about umask... When a process creates a file, it uses the open() or creat() system calls. Both calls have a mode parameter to set the mode of the file. The mode is the number you're mentioning as 600 or 644. The kernel does not use the mode directly as it creates the file. Instead the file's mode is set to the expression "mode & ~umask". A process can change its umask value via the umask() system call. But it inherits its original umask from the parent.
This leads to my suspect: a faulty parent. The parent process, when overloaded or something, is spawning children incorrectly.
I'm not a web expert nor a perl expert. But I think that the parent of of cgi scripts is the webserver itself. The error in the parent should be researched. Maybe a new release or something is available?
If I'm right, a quick fix would be to modify the cgi scripts to set their umask to 0. This is probably as simple as inserting "umask(0)" or something like that as the first line. Maybe Optimus_P can chime in here with the perl syntax for that.
It really seems like that's the problem. We're still narrowing down any patterns, but it here's any consistency we've found so far.
Roughly 11:00 pm - consistently wrong permissions
5:00 pm - 6:00 pm - consistently wrong permissions
10:00 am - 10:30 am - not as consistent as 5-6, but still pretty bad
Maybe the parent is being hit with another process on the server at these times, and is causing it to do what you described above.
We don't have any crons running at these times. I'm using top during these times to try to find any wierd processes, but have been unsuccesfull. I'll start looking into the parent for .cgi's and see if I can find something.
Thanks.
umask should be in your %ENV.
but befor we delv into that.
i would check the permissions of the parent directories.
also, if you happen to be writeing to a samba or sharity share permissions tend to get wacky from time to time.
being it happens at a semi regular intervel. (look like every 6-7 hours or so)
is it always the same upload directory you are haveing any errors? or is it always a random user in a random directory?
implamenting a new function to chmod the file in perl should just be adding a new function that accepts 1 paramerter and that should be the fullpath to the file.
at a command prompt umask -S should show you the current umask perms in a more readable format.
use@host$ perldoc -f umask
umask EXPR
umask Sets the umask for the process to EXPR and returns
the previous value. If EXPR is omitted, merely
returns the current umask.
The Unix permission "rwxr-x---" is represented as
three sets of three bits, or three octal digits:
0750 (the leading 0 indicates octal and isn't one of
the digits). The "umask" value is such a number
representing disabled permissions bits. The
permission (or "mode") values you pass "mkdir" or
"sysopen" are modified by your umask, so even if you
tell "sysopen" to create a file with permissions
0777, if your umask is 0022 then the file will
actually be created with permissions 0755. If your
"umask" were 0027 (group can't write; others can't
read, write, or execute), then passing "sysopen"
0666 would create a file with mode 0640 ("0666 &~
027" is 0640).
Here's some advice: supply a creation mode of 0666
for regular files (in "sysopen") and one of 0777 for
directories (in "mkdir") and executable files. This
gives users the freedom of choice: if they want
protected files, they might choose process umasks of
022, 027, or even the particularly antisocial mask
of 077. Programs should rarely if ever make policy
decisions better left to the user. The exception to
this is when writing files that should be kept
private: mail files, web browser cookies, .rhosts
files, and so on.
If umask(2) is not implemented on your system and
you are trying to restrict access for yourself
(i.e., (EXPR & 0700) > 0), produces a fatal error at
run time. If umask(2) is not implemented and you
are not trying to restrict access for yourself,
returns "undef".
Remember that a umask is a number, usually given in
octal; it is not a string of octal digits. See also
"oct", if all you have is a string.
Thanks for the advice. To answer your question, it's always a random user in a random directory. We're about half-way through changing our scripts to manually chmod the uploaded file, so I think I'm going to just let this problem go. I have one last thing to post; maybe this will be the secret ingredient needed to figure out this problem. We ran into the problem consistently again (which makes it roughly every 3 hours now), so I did
ps -ef > psbad.txt
and restarted apache. We had tried this a couple of weeks ago and it fixed the problem then. Again, it fixed the problem. So I did
ps -ef > psgood.txt
I'll load those files zipped in case anyone wants to look at it to see if they can figure out what the problem is. It looks to me like the same processes are running at both times, which makes me all the more confused. Thanks everyone for helping me out with this.
I can't figure out how to link to the file I uploaded, so you can get to the .txt's at http://www.proinnov.com/psbad.txt and http://www.proinnov.com/psgood.txt
apsedu 31656 28302 0 12:21 ? 00:00:00 [sideA.cgi <defunct>]
apsedu 31657 28302 0 12:21 ? 00:00:00 [sideB.cgi <defunct>]
chsmain 31658 31001 0 12:21 ? 00:00:00 [showfac2f.cgi <defunct>]
you dont happen to get any errors on your cgi scripts do you?
No, we don't get errors with those scripts. I've always been curious why they occasionally come up as scriptname <defunct>, but have never really payed attention to it because they always work.
being that they are httpd servers that are dieing befor the script closes could be a reason for your problem.
i would try and find out why the server is dieing befor the child. it could be a source of your problem.
the Ppid is
root 8553 1 0 Feb19 ? 00:00:00 /usr/sbin/httpd
apache 31001 8553 0 12:19 ? 00:00:00 /usr/sbin/httpd
chsmain 31658 31001 0 12:21 ? 00:00:00 [showfac2f.cgi <defunct>]
So I started roaming around looking for apache httpd bugs and found one that says
We're running apache 2.0.40. Does that mean that when these files are being uploaded they could be given the umask of a user that doesn't exist, which in turn would give the file the 600 permissions?
since the snipit specificly says that STDOUT can goto the wrong client when useing a redirect i would lean toward no i dont think this would be the problem. But it would be unwise to rule it all out.
the default umask should be in /etc/profile.
I just can't let this go! We updated Apache this weekend and still have the problem. It just makes me so happy. I looked in /etc/profile and and couldn't find anything about umask. I did locate umask and got this is response
/usr/lib/perl5/5.8.0/i386-linux-thread-multi/auto/POSIX/umask.al
/usr/share/man/man1/umask.1.gz
/usr/share/man/man2/umask.2.gz
/usr/share/man/man3/getumask.3.gz
I checked out the first item and got this
[root@maverickds POSIX]# cat umask.al
# NOTE: Derived from ../../lib/POSIX.pm.
# Changes made here will be lost when autosplit is run again.
# See AutoSplit.pm.
package POSIX;
#line 581 "../../lib/POSIX.pm (autosplit into ../../lib/auto/POSIX/umask.al)"
sub umask {
usage "umask(mask)" if @_ != 1;
CORE::umask($_[0]);
}
# end of POSIX::umask
1;
When I view POSIX.pm, it's a very long script that has an instance of both chmod and umask. Example:
sub chmod {
usage "chmod(mode, filename)" if @_ != 2;
CORE::chmod($_[0], $_[1]);
}
and
sub umask {
usage "umask(mask)" if @_ != 1;
CORE::umask($_[0]);
}
I have no clue what the script does, though. I can load it somewhere if that would help. I also looked at AutoSplit.pm at search.cpan.org and it seems to have something to do with the problem I'm having, but I've gotten into a field I've never been before. I don't even know if what I'm looking at is relevant to my problem. Is there a server error log file that I might be able to get some error messages from? The only ones I know of are the site-based error logs.
Thanks
in my installation of solaris umask is defined at the bottom of /etc/profile. (this sets it globally for all users)
umask and chmod are built into your default perl installation. well rather the way to use them is built into perl.
the perl modules ie *.pm are written in perl. most of the time if you are looking for a way to use a function in perl use the perldoc utility.
use perldoc to find the manpages for them.
perldoc -f chmod
perldoc -f umask
it will explane how to use them.
then its a matter of how you want to implament it.
like i said in my earlier post i would implament another function when the user uploads a file to a dir.
you need to capture the dir and filename then pass that info to the chmod command.
chmod LIST
Changes the permissions of a list of files. The
first element of the list must be the numerical
mode, which should probably be an octal number, and
which definitely should not a string of octal
digits: 0644 is okay, '0644' is not. Returns the
number of files successfully changed. See also
"oct", if all you have is a string.
$cnt = chmod 0755, 'foo', 'bar';
chmod 0755, @executables;
$mode = '0644'; chmod $mode, 'foo'; # !!! sets mode to
# --w----r-T
$mode = '0644'; chmod oct($mode), 'foo'; # this is better
$mode = 0644; chmod $mode, 'foo'; # this is best
You can also import the symbolic "S_I*" constants
from the Fcntl module:
use Fcntl ':mode';
chmod S_IRWXU|S_IRGRP|S_IXGRP|S_IROTH|S_IXOTH, @executables;
# This is identical to the chmod 0755 of the above example.
I didn't realize that's all POSIX.pm was doing. I know how to chmod and umask with perl, I'm just really annoyed by this bug and wanted to figure it out. I've spent too much time on it now, so I'll just change the scripts to chmod manually.
Thanks Optimus_P and Perderabo for all your help on this.