My first post here. I've been told certain things regarding HP-UX's .sh_history file which I'm not so sure I agree with completely.
These things are that the history file gets flushed on every clean shell exit (plausible, but I doubt it seeing as there are more than 11 thousand lines in the file as it stands) and that it's not very dependable (again, probably true, however if you're looking for a dead-standard command to make sure it wasn't run over a relatively long period of time, it should do just fine no?).
My take on it is that if there are thousands of lines, chances are that history file isn't getting flushed. If it's meant to, it's not configured correctly.
As for hunting down a supposedly repeated command ran by a certain user, if not the history file then what else can one check exactly (it's been established that no other means of recording commands have been put in place)?
I'm sorry if I come across as a bit irritated, especially as this is my first post on here, but that's because I am a bit irritated hehe! :wall:
Sorry, I didn't get your question...
If its flushing of .sh_history I saw things of the sort on AIX, maybe old HP-UX .. but not for users nor have I seen for root since HP-UX 10.XX there are some case at reboot (but we enter in customization stuff - not by default...).
If more about size, then the variable HISTSIZE is the culprit...
As you can see I have included in my .profile a line to write a "LOGIN " with date in my .sh_history to help me when I go through looking what silly things I could have typed or "I already been through this.." using the date to point me in the history.
If your wish is to flush, I suppose there are enough customization tips out on the net to search and I find that not all that wise and rather preferring to limit the size if needed, it can happen that it get corrupted (especially when big ) depending of what you have in your environment (.kshrc stuff, mine is almost 100 lines...) and so I being a little paranoid keep all when possible: I regularly reset .sh_history after saving the previous as .hist.vbe.<last-date-of-use> e.g:
ant:/home/vbe $ ll .hist.*
-rw-r----- 1 vbe bin 105230 Jun 3 2004 .hist.040603
-rw------- 1 vbe bin 6928 Sep 21 2004 .hist.040921
-rw------- 1 vbe bin 20924 Nov 8 2004 .hist.041108
-rw------- 1 vbe bin 80048 Feb 25 2005 .hist.050225
-rw------- 1 vbe bin 111570 Apr 25 2005 .hist.050424
-rw------- 1 vbe bin 119080 May 3 2005 .hist.050503_important_utmp
-rw------- 1 vbe bin 129466 May 25 2005 .hist.050524
-rw------- 1 vbe bin 151444 Jun 14 2005 .hist.050614
-rw------- 1 vbe bin 201640 Aug 25 2005 .hist.050825
-rw------- 1 vbe bin 214200 Aug 31 2005 .hist.050831
-rw------- 1 vbe bin 257188 Jan 16 2006 .hist.060115
-rw------- 1 vbe bin 255804 Jan 24 2006 .hist.060124
-rw------- 1 vbe bin 112040 Jul 7 2006 .hist.060707
-rw------- 1 vbe bin 23350 Sep 11 2006 .hist.060911
-rw------- 1 vbe bin 38371 Sep 22 2006 .hist.060921
-rw------- 1 vbe bin 38096 Oct 11 2006 .hist.061011
-rw------- 1 vbe bin 128874 Jun 22 2007 .hist.070622
-rw------- 1 vbe bin 194172 Oct 18 2007 .hist.071018
-rw------- 1 vbe bin 9386 Dec 5 2007 .hist.071205
-rw------- 1 vbe bin 94076 May 13 2008 .hist.080513
-rw------- 1 vbe bin 101366 May 16 2008 .hist.080516
-rw------- 1 vbe bin 103430 Mar 26 2009 .hist.090326
-rwx------ 1 vbe bin 122562 Sep 28 2009 .hist.090928
-rwx------ 1 vbe ocirt 83776 Apr 23 2010 .hist.100423
-rwx------ 1 vbe ocirt 140038 Sep 10 2010 .hist.100910
-rwx------ 1 vbe ocirt 230816 Apr 7 2011 .hist.110406
-rwx------ 1 vbe ocirt 305684 Aug 12 15:44 .hist.110812
-rwx------ 1 vbe ocirt 327478 Aug 25 15:58 .hist.110825
-rwx------ 1 vbe ocirt 342056 Sep 1 15:43 .hist.110901
-rwx------ 1 vbe ocirt 381352 Oct 11 10:11 .hist.111011
-rwx------ 1 vbe ocirt 357726 Oct 14 14:31 .hist.111014
Since you ask where else can we find what command were passed, except if user used script command before, there are no ways of knowing unless you, you have put something in place ( I have... but you know I am a bit paranoid...). If you ask, I believe you have good reasons, and I then understand your feelings, yes there are "guilty" users that erase all their activity on logout so when you ask "who has been and modified..." they join the "not me" chorus (explaining my paranoid attitude...) so you being responsible get the blame and your boss couldn't care less when you reply but all the system engineers and sysadm have the root passwd/access...
A few tips:
when strong suspicion, copy as root (using cron? the .profiles and .sh_*of the users you want to have an eye on - and root -very important!) in a place where root only has access (use user/group bin and put yourself in it and perm 770 ).
you can always try to use your backup software to see (can be very helpful some times)
etc..
We set $HISTFILE to include username, login date and time, and tty. Thus each session has a separate file - even if the user logs in multiple times. The previous history is not available to the user after log out. It is retained for analysis in the event of problems.
I'm stuck with a sysadmin telling me that the history files aren't sufficient to establish whether a certain account ran certain commands or not, which just blows my mind. Were it my system, the history files would be the very first thing I'd check.
Re-reading your first post. The Shell History is a feature of the particular Shell you run. It is not present in every Shell and it is not specific to HP-UX.
Your sysadmin is right. If the command is in a non-shared Shell History file then you can reasonably assume that the command was issued by that account (but not neccesarily by the owner of that account).
You cannot assume that the command did what you expect based on one line. You can create a Shell Script in your account with the same name as system command - but a completely different function (as those who create programs called "test" soon find out).
If the command is in nobody's Shell History file then you can make no assumptions.
What you can never do is use a Shell History file alone to accuse an individual of typing something.
Yep, under normal circumstances where users log in and do all sorts of funny stuff I'd agree with him. I probably should've elaborated in my original post a bit.
In this case, the account is used exclusively by an automated system, always using the same OS account and always running straight shell commands every so often. All the commands the automated system was configured to run show up in the history, and as often as we would expect it to show up.
My problem with this is a) how did they establish that this account ran the command they're talking about repeatedly? b) if all the other commands show up, why not the one they claim was run?
I accept that it's not as clear cut as every single command an OS account ever runs getting logged in the history files, but in this case I just don't think the sysadmin is on the right track. He's essentially resorted to shifting all blame on to the automated system.
Were it up to me, I'd want the following done:
Create a completely new, secure, dedicated OS account for use by the automated system only, as I seriously suspect that the OS account they're referring to might just be compromised.
Configure some serious auditing on this new account. Log who logged in from where and at what time. More to the point, log whenever something other than the automated system logs in to this account.
Have the sysadmins simply remove any privileges to commands they're not comfortable with users having access to.
Have the automated system connect to a test system first and have the sysadmins monitor the server for a specified amount of time to make sure nothing happens that they're not happy with.
If everyone's still happy at this point, carry out the same steps on the proper system.
Anyways, thanks again for all the help guys. And wish me luck!
This is another case...
In history files, you would expect to see all commands typed by the user if in a shellscript ther are many programs-commands called, since the user did not type them on the keyboard, very little chance will you have to see them in $HISTFILE...
Anything cron or other scheduler have their own log facilities that need to be configured adequatly to suit your request, in other words HISTFILE is set for a interactive shell only( logging the STDIN activity...)... This explains why you see nothing...
Which brings me to the other thing I neglected to mention (sorry, as it turns out I left out quite a lot of detail originally).
The automated system uses only Java libraries or plink.exe to connect from a Windows client machine. I know for a fact both the commands configured to run over the Java libs and plink.exe show up fine in the history.
EDIT: Scratch that, seems commands run over plink.exe do not show up... The plot thickens.
---------- Post updated 12-15-11 at 08:51 AM ---------- Previous update was 12-14-11 at 01:19 PM ----------
Quick update regarding this:
Checked, double-checked and checked again that any commands using Java libs show up in the history files. I'm not sure if every single command will show up, but I fully expect at least one instance of a command to show up at least once as the commands are all run every 10 minutes or so.
The commands using plink.exe are a bit trickier as they do not show up for some reason. I'm not sure how I'd go about running plink in such a way to make the O/S treat the session as a normal user SSH session (thereby dumping the commands ni the history files as well), but regardless, I'm reasonably sure the commands we run using plink are fine.
The only times we would use plink is if we connect to the HP-UX O/S by means of custom Python scripts and, since the bulk of these scripts were written by me and reside in a central location, I'm sure the command they claim was run would not have been kicked off by this.
At this point I must add that it turns out there's not just one automated system that connects to this server. It also turns out that the O/S account isn't exclusively used only by this one automated system...
I'll update again if anything worthwhile pops up. Thanks again for the help so far with this guys.
