I was attempting to run a backup at our business last Friday evening when I apparently locked up our whole mainframe. Rather than observing the current time and adding a minute or two to that for the backup start time (like I typically would), I accidentally entered the actual current time to the minute. For some reason that seemed to be a problem as the screen just hung there for quite a while. I 'killed' that login and pulled the tape out. Now when I try to log back in it says 'Backup starting' 'You must wait until process has terminated'. I figure there has to be some way to end this process so we can get back to business but I have no idea how. I'm very much a novice as far as this OS is concerned. I also had to replace the backup tape drive in the server recently and I'm seeing a lot of the message 'portmapper is not responding'. I'm actually considering disconnecting the tape drive from the server since I have no idea what to try on the software end of things. Any advice would be greatly appreciated!
Hello hometrics,
How do you start the backup?
If it is a shell script, what output do you get from running the appropriate shell with the trace flag set to call it is, e.g. if it's ksh you could try ksh -x /my/backup/script and see if it is looking at a lock file or some sort of a marker to stop you trying to run two together.
Can you share the code? (wrapped in CODE tags)
What OS, shell and 3rd party software (if there is any) are you using?
I'm sure we can help you out of this, but we need more details to work it out.
kind regards,
Robin
You probably need to look at the running processes on the system using:
# ps -eaf
and see if you can spot the backup process. One of the columns will be 'PID' which gives the process identifier.
If you can identify it then you can ask it politely to stop:
# kill -15 <PID>
and if it won't then nuke it with:
# kill -9 <PID>
If you need to post the 'ps' listing here.
Thank you both very much for your replies. I'll answer your questions to the best of my abilities despite my ignorance to this system.
Re: rbatte1, the OS is SCO OpenServer Release 5 and we are running FASPAC software (apparently a very popular program for wholesalers in the nuts and bolts industry). I don't start the backups with a shell script. Typically, when I can log in successfully and I'm in my FASPAC program, I can just go to 'Utilities' and then 'Backups' from the main menu. If there is any shell or command prompt on here I am unaware of how to access it. I just figured there had to be something I could do to end the process outside of the FASPAC software as I can't get that far anymore.
Re: hicksd8, your reply has me feeling encouraged. Unfortunately, I'm not even sure where I would enter this text.
It might not help much but I'll attach a shot of what I currently have on my screen. It does mention the command prompt but this is something I've never really had to use. Thanks for any continued support here. I really do appreciate your time.
Do you have the 'root' password for this box?
If you enter ALT-F2 (also try ALT-2) do you get a login prompt if you press return a couple of times?
Are you able to get to the command line. e.g by interrupting the menu (e.g. <Ctrl>C) , or by logging in as a different user? Is your sysadmin able to jump in?
Have you done a successful backup since you changed the tape drive? It may be that the new drive is not set to the same SCSI address as the old one.
You should be able to press <alt><F3> through <alt><f12> to get a new login session.
<alt><f2> will normally switch you to a graphical session that will require <ctrl><alt><f1> to exit from.
Without knowing the script that the backup option on the menu starts, it is difficult to say why it has hung.
Since it is highly unlikely that any programs other than the backup are running it is probably safe to switch to another session and restart the system as root with
init 6
You should be able to confirm that the tape drive is correctly configured using:
tape status
The default scsi address for the tape is 2.
Re: jgt, it seems like the backups have been completing fine since the installation of the new drive with the exception of this last one. I do have a password for root. I tried logging in as root and using the 'init 6' command. When I entered the 'tape status' command it looked like everything was okay. Beyond that, we do use <alt><f1> and <alt><f2> often to toggle back and forth between two sessions as we run two companies from the same building here. But I've tried logging in from another session, I've tried rebooting and shutting the system down. Nothing seems to end this process.
Re: hicksd8, I used the 'ps -eaf' command that you listed. The only thing that jumped out at me was one line where the value was 4 (it's 0 for everything else). So is 789 the PID (see attached) and I would enter that in the next command in place of the 'PID'?
It's not possible for me to see whether PID 789 is the correct one because you are not showing the right-hand side of the screen.
Does any process have 'tty01' showing against it? Your first screenshot seems to show this process is communicating with the console terminal.
If you have really rebooted then the process must end then. Nothing is running when the system is down. Perhaps it is restarting at boot time.
Do you have a manual for this backup software?
If you have re-booted the system, the process is no longer running. What is most likely is that there is a lock file that the backup procedure creates in order for no other processes to run until the backup completes.
You have to find this file and delete it.
Step one is to determine the script that the menu initiates to start the backup. With luck the menu is a script and not a compiled program.
Are you prevented from running the both companies?
Just wanted to post an update as I was able to get this resolved over the phone with tech support. What ultimately did the trick was the command prompt #su - <mynormallogin> which, I'm told, was a super user workaround which enabled me to login normally and get to the main menu for my FASPAC program. Once there I was able to run a new backup and that apparently cleared the file that was hanging up the system.
Thanks very much to everyone for your time and help and especially to jgt for offering to help onsite.
Thanks for sharing this!
Here is some information.
In
ps -eaf
the fourth column is "CPU usage percent".
The "portmapper is not responding" is a system console message. Such a message occurs asynchronously, is not related with your current action. But can be disturbing, so usually you better go to another login screen that is not a system console.
In this case some process on the system (usually an access attempt to NFS, network file system) wants a process "portmap" or "rpcbind" that is not running. You can check that with the "ps -eaf" command, and check its functionality with the "rpcinfo" command.)
But probably this is not important at all, and certainly not relevant for your problem.
Your problem was that you immediately started the backup and that freezes your session. The best thing you can do is wait until the backup is finished and your session continues. Watch the activity light on the tape, and do not interrupt/force-remove the tape!
I guess the backup program puts a /etc/nologin file with a message. Then a user cannot login, and the message is displayed instead. At the end the backup program deletes the /etc/nologin.
If interrupted the /etc/nologin remains.
A # su - userlogin is a good trick - it ignores the /etc/nologin.
Also a # rm /etc/login would have worked.
See this web page.