C4 scsci Tape library not detectable after lustarter patching on solaris 10 u4

sahil_shine · September 19, 2012, 5:33am

Hi All,

We have patched our solaris 10 u4 server with lustarter patch in order to get it prepared for solaris live upgrade.The patch installation went pretty fine and we did reboot the server after patching.But after patching we have noticed O.S is not able to detect tape library.Not sure what to do next.Any ideas ? :wall:

Regards,
Sahil

---------- Post updated at 01:32 AM ---------- Previous update was at 01:16 AM ----------

This is what i see with below commands

root@xyz # devfsadm -C -c tape -v
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/sg@w202200a0b86750c9,1:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/sg@w202200a0b86750c9,2:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/sg@w202200a0b86750c9,3:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/sg@w202200a0b86750c9,4:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0/sg@w202200a0b86750c9,5:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@31,700000/SUNW,qlc@0/fp@0,0/sg@w202300a0b86750c9,1:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@31,700000/SUNW,qlc@0/fp@0,0/sg@w202300a0b86750c9,2:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@31,700000/SUNW,qlc@0/fp@0,0/sg@w202300a0b86750c9,3:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@31,700000/SUNW,qlc@0/fp@0,0/sg@w202300a0b86750c9,4:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@31,700000/SUNW,qlc@0/fp@0,0/sg@w202300a0b86750c9,5:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@33,700000/SUNW,emlxs@0/fp@0,0/sg@w202400a0b8387c56,0:raw
devfsadm[23147]: verbose: no devfs node or mismatched dev_t for /devices/pci@33,700000/SUNW,emlxs@0,1/fp@0,0/sg@w203500a0b8387c56,0:raw

---------- Post updated at 01:33 AM ---------- Previous update was at 01:32 AM ----------

root@xyz # cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
SB0                                         disconnected unconfigured unknown
SB1                                         disconnected unconfigured unknown
SB2                            System_Brd   connected    configured   ok
SB2::cpu1                      cpu          connected    configured   ok
SB2::memory                    memory       connected    configured   ok
SB2::pci2                      io           connected    configured   ok
SB2::pci3                      io           connected    configured   ok
SB2::pci8                      io           connected    configured   ok
SB3                            System_Brd   connected    configured   ok
SB3::cpu0                      cpu          connected    configured   ok
SB3::cpu1                      cpu          connected    configured   ok
SB3::memory                    memory       connected    configured   ok
SB3::pci0                      io           connected    configured   ok
SB3::pci1                      io           connected    configured   ok
SB3::pci2                      io           connected    configured   ok
SB3::pci3                      io           connected    configured   ok
SB3::pci8                      io           connected    configured   ok
SB4                                         disconnected unconfigured unknown
SB5                                         disconnected unconfigured unknown
SB6                                         disconnected unconfigured unknown
SB7                                         disconnected unconfigured unknown
SB8                                         disconnected unconfigured unknown
SB9                                         disconnected unconfigured unknown
SB10                                        disconnected unconfigured unknown
SB11                                        disconnected unconfigured unknown
SB12                                        disconnected unconfigured unknown
SB13                                        disconnected unconfigured unknown
SB14                                        disconnected unconfigured unknown
SB15                                        disconnected unconfigured unknown
c0                             scsi-bus     connected    configured   unknown
c0::dsk/c0t0d0                 disk         connected    configured   unknown
c0::dsk/c0t1d0                 disk         connected    configured   unknown
c3                             scsi-bus     connected    configured   unknown
c3::ob0                        med-changer  connected    configured   unknown
c3::ob1                        tape         connected    configured   unknown
c3::ob2                        tape         connected    configured   unknown
c4                             scsi-bus     connected    unconfigured unknown
c6                             fc-private   connected    configured   unknown
c6::202200a0b86750c9           disk         connected    configured   unknown
c7                             fc-private   connected    configured   unknown
c7::202300a0b86750c9           disk         connected    configured   unknown
c8                             fc-private   connected    configured   unknown
c8::202400a0b8387c56           disk         connected    configured   unknown
c9                             fc-private   connected    configured   unknown
c9::203500a0b8387c56           disk         connected    configured   unknown
iou#0-pci#3                    etherne/hp   connected    configured   ok
iou#0-pci#4                    unknown      empty        unconfigured unknown
iou#1-pci#0                    scsi/hp      connected    configured   ok
iou#1-pci#1                    fibre/hp     connected    configured   ok
iou#1-pci#2                    fibre/hp     connected    configured   ok
iou#1-pci#3                    unknown      empty        unconfigured unknown
iou#1-pci#4                    fibre/hp     connected    configured   ok

hicksd8 · September 19, 2012, 7:47am

What hardware platform are we talking about exactly?

sahil_shine · September 19, 2012, 8:56am

Sun M5000

hicksd8 · September 19, 2012, 9:49am

Hmmmm! M5000 ??

More questions I'm afraid.....

What version of firmware is on the box? (I'm thinking many patch operations can fall over due to out-of-date firmware.)

What is the QLogic card exactly? (I'm thinking do you need patch 125166-10 to make it work?)

What is the Emulex card exactly? (I'm thinking to you need patch 120222-26 to make it work?)

Which card is the library connected to anyway?

I'm not convinced we know enough to pass judgement yet.

sahil_shine · September 19, 2012, 10:02am

Hi,

The firmware on m5000 box is latest one.We had upgraded it last month.The C4 tape is connected via QLogic card.

hicksd8 · September 19, 2012, 10:04am

Is patch 125166-10 on the system?

sahil_shine · September 19, 2012, 10:07am

Thanks for your quick reply.I had a quick look.It's not present.But tape library was detectable before lustarter patch being applied.

hicksd8 · September 19, 2012, 10:11am

I know what you're saying but any type of patching can give unexpected results (more so these days so it seems, especially 10u6 thru' 10u9) where firmware levels are involved.

And...the note about 125166-10 being required for some QL cards to work appears in the firmware release notes (strange though that seems).

If it was me I'd give that patch a go and post the results.

sahil_shine · September 19, 2012, 10:18am

I just had a look seems 125166-10 patch is older one an and newer version available for qlogic is 146489-06 .Is this looks ok?

hicksd8 · September 19, 2012, 10:21am

I reckon you're right.

You seem to have an Emulex card in there too. What's that used for? Do you need a patch for that one?

Are you saying 146489-06 is already on the system?

sahil_shine · September 19, 2012, 10:35am

I think the qlogic card is used for fc storage and emulex card is used for connecting scsi tape drive.I guess then i need patch for emulex card ...The patch 146489-06 is not yet installed on my system ..i need to install that then i think.

hicksd8 · September 19, 2012, 10:46am

Yes, I think so.

I'd even try the original patch(es) if the latest one's don't solve the problem. Uninstall one patch and install the other one.

If you've got the latest firmware then I can't see it being anything other than patches for add-on cards.

Perhaps other forum members will think differently.

sahil_shine · September 19, 2012, 12:14pm

I had a relook seems my c4 tape library is not using either the qlogic/ emulex card.But its using a scsi hba card .

hicksd8 · September 19, 2012, 1:26pm

Do you mean that it's connected to an onboard (M5000 built-in/motherboard) controller?

So the listing in your post#1 showing qlc and emlx not being recognised is of no relevance to the C4 not working?

I'm not too familiar with the M5000 but can you do a "probe-scsi-all" command from the service processor ("ok" prompt)?

If so, what does that say? Should see the relevant controller and all scsi devices ie, the jukebox.

sahil_shine · September 20, 2012, 7:40am

I had also logged a case with sun with regard to this, seems the devices are present under O.S but this were not shown under cfgadm -al,iostat -En or under /dev/rmt/ * output but when i give cfgadm -alv, it does shows the devices.I have done previously a reconfiguration reboot but Sun has advised to do a probe-scisi-all from O.K prompt.I will give it a try today and let's see

gull04 · September 20, 2012, 8:45am

Hi Sahil,

Did you apply the latest Solaris 10 Recommended before the lu starter patches?

Regards

Dave

hicksd8 · September 20, 2012, 12:08pm

When you probe-scsi-all, if the library isn't listed try very hard to get it listed by resetting the library, power cycling the library, wiggle the scsi cables, replug the cables, et al. While the system is down (ok prompt) gives you the opportunity to run probe-scsi-all repeatedly to see what's there.

To your credit you assumed that the problem was caused by lustarter patching (which is the correct approach for an IT pro "last thing I did caused the problem"). It is however possible that the reboot caused the issue to appear not the lustarter patch. I would begin to think outside the box at a coincidence occuring.

Repeat, try very hard to get probe-scsi-all to see the library whilst the system is down. Might also power cycle the M5000 if you can't get a result.

sahil_shine · September 21, 2012, 4:44am

Still no luck.Below is the output from ok prompt.I see tape library here but sill nothing under /dev/rmt/*.

{68} ok reset-all
Resetting...
POST Sequence 01 CPU Check
POST Sequence 02 Banner
LSB#02 (XSB#00-1): POST 2.17.0 (2011/11/17 10:29)
LSB#03 (XSB#01-0): POST 2.17.0 (2011/11/17 10:29)
POST Sequence 03 Fatal Check
POST Sequence 04 CPU Register
POST Sequence 05 STICK
POST Sequence 06 MMU
POST Sequence 07 Memory Initialize
POST Sequence 08 Memory
POST Sequence 09 Raw UE In Cache
POST Sequence 0A Floating Point Unit
POST Sequence 0B SC
POST Sequence 0C Cacheable Instruction
POST Sequence 0D Softint
POST Sequence 0E CPU Cross Call
POST Sequence 0F CMU-CH
POST Sequence 10 PCI-CH
POST Sequence 11 Master Device
POST Sequence 12 DSCP
POST Sequence 13 SC Check Before STICK Diag
POST Sequence 14 STICK Stop
POST Sequence 15 STICK Start
POST Sequence 16 Error CPU Check
POST Sequence 17 System Configuration
POST Sequence 18 System Status Check
POST Sequence 19 System Status Check After Sync
POST Sequence 1A OpenBoot Start...
POST Sequence Complete.

SPARC Enterprise M5000 Server, using Domain console
Copyright (c) 1998, 2011, Oracle and/or its affiliates. All rights reserved.
Copyright (c) 2011, Oracle and/or its affiliates and Fujitsu Limited. All rights reserved.
OpenBoot 4.33.5.a, 49152 MB memory installed, Serial #70974532.
Ethernet address 0:14:4f:3a:fc:44, Host ID: 843afc44.



{48} ok probe-scsi-all
/pci@33,700000/SUNW,emlxs@0,1
Device  ALPA e2  WWPN 203500a0b8387c56
   LUN  0     Disk     SUN     LCSM100_F       0670
   LUN  1f     Disk     SUN     Universal Xport 0670

/pci@33,700000/SUNW,emlxs@0
Device  ALPA ef  WWPN 202400a0b8387c56
   LUN  0     Disk     SUN     LCSM100_F       0670
   LUN  1f     Disk     SUN     Universal Xport 0670

/pci@31,700000/SUNW,qlc@0
QLogic QLE2460 Host Adapter Driver(SPARC): 1.24  11/15/06
Firmware version 4.00.26
Adapter Loop-ID - 0
**************** Local Loop Devices *****************
  Dev# 0   Loop-ID   2    Port WWN 202300a0b86750c9
   LUN 1    DISK  SUN     LCSM100_F       0735
   LUN 2    DISK  SUN     LCSM100_F       0735
   LUN 3    DISK  SUN     LCSM100_F       0735
   LUN 4    DISK  SUN     LCSM100_F       0735
   LUN 5    DISK  SUN     LCSM100_F       0735
   LUN 1f    DISK  SUN     Universal Xport 0735


/pci@30,600000/pci@0/pci@9/SUNW,qlc@0
QLogic QLE2460 Host Adapter Driver(SPARC): 1.24  11/15/06
Firmware version 4.00.26
Adapter Loop-ID - 1
**************** Local Loop Devices *****************
  Dev# 0   Loop-ID   0    Port WWN 202200a0b86750c9
   LUN 1    DISK  SUN     LCSM100_F       0735
   LUN 2    DISK  SUN     LCSM100_F       0735
   LUN 3    DISK  SUN     LCSM100_F       0735
   LUN 4    DISK  SUN     LCSM100_F       0735
   LUN 5    DISK  SUN     LCSM100_F       0735
   LUN 1f    DISK  SUN     Universal Xport 0735


/pci@30,600000/pci@0/pci@8/pci@0,1/scsi@1,1

/pci@30,600000/pci@0/pci@8/pci@0,1/scsi@1
Target 0
  Unit 0   Removable Device type 8     QUANTUM PX500           0033
Target 1
  Unit 0   Removable Tape     HP      Ultrium 3-SCSI  G56Z
Target 2
  Unit 0   Removable Tape     HP      Ultrium 3-SCSI  G56Z

/pci@30,600000/pci@0/pci@8/pci@0/scsi@1

MPT Version 1.05, Firmware Version 1.11.00.00

Target 0
Unit 0   Disk     SEAGATE ST973402SSUN72G 0603    143374738 Blocks, 73 GB
  SASAddress 5000c50007e0ba81  PhyNum 0
Target 1
Unit 0   Disk     SEAGATE ST973402SSUN72G 0400    143374738 Blocks, 73 GB
  SASAddress 5000c50007750c6d  PhyNum 1

Sun also asked me below steps but no luck :wall:


login as: root
Using keyboard-interactive authentication.
Password:
Last login: Fri Sep 21 12:26:15 2012 from wl2006223.ap.ti
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
You have mail.
Sourcing //.profile-EIS.....
root@xyz # add_drv st
("st") already in use as a driver or alias.
root@xyz # modload /kernel/drv/st
root@xyz # devfsadm -Cv -i st
devfsadm: driver failed to attach: st
exit status = 1
root@xyz # cfgadm -alv -o show_FCP_dev
Ap_Id                          Receptacle   Occupant     Condition  Information
When         Type         Busy     Phys_Id
c6                             connected    configured   unknown
unavailable  fc-private   n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc
c6::202200a0b86750c9,0         connected    configured   failing
unavailable  unavailable  n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,0
c6::202200a0b86750c9,1         connected    configured   unknown
unavailable  disk         n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,1
c6::202200a0b86750c9,2         connected    configured   unknown
unavailable  disk         n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,2
c6::202200a0b86750c9,3         connected    configured   unknown
unavailable  disk         n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,3
c6::202200a0b86750c9,4         connected    configured   unknown
unavailable  disk         n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,4
c6::202200a0b86750c9,5         connected    configured   unknown
unavailable  disk         n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,5
c6::202200a0b86750c9,31        connected    configured   unknown    SUN Universal Xport
unavailable  disk         n        /devices/pci@30,600000/pci@0/pci@9/SUNW,qlc@0/fp@0,0:fc::202200a0b86750c9,31
c7                             connected    configured   unknown
unavailable  fc-private   n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc
c7::202300a0b86750c9,0         connected    configured   failing
unavailable  unavailable  n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,0
c7::202300a0b86750c9,1         connected    configured   unknown
unavailable  disk         n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,1
c7::202300a0b86750c9,2         connected    configured   unknown
unavailable  disk         n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,2
c7::202300a0b86750c9,3         connected    configured   unknown
unavailable  disk         n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,3
c7::202300a0b86750c9,4         connected    configured   unknown
unavailable  disk         n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,4
c7::202300a0b86750c9,5         connected    configured   unknown
unavailable  disk         n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,5
c7::202300a0b86750c9,31        connected    configured   unknown    SUN Universal Xport
unavailable  disk         n        /devices/pci@31,700000/SUNW,qlc@0/fp@0,0:fc::202300a0b86750c9,31
c8                             connected    configured   unknown
unavailable  fc-private   n        /devices/pci@33,700000/SUNW,emlxs@0/fp@0,0:fc
c8::202400a0b8387c56,0         connected    configured   unknown
unavailable  disk         n        /devices/pci@33,700000/SUNW,emlxs@0/fp@0,0:fc::202400a0b8387c56,0
c8::202400a0b8387c56,31        connected    configured   unknown    SUN Universal Xport
unavailable  disk         n        /devices/pci@33,700000/SUNW,emlxs@0/fp@0,0:fc::202400a0b8387c56,31
c9                             connected    configured   unknown
unavailable  fc-private   n        /devices/pci@33,700000/SUNW,emlxs@0,1/fp@0,0:fc
c9::203500a0b8387c56,0         connected    configured   unknown
unavailable  disk         n        /devices/pci@33,700000/SUNW,emlxs@0,1/fp@0,0:fc::203500a0b8387c56,0
c9::203500a0b8387c56,31        connected    configured   unknown    SUN Universal Xport
unavailable  disk         n        /devices/pci@33,700000/SUNW,emlxs@0,1/fp@0,0:fc::203500a0b8387c56,31
root@xyz # ls -l /dev/rmt/*
/dev/rmt/*: No such file or directory
root@xyz #

---------- Post updated at 12:44 AM ---------- Previous update was at 12:43 AM ----------

No i haven't applied latest recommended patches yet.

gull04 · September 21, 2012, 7:28am

Hi Sahil,

The general recommendation from Oracle is that the latest recommended are applied first and then the LU, also I'd be tempted to check that all the firmware levels are compatible.

Including the SCSI adapter and the Tape Array, don't forget the HBA's and the ILOM. Generally the vendors will have a support matrix some where and you'll have to put every thing together your self.

Regards

Dave

jim_mcnamara · September 21, 2012, 12:03pm

More directly:
@sahil_shine - showing us errors and confugrations does not answer basic questions:

what is your patch level? Are you current?

We have to rule out possible problems before we can give you a good answer.