Scan and create file systems in new LUNs

fretagi · September 29, 2015, 5:52am

Hi

I have a system running

 uname -a
Linux cmoveldb02 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

and I was told by the NetApp Admin that he has provided 5 LUNs of 1Tb , 200Gb and remaining 3 of 300Gb , but I am not able to see the new LUNs.
I have type the following:

root@cmoveldb02:~# ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sda2  /dev/sda5

them rebooted the system, after run the same command ls , which provided me with same results as above.

them checking on google, I found the following steps:

root@cmoveldb02:/var/log# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | wc -l
6
root@cmoveldb02:/var/log#

them:

root@cmoveldb02:/var/log# ls /sys/class/fc_host
host11  host9
root@cmoveldb02:/var/log#

them:

root@cmoveldb02:/var/log# echo "1" > /sys/class/fc_host/host9/issue_lip
root@cmoveldb02:/var/log# echo "1" > /sys/class/fc_host/host11/issue_lip
root@cmoveldb02:/var/log#

Nothing appears to happen, them:

root@cmoveldb02:/var/log# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | wc -l
6
root@cmoveldb02:/var/log#

so it still showing the same disks.....

Please can you help?

Peasant · September 29, 2015, 7:56am

Have you checked the messages log after issuing lip ?
Messages there should give you some clue what is going on.

There is no need for it btw, this is how you rescan, but lip will also work (this is mostly used when new storage is added to machine)

echo "- - -" > /sys/class/fc_host/hostN/scan

Where N is the number you know from previous output.

Are you using multipath ?
You should not use those devices directly (/dev/sd....) but their multipath name (/dev/mapper/...) or else you risk if one card dies multipath will not work.

Check out this document :

fretagi · September 29, 2015, 8:08am

the following command you gave gave :

root@cmoveldb02:/# echo "- - -" > /sys/class/fc_host/host9/scan
-bash: /sys/class/fc_host/host9/scan: No such file or directory
root@cmoveldb02:/#

I do not have the /var/adm/messages file but I do have

root@cmoveldb02:/var/log# tail syslog
Sep 29 12:23:13 cmoveldb02 multipathd: dm-1: add map (uevent)
Sep 29 12:23:13 cmoveldb02 kernel: [595528.836459] device-mapper: table: 252:2: multipath: error getting device
Sep 29 12:23:13 cmoveldb02 kernel: [595528.837159] device-mapper: ioctl: error adding target to table
Sep 29 12:23:13 cmoveldb02 kernel: [595528.837862] device-mapper: table: 252:2: multipath: error getting device
Sep 29 12:23:13 cmoveldb02 kernel: [595528.838560] device-mapper: ioctl: error adding target to table
Sep 29 12:23:13 cmoveldb02 multipathd: dm-0: add map (uevent)
Sep 29 12:23:13 cmoveldb02 udevd[5417]: inotify_add_watch(6, /dev/dm-2, 10) failed: No such file or directory
Sep 29 12:23:13  udevd[5417]: last message repeated 2 times
Sep 29 12:23:13 cmoveldb02 udevd[2486]: inotify_add_watch(6, /dev/dm-2, 10) failed: No such file or directory
Sep 29 13:17:01 cmoveldb02 CRON[5674]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
root@cmoveldb02:/var/log#

But I am not sure how to interpret the lines were multipath is mentioned

Peasant · September 29, 2015, 11:41am

Check out the output of

dmsetup info

Locate the disk with 252:2 major/minor number.
Check if that disk is local disk and blacklist that disk so multipath is not active.

Refer to :

Section : Ignoring Local Disks When Generating Multipath Devices

Restart multipath daemon.

Post output of :

multipath -v3
ls -lrt /dev/mapper/*

Rescan the fabric. You have some cards inside from manufacturer X.
Check out the documentation regarding scanning from Linux host, i cannot say for sure.
Most makers provide some sort of utilities to rescan (scripts etc.).. you will have to figure that one out

---------- Post updated at 17:41 ---------- Previous update was at 17:37 ----------

Sorry i cannot edit posts (issue with browser ), it will just double post.

Read the documentation on provided links.
You will need to configure the multipath.conf, instructions are there (Configuring storage devices).

Hope the helps
Regards
Peasant.

fretagi · September 30, 2015, 3:03am

please see the outputs of the commands:

root@cmoveldb02:~# dmsetup info
Name:              cmovel--db02-swap_1
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        2
Event number:      0
Major, minor:      252, 1
Number of targets: 1
UUID: LVM-yrAv2YJL6WHciOiYQFvZgru1Jtl6vM2pmKypDknOQh3HwDfaZddnG6uJgB7ULMmh

Name:              cmovel--db02-root
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        1
Event number:      0
Major, minor:      252, 0
Number of targets: 1
UUID: LVM-yrAv2YJL6WHciOiYQFvZgru1Jtl6vM2p2MaakDmReoiXqrjs4EjaasdyB0TuwAjd

root@cmoveldb02:~#

and

root@cmoveldb02:~# multipath -v3
Sep 30 09:02:53 | ram0: device node name blacklisted
Sep 30 09:02:53 | ram1: device node name blacklisted
Sep 30 09:02:53 | ram2: device node name blacklisted
Sep 30 09:02:53 | ram3: device node name blacklisted
Sep 30 09:02:53 | ram4: device node name blacklisted
Sep 30 09:02:53 | ram5: device node name blacklisted
Sep 30 09:02:53 | ram6: device node name blacklisted
Sep 30 09:02:53 | ram7: device node name blacklisted
Sep 30 09:02:53 | ram8: device node name blacklisted
Sep 30 09:02:53 | ram9: device node name blacklisted
Sep 30 09:02:53 | ram10: device node name blacklisted
Sep 30 09:02:53 | ram11: device node name blacklisted
Sep 30 09:02:53 | ram12: device node name blacklisted
Sep 30 09:02:53 | ram13: device node name blacklisted
Sep 30 09:02:53 | ram14: device node name blacklisted
Sep 30 09:02:53 | ram15: device node name blacklisted
Sep 30 09:02:53 | loop0: device node name blacklisted
Sep 30 09:02:53 | loop1: device node name blacklisted
Sep 30 09:02:53 | loop2: device node name blacklisted
Sep 30 09:02:53 | loop3: device node name blacklisted
Sep 30 09:02:53 | loop4: device node name blacklisted
Sep 30 09:02:53 | loop5: device node name blacklisted
Sep 30 09:02:53 | loop6: device node name blacklisted
Sep 30 09:02:53 | loop7: device node name blacklisted
Sep 30 09:02:53 | sda: not found in pathvec
Sep 30 09:02:53 | sda: mask = 0x1f
Sep 30 09:02:53 | sda: dev_t = 8:0
Sep 30 09:02:53 | sda: size = 859525120
Sep 30 09:02:53 | sda: subsystem = scsi
Sep 30 09:02:53 | sda: vendor = Sun
Sep 30 09:02:53 | sda: product = cmoveldb02
Sep 30 09:02:53 | sda: rev = V1.0
Sep 30 09:02:53 | sda: h:b:t:l = 8:0:0:0
Sep 30 09:02:53 | sda: serial = 953FCC2D
Sep 30 09:02:53 | sda: get_state
Sep 30 09:02:53 | sda: path checker = directio (internal default)
Sep 30 09:02:53 | sda: state = running
Sep 30 09:02:53 | directio: starting new request
Sep 30 09:02:53 | directio: io finished 4096/0
Sep 30 09:02:53 | sda: state = 3
Sep 30 09:02:53 | sda: prio = const (internal default)
Sep 30 09:02:53 | sda: prio =  (internal default)
Sep 30 09:02:53 | sda: const prio = 1
Sep 30 09:02:53 | sda: getuid = /lib/udev/scsi_id --whitelisted --device=/dev/%n (internal default)
Sep 30 09:02:53 | sda: uid = SSun     cmoveldb02     953FCC2D (callout)
Sep 30 09:02:53 | dm-0: device node name blacklisted
Sep 30 09:02:53 | dm-1: device node name blacklisted
Sep 30 09:02:53 | sr0: device node name blacklisted
===== paths list =====
uuid                             hcil    dev dev_t pri dm_st chk_st vend/prod/
SSun     cmoveldb02     953FCC2D 8:0:0:0 sda 8:0   1   undef ready  Sun,cmovel
Sep 30 09:02:53 | sda: ownership set to SSun     cmoveldb02     953FCC2D
Sep 30 09:02:53 | sda: not found in pathvec
Sep 30 09:02:53 | sda: mask = 0xc
Sep 30 09:02:53 | sda: get_state
Sep 30 09:02:53 | sda: state = running
Sep 30 09:02:53 | directio: starting new request
Sep 30 09:02:53 | directio: io finished 4096/0
Sep 30 09:02:53 | sda: state = 3
Sep 30 09:02:53 | sda: const prio = 1
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: pgfailover = -1 (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: pgpolicy = failover (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: selector = round-robin 0 (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: features = 0 (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: hwhandler = 0 (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: rr_weight = 1 (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: minio = 1000 (config file default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: no_path_retry = NONE (internal default)
Sep 30 09:02:53 | pg_timeout = NONE (internal default)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: set ACT_CREATE (map does not exist)
Sep 30 09:02:53 | SSun     cmoveldb02     953FCC2D: domap (0) failure for create/reload map
root@cmoveldb02:~#

and

root@cmoveldb02:~# ls -lrt /dev/mapper/*
crw------- 1 root root 10, 236 Sep 22 14:57 /dev/mapper/control
lrwxrwxrwx 1 root root       7 Sep 29 12:23 /dev/mapper/cmovel--db02-swap_1 -> ../dm-1
lrwxrwxrwx 1 root root       7 Sep 29 12:23 /dev/mapper/cmovel--db02-root -> ../dm-0
root@cmoveldb02:~#