No space left on device but free space and inodes are available...

UNIX Solaris
1

hi guys, me again :wink:

i recently opened a thread about physical to zone migration.

My zone is mounted over a "bigger" LUN (500GB) and step is now to move the old files, from the physical server, to my zone.
We are talking about 22mio of files.

i used rsync to do that and every time at 93.9% of rsync the task goes in error telling me "No space left on device"

here the infos i have collected...

df -h
/storedge              443G   427G    16G    97%    /storedge

df -o i /storedge
Filesystem     iused    ifree       %iused  mounted on
/storedge           22124418 33791806    40%   /storedge

and every time when this occures i cannot reboot my zones anymore... :mad:

[root@sunglb02ps] / => zoneadm -z archiprod boot
zoneadm: zone 'archiprod': "/usr/lib/fs/ufs/mount /dev/dsk/c4t6d7s4 /zones/archiprod/root/logs" failed with exit code 32
zoneadm: zone 'archiprod': call to zoneadmd failed

have somebody a suggestion for me?
thx

2

Have you done a zfs list to see if it says there is any free space?

There may be metadata overhead which may not be listed in the df -h command.

3

we don't use zfs at the moment and my affected zone is solaris9 branded! :slight_smile:

4

UFS?

man tunefs
5

the error here shows that the zone is in /zones, where does /storedge in this matter?

i don't know if i get it right, you have a physical server running Solaris 9 that you want to migrate to a branded zone on host sunglb02ps?

also, it might be that the rsync process has messed up the zone configuration. can you post your zone configuration and the list of the zones on the host? and if you may, also, post how you run your rsync command?

6

/storedge on device /dev/dsk/c4t6d7s4 where is additionaly mounted into the zone:

[root@sunglb02ps] / => zonecfg -z archiprod info
zonename: archiprod
zonepath: /zones/archiprod
brand: solaris9
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: shared
hostid:
fs:
        dir: /logs
        special: /dev/dsk/c4t6d7s4
        raw: /dev/rdsk/c4t6d7s4
        type: ufs
        options: []
fs:
        dir: /storedge
        special: /dev/dsk/c4t6d7s3
        raw: /dev/rdsk/c4t6d7s3
        type: ufs
        options: []
net:
        address: x.x.x.x
        physical: bge0
        defrouter: x.x.x.x
capped-memory:
        physical: 2G
7

You should monitor the free space and inodes during the file transfer by regularly running both of the df commands. That will tell which ones of disk space or inodes is undersized.

8

yes but at the time the rsync stoped the free space was at 16GB and the inodes was used at 40%!

i was able to create some files in it. (with touch file1 file2 file3) but a "mkfile 10m test" tell me that there was no free space on that device?!

9

based on your zoneconfig, /storedge is on c4t6d7s3 and /log is the one on c4t6d7s4

fs:
        dir: /logs
        special: /dev/dsk/c4t6d7s4
        raw: /dev/rdsk/c4t6d7s4
        type: ufs
        options: []
fs:
        dir: /storedge
        special: /dev/dsk/c4t6d7s3
        raw: /dev/rdsk/c4t6d7s3
        type: ufs
        options: []

this error might be because you have mounted the slice during the sync and when the zone booted, it can't mount the filesystem on the zone because its busy. if you encounter this error again, try mounting the slice to a different mount point on the global zone to check if you will be able to mount it.