LTO5 Data cartridge has 1.5 TB (1500GB) native capacity but when we are taking our 1.44 TB (1475 GB) filesystem backup using backupby filename on these data cartridges it does not fully finish on one cartridge instead it requires another volume to backup the remaining files. I am unable to find out why this happens as we have already 25B less size of filesystem then the capacity of cartridge. For the resolution of this we have reduced our back up filesystem down-to 1375 GB in order to complete it on one volume.
Note that filesystem is backupby filename having default block size 51200 bytes (100 blocks).
Michael its OK fine that metadata is acquiring that space on tape but we noted this happens some times not all the times.
We faced problem again from last two days that our total data filesystem size is 1200GB & we are taking the backup of the same on LTO5 which has capacity of 1500GB. For the resolution of this we further moved our data from filesystem & bring the filesystem size down to 1000GB & took the backup which was successful. I am unable to understand what amount of metatdata was written on tape. Before LTO 5 we were taking 750 GB filessytem size backup on LTO4 which has native capacity of 800GB and we didn't face problem on that. But I don't know what happens with the LTO5.
I can only speculate as you haven't described your environment completely.
What exactly do you mean by "backup by filename"? Might it be that you have sparse files on this filesystem which get expanded during the backup? (This would mean the restore would also fail because the FS won't provide enough space to accomodate the expanded file(s).)
Many people have issues with LT05 capacities. You mention that your tape drive is 1.5TB native. Does that mean that it's circa 3TB with compression?? What model of LT05 is it exactly? Is hardware compression enabled on it?
Also, what is your backup utility? Are you using tar or cpio or some backup suite? With Unix commands you can normally set the tape capacity and, if you do that, the number of bytes backed up is "counted" as the tape is written and the command calls for a new volume when that number of bytes is reached. It has nothing to do with hitting the actual end-of-tape or anything like that. With some backup suites the tape capacity is often set in a configuration file somewhere.
As other respondees have said, please post more information of your configuration.
Meantime, search for "LT05 compression" on the web.
OS : AIX
Version : 6100-08-02-1316
Filesystem Size which will be backed up: 1400 GB
LTO5 Cartridge Size : 1.5TB Native 3 TB Compressed
LTO5 Tape drive Attributes:
Software Compression: Using compress command
Command used to backup files: find * -print|backup -ivf '/dev/rmt0' '-U'
alt_pathing no Enable Alternate Pathing Support True
autoload no Use Autoloading Feature at End-of-Tape True
block_size 0 Block Size (0=Variable Length) True
compress yes Use Hardware Compression on Tape True
debug_trace no Debug Trace Logging Enabled True
dev_status N/A False
devtype ULT3580- Device Type False
hh_refresh yes Half height refresh Drive False
location Location True
logging no Activate volume information logging True
lun_id 0x0 Logical Unit Number True
max_log_size 500 Maximum size of log file (in # of entries) True
new_name New Logical Name True
node_name 0x2001000e111484e1 World Wide Node Name False
primary_device rmt0 Primary Logical Device False
reserve_key Persistent Reservation Key True
reserve_type reserve_6 Reservation Type True
retain_reserve no Retain Reservation True
rew_immediate no Use Immediate Bit in Rewind Commands True
scsi_id 0x40fe1 SCSI Target ID True
space_mode SCSI Backward Space/Forward Space Record Mode True
sys_encryption no Use System Encryption FCP Proxy Manager True
trace_logging yes Trace Logging Enabled True
trailer_labels no Trailer Label Processing True
wrt_encryption custom System Encryption for Write Commands at BOP True
ww_name 0x2002000e111484e1 World Wide Port Name False
Method:
We have an empty filesystem having size 1400GB. We ftp all the dump files in this filesystem which were already compressed on other AIX lpars using compress command. When the 1400GB filesytem gets 100% full we start taking the backup on LTO5 cartridge using find * -print|backup -ivf '/dev/rmt0' '-U'
This sheds a little more light onto the problem. Unfortunately it rules out my suggestion, because backup - unlike tar - will not expand sparse files. I am not sure how compress handles these.
As you have hardware compression enabled i suggest not using compress on the original files, because they can't compressed two times anyway. Let the hardware in the tape drive deal with compression and save CPU time.
I have to admit this will not solve the enigma, but it should improve matters nevertheless.
But let me inform you our uncompressed total data size is 1350 GB & when we compress the the data files it will become around 200 GB. So with the the help of compress we are able to hold 5-6 days data in a 1400 GB filesystem which will be backed up on cartridge once in a week. It is also save our storage space for not holding uncompressed data which is huge in size.
Some things to clarify: you can compress a file only once. If you use the utility "compress" (or "gzip", "bzip2" or any other packing program), you have compressed it once, using some CPU time to do so. If you want to restore it you will need CPU time too to uncompress it.
If you use the hardware compression, on the other hand, you will still compress it (and about the same ratio as with "compress"), but use no CPU time to do so. The same, when you restore it.
The size data for the LTO5 tape are pure estimations. OK, 1500GB of uncompressed size is a fact, but "3000GB compressed" simply means: we estimate that our average packing rate will be 1:2, regardless of what is thrown in the drives way, so we say it has 1500 x 2 = 3000GB of compressed capacity. In reality the compression ratio one really achieves varies widely, depending on what a file contains. Picture data like bitmaps usually can easily be compressed by a factor of 1:10, the same with DB-files, while executable files give considerably lower ratios, typically ~1:2 with a LZ78-based algorithm.
That means: if you use solely hardware compression will put you in about the same situation as now or as using software compression (the "compress" utility) alone. Everything else is just a waste of resources, because in your current situation the software compression does all the work and the hardware compression achieves most likely absolutely nothing.
