LTO tar archives - multiple sessions on one tape

Sabre · March 21, 2024, 10:22am

No issues creating an LTO backup on Debian .. SAS3 with LTO9

tar -czf */dev/st0 /data1

Backing up a second directory to the same tape writes over the first set of data.

tar -czf */dev/st0 /data2

Anyone doing this with LTO care to share how to move the tape pointer to the end of the first job so the second archive does not write over the first ?

Does the first job need an EOF marker ?

Thanks !

munkeHoller · March 21, 2024, 12:29pm

@Sabre , Welcome.

NB: I don't have a tape drive .. (used to have some back in the 90's ... but vague memory of dealing with them)

Take a read at https://linux.die.net/man/1/mt

I think you need to (re)position the tape at the end of the last tarball written
from what I can determine from that man page.

found some potential examples

here (examples #14,#15)

and here

take a look and let us know how you get on.

MadeInGermany · March 21, 2024, 1:10pm

The base tape device like /dev/st0 is "rewinding". By convention an n means "not rewinding".
So go for

tar -czf /dev/st0n /data1
tar -czf /dev/st0n /data2

and use the mt command for checking the current file mumber (status), fast-forward (fsf), rewind.

mt -f /dev/st0n status
mt -f /dev/st0n rewind
mt -f /dev/st0n status
mt -f /dev/st0n fsf 1
mt -f /dev/st0n status

In the previous post the referred articles speak of /dev/nst0
Check with
ls /dev/*st0*
which devices exist.

DrScriptt · March 21, 2024, 10:04pm

+1 to what @munkeHoller and @MadeInGermany suggest.

Do you need to do separate tape sections of tape / separate tape archives on the same tape? -- This seems like something someone might not be aware of when they need to do a restore; e.g. mt to the 3rd tar type thing.

Can you do a single monolithic tar that preserves the directory structure?

Sabre · March 25, 2024, 9:34am

Thanks for your searches.

I did extensive searching and read the tar manual a bunch before this thread. I did move the tape around with FSF, but could not get the second tar to play back. I could position the tape correctly, checked with status and would run the second backup. First first was fine to replay. But I could not get the second to replay.

The file structure in /dev/ looks like this when the drive is powered on:

crw-rw---- 1 root tape      9, 128 Mar 25 19:32 nst0
crw-rw---- 1 root tape      9,   0 Mar 25 19:32 st0
drwxr-xr-x 4 root root          80 Mar 25 19:32 tape
crw-rw---- 1 root tape      9, 224 Mar 25 19:32 nst0a
crw-rw---- 1 root tape      9, 160 Mar 25 19:32 nst0l
crw-rw---- 1 root tape      9, 192 Mar 25 19:32 nst0m
crw-rw---- 1 root tape      9,  96 Mar 25 19:32 st0a
crw-rw---- 1 root tape      9,  32 Mar 25 19:32 st0l
crw-rw---- 1 root tape      9,  64 Mar 25 19:32 st0m

Tape directory contains two pointers:

root@host:/dev/tape/by-path# ls -lt
total 0
lrwxrwxrwx 1 root root  9 Mar 25 19:32 pci-0000:02:00.0-sas-phy6-lun-0 -> ../../st0
lrwxrwxrwx 1 root root 10 Mar 25 19:32 pci-0000:02:00.0-sas-phy6-lun-0-nst -> ../../nst0

I have no problems running a backup to either /dev/st0 OR /dev/nst0. So its not a path issue.

The purpose of a second tar is simply a second backup. They both fit when you have 15TB uncompressed on a tape

Currently running this successfully for a single backup:

tar --label="backup_01_25-3-24" --create --blocking-factor=1024 -b512 -v --multi-volume /data1/ /data2/ /data3/ | sudo mbuffer -m 8G -P 80 -s 262144 -o /dev/st0

mbuffer works amazing combined with tar to keep the SAS pipe busy.. especially when you have some old Sata2 luns.

hicksd8 · March 25, 2024, 12:31pm

I'm coming to this topic very late I know but I'm trying to understand.

So you still have a problem???

I don't think that you've stated which operating system it is; what is it?

Let me just state a few simple facts just to ensure it's crystal and we're all not missing a point here.

By default a tape device will 'rewind' after completing an operation; read or write or whatever.
To avoid a 'rewind' you must specify the 'no rewind' device. So if you write a tar archive to a newly mounted tape using a 'no rewind' device it will stop dead at the end of writing and not rewind. If you were writing the archive using the verbose switch (-v) and watching the file list as the operation proceeded, you should hear no noise from the tape drive at the end, i.e. no rewind is performed. You could then write a second tar archive thereby storing two tar archives on the tape sequentially.
Each of those archives is regarded as a file. (Imagine those archives were written to disk where they would appear as two separate files).
Now, assuming the tape is dismounted/unloaded you should be able to skip over the first archive by either (a) using 'mt' command to skip over one file as posted by @MadeInGermany , or (b) using 'tar -tv' to verbose list the first archive to the screen whilst specifying a 'no rewind' device thereby parking the tape at the end of the first archive. You should now be ready to read/list/restore the second tar archive.

It should be as simple as that. Please post back whatever doesn't seem to work for you.

Sabre · March 29, 2024, 4:24am

Below will backup 5 jobs. This is simple tar commands with no tape buffer consideration, as i detailed above.

For the previous poster, yes that is all known.. OS was Ubuntu.

Was hoping to hear from those using this. Anyhow, we got there.

<Create 5 Backup files>

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
mt -f /dev/nst0 weof 1
mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 0
block number = 0

root@host:/data1/test_dir_1# tar cvf /dev/nst0 /data1/test_dir_1
tar: Removing leading `/' from member names
/data1/test_dir_1/
/data1/test_dir_1/clip01.mp4
/data1/test_dir_1/clip02.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 1
block number = 0

root@host:/data1/test_dir_1# mt -f /dev/nst0 weof 1

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 2
block number = 0

root@host:/data1/test_dir_1# tar cvf /dev/nst0 /data1/test_dir_2
tar: Removing leading `/' from member names
/data1/test_dir_2/
/data1/test_dir_2/clip03.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 3
block number = 0

root@host:/data1/test_dir_1# mt -f /dev/nst0 weof 1

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 4
block number = 0

root@host:/data1/test_dir_1# tar cvf /dev/nst0 /data1/test_dir_3
tar: Removing leading `/' from member names
/data1/test_dir_3/
/data1/test_dir_3/clip04.mp4
/data1/test_dir_3/clip05.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 5
block number = 0

root@host:/data1/test_dir_1# mt -f /dev/nst0 weof 1

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 6
block number = 0

root@host:/data1/test_dir_1# tar cvf /dev/nst0 /data1/test_dir_4
tar: Removing leading `/' from member names
/data1/test_dir_4/
/data1/test_dir_4/clip06.mp4
/data1/test_dir_4/clip07.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 7
block number = 0
root@host:/data1/test_dir_1# mt -f /dev/nst0 weof 1
root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 8
block number = 0

root@host:/data1/test_dir_1# tar cvf /dev/nst0 /data1/test_dir_5
tar: Removing leading `/' from member names
/data1/test_dir_5/
/data1/test_dir_5/clip08.mp4
/data1/test_dir_5/clip09.mp4
/data1/test_dir_5/clip10.mp4
/data1/test_dir_5/clip11.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 9
block number = 0

root@host:/data1/test_dir_1# mt -f /dev/nst0 weof 1

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 10
block number = 0

--------------------------------------------------

root@host:/data1/test_dir_1# mt -f /dev/st0 rewind

--------------------------------------------------

root@host:/data1/test_dir_1# mt -f /dev/nst0 status
drive type = 114
drive status = 1577058304
sense key error = 0
residue count = 0
file number = 0
block number = 0

<echo contents of archive>

root@host:/data1/test_dir_1# tar tvf /dev/nst0
drwxr-xr-x user/user         0 2024-03-29 14:08 data1/test_dir_1/
-rwxr--r-- user/user 533079872 2023-08-16 21:57 data1/test_dir_1/clip01.mp4
-rwxr--r-- user/user 311155345 2023-08-25 22:06 data1/test_dir_1/clip02.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 asf 2

root@host:/data1/test_dir_1# tar tvf /dev/nst0
drwxr-xr-x user/user         0 2024-03-29 14:08 data1/test_dir_2/
-rwxr--r-- user/user 311155345 2023-08-25 22:06 data1/test_dir_2/clip03.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 asf 4

root@host:/data1/test_dir_1# tar tvf /dev/nst0
drwxr-xr-x user/user         0 2024-03-29 14:08 data1/test_dir_3/
-rwxr--r-- user/user 2997280033 2023-05-04 08:03 data1/test_dir_3/clip04.mp4
-rwxr--r-- user/user  100000000 2023-06-19 21:17 data1/test_dir_3/clip05.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 asf 6

root@host:/data1/test_dir_1# tar tvf /dev/nst0
drwxr-xr-x user/user         0 2024-03-29 14:09 data1/test_dir_4/
-rwxr--r-- user/user 378574565 2023-06-09 21:05 data1/test_dir_4/clip06.mp4
-rwxr--r-- user/user 119505113 2023-06-02 22:18 data1/test_dir_4/clip07.mp4

root@host:/data1/test_dir_1# mt -f /dev/nst0 asf 7

root@host:/data1/test_dir_1# mt -f /dev/nst0 asf 8

root@host:/data1/test_dir_1# tar tvf /dev/nst0
drwxr-xr-x user/user         0 2024-03-29 14:04 data1/test_dir_5/clip08.mp4
-rwxr--r-- user/user 620793818 2020-04-17 09:29 data1/test_dir_5/clip09.mp4
-rwxr--r-- user/user 304693160 2020-04-17 09:25 data1/test_dir_5/clip10.mp4
-rwxr--r-- user/user 782159799 2020-04-17 09:30 data1/test_dir_5/clip11.mp4
-rwxr--r-- user/user 304693160 2020-04-17 09:22 data1/test_dir_5/clip12.mp4

<out of sequence - goto any archive file>

root@host:/data1/test_dir_1# mt -f /dev/nst0 asf 4

root@host:/data1/test_dir_1# tar tvf /dev/nst0
drwxr-xr-x user/user         0 2024-03-29 14:08 data1/test_dir_3/
-rwxr--r-- user/user 2997280033 2023-05-04 08:03 data1/test_dir_3/
-rwxr--r-- user/user  100000000 2023-06-19 21:17 data1/test_dir_3/
root@host:/data1/test_dir_1#

system · April 5, 2024, 4:24am

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.