Inserting duplicate lines in a file

UNIX for Advanced & Expert Users
1

Hi,

I copied the contents of a binary file into a .text file using hd (hexdump) command. The data in binary file is such that I get in many places like following

00000250  00 00 00 00 3f 2d 91 68  3f 69 fb e7 00 00 00 00  |....?-.h?i......|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000002e0  00 00 00 00 00 00 00 00  c9 0d 19 e0 48 c3 50 00  |............H.P.|
000002f0  49 74 24 00 4a 74 24 00  4b 18 96 80 c3 3a 8e 63  |It$.Jt$.K....:.c|
00000300  c2 bd d6 87 3f 1c e6 5c  00 00 00 00 3c 38 db ff  |....?..\....<8..|

where first column is hexadecimal address, 2nd-17th are hex data and last column contains char values corresponding to hex data. Obviously, * denotes block of arbitrary no of lines which are identical to the very previous line. In all cases, this previous line contains all zeros. Hence, this block is made of all zeros - a fact that can be exploited.

I want to replace * with corresponding block of zeros. Like this

00000250  00 00 00 00 3f 2d 91 68  3f 69 fb e7 00 00 00 00  |....?-.h?i......|
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000270  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000280  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000290  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000002a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000002b0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000002c0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000002d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000002e0  00 00 00 00 00 00 00 00  c9 0d 19 e0 48 c3 50 00  |............H.P.|
000002f0  49 74 24 00 4a 74 24 00  4b 18 96 80 c3 3a 8e 63  |It$.Jt$.K....:.c|
00000300  c2 bd d6 87 3f 1c e6 5c  00 00 00 00 3c 38 db ff  |....?..\....<8..|

Actually you need not worry about the last column or 1st column either. All that is required is to insert block of size that depends on addresses of previous and subsequent lines ie, output file can contain only 1st-17th or 2nd-17th columns.

Or, a much simpler and quick alternative is to provide some command or format of output file in which * (or any other symbol) is not used to denote duplicate lines ie, in output file, it prints the entire block of zeros if it's there. Note : This output file must be human readable. I tried copying using hd command in a .txt file and it doesn't open.

Thanks in advance.

2 
     -v      Cause hexdump to display all input data.  Without the -v option,
             any number of groups of output lines, which would be identical
             to the immediately preceding group of output lines (except for
             the input offsets), are replaced with a line comprised of a sin-
             gle asterisk.
1 Like
3

Thanks a lot! It really works. :slight_smile: