Merge multiple columns into one using cat

UNIX for Beginners Q & A
1

I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file.

file1.txt:
1234
1234
1234

file2.txt:
aaaa
bbbb
cccc
dddd

cat file1.txt file2.txt > file3.txt
file3.txt:
1234
1234
1234aaaa
bbbb
cccc
dddd

Could this be an error associated with the command or a bug?
Thanks.

2

Hello geomarine,

It is working well and fine for me and moreover I am getting different Input_file(s) output from new line only as follows.

 
cat file1.TXT file2.TXT > file3.TXT
cat file3.TXT
1234
1234
1234
aaaa
bbbb
cccc
dddd

Could you please check once if your Input_file is having carriage returns in it? By doing cat -v Input_file if yes then kindly remove them from following command.

 
tr -d '\r' < Input_file > temp_file && mv temp_file Input_file

Let us know how it goes then, I hope this helps.

Thanks,
R. Singh

1 Like
3

Probably the last line does not have a closing linefeed character. Proper Unix files are required to have this..

See if this works:

echo | cat file1.txt - file2.txt > file3.txt
1 Like
4

Thanks RavinderSingh13 and Scrutinizer . I check to see if there are carriage returns with:

cat -v Input_file

result:

cat -v file3.txt 
1234^M
1234^M
1234aaaa^M
bbbb^M
cccc^M
dddd

using this command suggested:

tr -d '\r' < file1.txt > temp_file && mv temp_file file1.txt

the carriage return was removed. However, the result is still same:

1234
1234
1234aaaa
bbbb
cccc
dddd

when I use this command:

echo | cat file1.txt - file2.txt > file3.txt

the result is:

123412341234aaaabbbbccccdddd

I am using BASH shell.

5

What OS are you using?

6

Windows 7 Professional

7

You have shown us that you have two input files with <carriage-return><new-line> line separators and no final line terminators (i.e., DOS text file format).

You have shown us several outputs that you have gotten from what you originally tried and form what others have suggested and you have told us that none of them are what you want. But, from your description and from your examples, I don't know whether you're trying to get all lines from each input file combined into single lines (one for each input file) in the output, all input lines from all files combined into a single output line (sequentially or pairwise), pairs of lines (one from each input file) combined into output lines, or something else.

Please show us the output you are hoping to get from your sample input files and tell us whether you want that output to be in DOS text file format (like your input files) or in standard UNIX text file format (i.e. <new-line> line terminators on every line with no <carriage-return> characters).

1 Like
8

Thanks Don Cragun,
My objective is to have a single column data i.e. <new-line> line terminators on every line with no <carriage-return> as follow:
file1.txt:

1234
1234
1234

file2.txt:

aaaa
bbbb
cccc
dddd

cat file1.txt file2.txt > file3.txt

file3.txt:

1234
1234
1234
aaaa
bbbb
cccc
dddd

Thanks.

9

Hmmm - repeating your unmodified problem description again doesn't really help, esp. when the error seems to have been tracked down by Scrutinizer - the (first?) file doesn't have the necessary line terminator on the last line. You'll have to repair that, esp. as his proposal doesn't work on your system. What would be the result of

{ cat file1; echo; cat file2; } > file3
1 Like
10

You can use the following to directly process DOS format text files without manually stripping out the <carriage-return> characters from your input files...

Create a file named merge containing:

for file in "$@"
do	tr -d '\r' "$file"
	echo
done

Then you can issue the command:

sh merge file1.txt file2.txt > file3.txt

to get what you want as long as the file pathnames you pass to merge as operands are DOS format text files.

The file1.txt and file2.txt are sample file pathname operands gleaned from post #1 in this thread. This script will work with one or more file pathname operands.

1 Like
11

Hi geomarine...

Think of your problem and how to get round it.
You want a single file in UNIX format.

Here is a DEMO of your two files:

#!/bin/sh
# All longhand to see the sequence of events.

echo "Create files as per OP."
printf "1234\r\n1234\r\n1234" > /tmp/file1.txt
printf "aaaa\r\nbbbb\r\ncccc\r\ndddd" > /tmp/file2.txt
echo "Join together to test."
cat /tmp/file1.txt /tmp/file2.txt > /tmp/file3.txt
cat -v /tmp/file3.txt
echo ""
echo "Remove ^M, <CR> character and create temp files 4 and 5..."
tr -d '\r' < /tmp/file1.txt > /tmp/file4.txt
tr -d '\r' < /tmp/file2.txt > /tmp/file5.txt
cat /tmp/file4.txt /tmp/file5.txt > /tmp/file3.txt
echo "...and check the error exists."
cat -v /tmp/file3.txt
echo ""
echo "Now add the newlines in the correct place, longhand."
cat /tmp/file4.txt > /tmp/file3.txt
echo "" >> /tmp/file3.txt
cat /tmp/file5.txt >> /tmp/file3.txt
echo "" >> /tmp/file3.txt
echo "Finalisation."
cat /tmp/file3.txt
echo "Check file validity using 'hexdump'."
hexdump -C /tmp/file3.txt

Result: OSX 10.13.4, default bash terminal calling OSX 'sh'.

Last login: Tue May 29 17:17:20 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> chmod 755 CATFiles.sh
AMIGA:barrywalker~/Desktop/Code/Shell> ./CATFiles.sh
Create files as per OP.
Join together to test.
1234^M
1234^M
1234aaaa^M
bbbb^M
cccc^M
dddd
Remove ^M, <CR> character and create temp files 4 and 5...
...and check the error exists.
1234
1234
1234aaaa
bbbb
cccc
dddd
Now add the newlines in the correct place, longhand.
Finalisation.
1234
1234
1234
aaaa
bbbb
cccc
dddd
Check file validity using 'hexdump'.
00000000  31 32 33 34 0a 31 32 33  34 0a 31 32 33 34 0a 61  |1234.1234.1234.a|
00000010  61 61 61 0a 62 62 62 62  0a 63 63 63 63 0a 64 64  |aaa.bbbb.cccc.dd|
00000020  64 64 0a                                          |dd.|
00000023
AMIGA:barrywalker~/Desktop/Code/Shell> _
1 Like
12

Please don't forget the requestor is using Win 7 Pr., probably with a cygwin or busybox setup, both with a (limited?) set of not-necessarily-*nix-compatible tools. I'm not sure that s/he will really benefit from a pure *nix result file without \r line terminators on the Win 7 system.

2 Likes
13

I was well aware of the situation but he did stress that he wanted it in UNIX format in an earlier post.
Good point though, but it is just as easy to put '\r\n's back if that becomes a necessity.

1 Like
14

My Cygwin crashed and I have to re-install it so I couldn't test the recommendations. I have tested the 'cat' on a windows 10 OS and it works fine. So, the problem is with my computer though am not sure what could be wrong while such a simple command to fail. I decided to move my data to another system for the operation. Thanks