]I would like to make the second file label 'b' print down the first file label 'a', like shifting down the file creating new lines I want it to print all the way down until the first line of the second file hit the last line of the first file. Would I have to put this into a file itself or could I do it from the terminal?
What it does now:
paste -d ' ' a b
1 a
2 b
3 c
4 d
5 e
6
7
8
9
Desired output:
1 ↓
2 a
3 b
4 c
5 d
6 e
7
8
9
1
2
3 ↓
4 a
5 b
6 c
7 d
8 e
9
Re-looping
1 b
2 c
3 d
4 e
5
6
7 ↓
8 List is about to loop
9 a
#!/bin/bash
# Create temporary file to hold newlines
touch /tmp/$$
# Open both files to read line-by-line
exec 5<file1
exec 6<file2
# Read lines from both until file2 runs out
while read LINE <&6 && read LINE <&5
do
:
done
# Run loop same number of times as lines left in file1.
while read LINE <&5
do
cat /tmp/$$ file2 | paste file1 /dev/stdin
echo >> /tmp/$$
echo
done
cat /tmp/$$ file2 | paste file1 /dev/stdin
rm -f /tmp/$$ # Delete temporary file
# Close file descriptors
exec 5>&-
exec 6>&-
Re-looping I'm less certain of. Perhaps the files themselves should be generated at need. How large are they in a non-trivial example?
Just following the example posted. Using files a and b to create c loop example:
awk '
NR==FNR {a[c++]=$0; next;}
{b[++d]=$0;}
END { for (i=0; i<c; i++) {
for (j=0; j<c; j++) print a[j], b[(j-i)<0?(c+j-i):(j-i)];
print _;
}
}' a b > c
I just want the whole second file to print through the first file. I didn't want the line to have breaks. I was just showing an example of a file completely printing through another.
And how to merge the files together instead of having a space between them?
awk '
NR==FNR {a[c++]=$0; next;}
{b[++d]=$0;}
END { for (i=0; i<c; i++) {
for (j=0; j<c; j++) {
v=j-i;
if (v < 0) v=v+c;
print a[j], b[v];
}
print _;
}
}' a b > c
Does the example posted accomplish what you are looking for or set you in the right track? What is it that you had to go back and re-do to make it work? Can you post the solution you have?
The first example was right, I had removed one of the letters on line 5 by mistake. I'm still using the first example you gave.
I had just wanted the whole second file to print through the first file. I didn't want the line to have breaks. I was just showing an example of a file completely printing through another.
If you are having problems trying to open a large file with awk , please give us details on what you are tying to do, how big the file is, what operating system (including version) you're using, and what version of awk you're using.
I'm trying to get the file on the right to print or generated down the file on the left without the left file printing too. I don't want the left file to print at all. I also want to merge both lines as it prints down the file. Or if there is a way to print files in opposite directions creating new lines. I don't want repeated lines.
The files I am now testing are 2.5gbs each, will try to test 40+gbs later.
I guess I wasn't clear enough in my request. Please explain what diagnostics or erroneous output you have received from awk that make you believe awk is unable to process large files on your system. I don't know of any reason (other than lack of memory) that would keep awk on xfce xubuntu from processing large files.
What awk script were you running when you got the diagnostic: line 11: 7605 killed ?
How big were the two files you were having awk read into memory when you got that diagnostic.
How much memory is installed on your system?
If you were redirecting the output from your awk script to a file, what was the size of that output file and how much free space was there on the filesystem to which you were writing that file when awk was killed?
I only put 2 files in the shell script, I had put 2 very small files to test it out and I was getting outputs from that. Then when I tried 2 bigger files, it didn't have any output after that then it said killed.