Joining lines from two files - please help

chandra004 · July 21, 2006, 4:35pm

Hello,
I have 2 files say
File 1 has
ABC
DEF
GHI
File 2 has
123
456
789

I need output as
ABC 123
DEF 456
GHI 789

I tried awk and sed but not able to get it in the right way.
Please help. Thanks

tmarikle · July 21, 2006, 4:40pm

paste file1 file2 > joined_file

chandra004 · July 21, 2006, 5:31pm

That helps, Thanks. I am looking for some thing like, even if 123 is in a different line on file2, it has to print next to ABC.
What I am trying to do is, I am doing ls -lrt on two directories which have same files ,and printing filename and the size to a file.
Now I want those two files with filename and filesize to be in the output file so that I can compare the file sizes.
When I say paste, it works when there are exactly same files in the directories, but if there is an extra file in any one dir, the pattern changes.

Any ideas ???

tmarikle · July 21, 2006, 6:43pm

You could try something like this:

    printf "%-40s %-15s %-15s %-15s\n" "File" "Size Location 1" "Size Location 2" "Difference"
    print "========================================" "===============" "===============" "==============="
    for i in $(ls -1F /your/first/path)
    do
        [[ ${i} = */ ]] && continue
        FILE_SIZE_1=$(ls -l /your/first/path/${i} | nawk '{print $5}')
        FILE_SIZE_2=$(ls -l /your/second/path/${i} | nawk '{print $5}')
        printf "%-40s %15d %15d %15d\n" "${i}" "$FILE_SIZE_1" "$FILE_SIZE_2" "$(( FILE_SIZE_1 - FILE_SIZE_2 ))"
    done

chandra004 · July 24, 2006, 11:34am

Thank you very much, 'tmarikle'. It really helped and I am trying to extend your script for recursively checking in any/all of the directories also under the path. Not able to make the loop work. Can you advise?

Thanks
chandra

tmarikle · July 24, 2006, 12:01pm

What does your loop look like now? What habe you tried?

chandra004 · July 24, 2006, 12:38pm

I am not able to go anywhere as I couldn't make the loop to be recursive such that if it sees a directory under it, it should go back to see if there are further directories or else execute the script you gave. But not able to get it right at all.

Need help !!!

tmarikle · July 24, 2006, 1:11pm

I would change the loop and use the "find" command but this complicates the inner parts of the query a little.

# Define your target directories
YOUR_ROOT_DIR=/your/first/path/
YOUR_MIRROR_DIR=/your/second/path/

printf "%-40s %-15s %-15s %-15s\n" "File" "Size Location 1" "Size Location 2" "Difference"
print "========================================" "===============" "===============" "==============="

# Recurse through "YOUR_ROOT_DIR" and find files and their sizes
for i in $(find ${YOUR_ROOT_DIR} -name \* -ls | nawk '$3 !~ /d.*/ {print $7, $11}')
do
    # Strip file size and "YOUR_ROOT_DIR" from find + awk output
    FILE_BASE=${i##+([0-9       ])${YOUR_ROOT_DIR}}

    # Get file size from YOUR_ROOT_DIR
    FILE_SIZE_1=${i%%+([        ]*)}

    # Fetch file size from YOUR_MIRROR_DIR
    FILE_SIZE_2=$(ls -l ${YOUR_MIRROR_DIR}${FILE_BASE} | nawk '{print $5}')

    # Output the results
    printf "%-40s %15d %15d %15d\n" "${FILE_BASE}" "$FILE_SIZE_1" "$FILE_SIZE_2" "$(( FILE_SIZE_1 - FILE_SIZE_2 ))"
done

chandra004 · July 24, 2006, 2:41pm

I am getting two kinds of errors while running it. In the lowest idrectory, even for the file it is looking for '/' in the end.

ls: /home/rpmuser/COB_bw_5.3.2/home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war: No such file or directory

./filesize2.sh[23]: /home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war: unexpected `/'

In the script,
/home/rpmuser/COB_bw_5.3.2 YOUR_ROOT_DIR
/home/rpmuser/COB_bw_5.3.0 YOUR_MIRROR_DIR

Please...

tmarikle · July 24, 2006, 3:27pm

Let's try again:

# Define your target directories
YOUR_ROOT_DIR=/home/rpmuser/COB_bw_5.3.2
YOUR_MIRROR_DIR=/home/rpmuser/COB_bw_5.3.0

printf "%-40s %-15s %-15s %-15s\n" "File" "Size Location 1" "Size Location 2" "Difference"
print "========================================" "===============" "===============" "==============="

# Recurse through "YOUR_ROOT_DIR" and find files and their sizes
for i in $(find ${YOUR_ROOT_DIR} -name \* -ls | nawk '$3 !~ /d.*/ {print $7 ":" $11}')
do
    # Strip file size and "YOUR_ROOT_DIR" from find + awk output
    FILE_BASE=${i##*:${YOUR_ROOT_DIR}/}
    # Get file size from YOUR_ROOT_DIR

    FILE_SIZE_1=${i%%:*)}

    # Fetch file size from YOUR_MIRROR_DIR
    if [[ -f ${YOUR_MIRROR_DIR}/${FILE_BASE} ]]
    then
        FILE_SIZE_2=$(ls -l ${YOUR_MIRROR_DIR}/${FILE_BASE} | nawk '{print $5}')
        DIFF=$(( FILE_SIZE_1 - FILE_SIZE_2 ))
    else
        FILE_SIZE_2="Not found"
        DIFF="???"
    fi

    # Output the results
    printf "%-40s %15d %15s %15s\n" "${FILE_BASE}" "$FILE_SIZE_1" "$FILE_SIZE_2" "$DIFF"
done

chandra004 · July 24, 2006, 4:43pm

While executing the script - particularly the line -
FILE_SIZE_1=${i%%:*)},
I am getting
6519 6519
/home/rpmuser/COB_bw_5.3.0/5.3/doc/pdg/dg18.htm printf: /home/rpmuser/COB_bw_5.3.0/5.3/doc/pdg/dg18.htm: expected a numeric value

The same message is being shown for
FILE_SIZE_1=${i%%+([ ]*)}

Can you suggest what the issue is ??

tmarikle · July 24, 2006, 5:21pm

Show me the output of this:

find ${YOUR_ROOT_DIR} -name \* -ls | nawk '$3 !~ /d.*/ {print $7 ":" $11}

chandra004 · July 24, 2006, 5:29pm

Without putting it in the for loop and just run it, you get the output:

32674 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/license.htm
3108 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/lof.htm
2790 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/lot.htm
3102 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/preface.htm
5815 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/preface2.htm
3457 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/preface3.htm
2693 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/title.htm
12551 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/admin/toc.xml
920 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/index.htm
67726 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/html/license.htm
1034294 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_concepts.pdf
1058579 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_error_codes.pdf
208336 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_quickstart.pdf
2722852 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_process_design_guide.pdf
1233363 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_administration.pdf
192414 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_installation.pdf
4308452 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/doc/bw/pdf/tib_bw_palette_reference.pdf
594368 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/tib_bw_release_notes.pdf
651241 /home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/_uninst/Tibuninstall.bin

But when I put it in the loop, I get error.

tmarikle · July 24, 2006, 5:44pm

That can't be the output. Each file size must be separated by a colon ":".

read the AWK output: {print $7 ":" $11}

The commands inside the loop are parsed based on the colon character.

chandra004 · July 24, 2006, 6:40pm

Now, I get this error:

5.3/examples/activities/Java/javaCode/JavaTest.zip printf: 17102:/home/rpmuser/TIB_bw_5.3/TIB_bw_5.3.0/5.3/examples/activities/Java/javaCode/JavaTest.zip: value not completely converted
17102

for this line of code:
FILE_SIZE_1=${i%%:*)}
Without that, as you said the output just for line find....
4625:/home/rpmuser/COB_bw_5.3.0/5.3/doc/bw/html/pdg/dg16.htm
4122:/home/rpmuser/COB_bw_5.3.0/5.3/doc/bw/html/pdg/dg17.htm

If I run with for loop, I get :
File Size Location 1 Size Location 2 Difference
5.3/admin-plugins/BWAdmin.war
5.3/admin-plugins/UDDIAdmin.war
I am getting File_base but File_Size_1 is giving me issues.

chandra004 · July 24, 2006, 6:59pm

I am getting all zeroes for the File_Size_1. Now i am getting the output but as the File_size_1 value is 0, all others are messed up.
Am I putting it wrong somewhere ??

chandra004 · July 24, 2006, 7:23pm

Alright, something's going on with with me for last 30 mins.
But I feel it is not able to differentiate the filesize..coz I am getting:
File Size Location 1 Size Location 2 Difference
./filesize3.sh[31]: 5336821:/home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war: unexpected `:'

With Earlier script that you sent, I was getting
./filesize2.sh[23]: /home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war: unexpected `/'

chandra

tmarikle · July 24, 2006, 7:33pm

Well even though I don't have your system in front of me and even though I did throw this together quickly, your examples aren't lining up with your earlier posts so it's very hard to solve your problem.

doesn't really agree with

My reply was simply to guide you into one possible solution and I essentially hard coded my assumptions. Please clarify the requirements if you need further help.

chandra004 · July 24, 2006, 11:04pm

hey tmarikle,
I am sorry to give you wrong data. Actually I am trying this in two diferent boxes. I was replicating the scenario which I was giving you all the time. But I posted you the output of the other box.
Its not to misguide you. But the issue I am having is, as I told you in my previous mail and I am not able to figure it out and I see it in both boxes.
File Size Location 1 Size Location 2 Difference
./filesize3.sh[31]: 5336821:/home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war: unexpected `:'
for this line of code:
FILE_SIZE_1=${i%%:*)} is what I am not able to understand. When I comment it out, I get '0' for Filesize1 and other results are right.

Please let me know if you have a resolution and thank you once again for your time.

tmarikle · July 25, 2006, 11:46am

Actually there is somewhat of an error on that line but was working in my environment.

The expression ${i%%:)} should lose the right parenthesis and should look like ${i%%:}

Ultimately you are trying to remove all of the text from "5336821:/home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war" to the right of, and including, the colon. The output should be 5336821 by itself. Also, your shell matters, I use KSH and this won't work with every shell. Let's first prove whether this is a problem by trying the following:

Assign a variable, i, your string of text: i='5336821:/home/rpmuser/COB_bw_5.3.0/5.3/admin-plugins/BWAdmin.war'
print the output using the expression that I provided you: print "${i%%:*}"

You should get the 5336821 portion of the string by itself.

Alternatively, you can use cut or awk to accomplish the same thing.

print "${i}" | nawk -F: '{print $1}'
or
print "${x}" | cut -d: -f1

It's your choice but I prefer to use the first method since it utilizes shell built-in functionality and doesn't require another process.