Tab Delimiter to csv in python:perl

UNIX for Beginners Q & A
1

Hi
How do we handle delimiter values in python or perl to convert the field values from 3 columns to 2 columns into csv file

Columns Values

Product : Contain one value
ID : Ignore this
Features : Contains multiple values separated by tab

$cat Check_File

Product                ID                   Features 
Apple-for-Laptop 1.2.13.22 itunes app-stores health    # [ Product1 ]
Samsung-for-Mobile 5.43.67  music play-store long-battery    # [ Product2 ]
Nokia 4.1.2 long-battery    # [ Product3 ]

Expected format :

Product, Features
Apple-for-Laptop,itunes
Apple-for-Laptop,app-stores
Apple-for-Laptop,health   
Samsung-for-Mobile,music 
Samsung-for-Mobile,play-store 
Samsung-for-Mobile,long-battery 
Nokia,long-battery

Is there better way to handle this in python/perl . The file is big with 200 Million records.

2

Better than what? Any attempts / ideas / thoughts from your side?

This or this may come close to what you need.

3

Have tried with below option but looks like i am missing something as multiple lines are not getting picked as per expected format in thread

#!/bin/sh
set -f # unquoted $f3
IFS='[\t]'
while IFS=" " read f1 f3
do
printf "$f1, %s\n" $f2
done <   Check_File
4

That is close - try this

while read product id features
do
    printf "${product}, %s\n" ${features%#*}
 done <  Check_File

Your example was using only two variables (f1, f3) to read the fields but using a different one (f2) to print. You were also not deleting the comment at the end of each line.

Andrew

2 Likes
5

Yes this works .
How to transfer the print output to another path or file Ex : /tmp/new_file
I tried using echo , but if i run the code multiple times , the results are getting appended with existing results and doubling up.
Instead of printing in below code - How to ensure the results here are moved to /tmp/new_file . Even with multiple runs , only the latest results should be moved to new/file.

6

Redirection works identical for input as well as for output. Try

$ mv /tmp/new_file /tmp/old_file
$ while read product id features
     do    printf "${product}, %s\n" ${features%#*}
     done < Check_File >/tmp/new_file
4 Likes