Scripting issue

Shell Programming and Scripting
1

Hello,

I have an ASCII file (many files of the same format, but different dates and numbers) in the format like below:

2008.01.02,08:00,1.46520,1.46520,1.46410,1.46440,70
2008.01.02,08:05,1.46450,1.46560,1.46440,1.46540,79
2008.01.02,08:10,1.46530,1.46540,1.46490,1.46500,46
2008.01.02,08:15,1.46490,1.46500,1.46470,1.46500,43
2008.01.02,08:20,1.46510,1.46600,1.46500,1.46580,44
. . .

Here 2008 - is a year. Other lines might start with year 1999 or 2004, etc.

Need to convert it to the following style:

2008-01-02 08:00 1,46520 1,46520 1,46410 1,46440 70
2008-01-02 08:05 1,46450 1,46560 1,46440 1,46540 79
2008-01-02 08:10 1,46530 1,46540 1,46490 1,46500 46
2008-01-02 08:15 1,46490 1,46500 1,46470 1,46500 43
2008-01-02 08:20 1,46510 1,46600 1,46500 1,46580 44
. . .

After that need to split the new file according to a year (and move each one to its own directory - "2008" or "1999", "2004" and so on).

I have Ubuntu installed, so perl, awk or whatever should be there (unfortunately I'm not familiar with how to programming them). Maybe anybody can help with this task.

Thank you!

2

This is a start up

awk -F, '{gsub(/\./,"-",$1);gsub(/\./,",");print > substr($0,1,4)}' file
3

Thank you, danmero!

The conversion is excellent, just all new files need to have the same =original)
filename, but each of them need to be located in directory according to a year (the directories will be created in the first run and following runs will use the same directories or we can assume that directories already exist).

For Example: filename.txt ===>>>
1999/filename.txt
2000/filename.txt
2001/filename.txt
2002/filename.txt
. . .

Maybe you know how to do this?

Thank you very much!

4

I hope it helps...

 
cut -d"." -f1 < filename.txt | sort -u | while read year
do
mkdir ${year}
cd ${year}
awk -F, -v yr="${year}" '{if(yr==substr($0,1,4)){gsub(/\./,"-",$1);
gsub(/\./,",");print > "filename.txt"}}' /location/of/filename.txt
cd ..
done
5

This works fine!
Thank you for your help!

[COLOR="\#738fbf"]

---------- Post updated at 01:03 PM ---------- Previous update was at 12:58 PM ----------

One more question - I copied the previously mentioned code into a file.

  • Is it possible to create a variable from "filename.txt", so each time (for every new file) I will need to change it only in one place? (currently it is mentioned 3 times)

Thank you!

 
cut -d"." -f1 < filename.txt | sort -u | while read year
do
mkdir ${year}
cd ${year}
awk -F, -v yr="${year}" '{if(yr==substr($0,1,4)){gsub(/\./,"-",$1);
gsub(/\./,",");print > "filename.txt"}}' /location/of/filename.txt
cd ..
done
6

perl:

my %hash;
open FH,"<a.txt";
while(<FH>){
  chomp;
  my @tmp = split(",",$_);
  my @tmp1= split("[.]",$_,2);
  my $year=$tmp1[0];
  $tmp[0]=~s/[.]/-/g;
  map {s/[.]/,/;} @tmp[2..5];
  my $str=$tmp[0]." ".$tmp[1]." ".$tmp[2]." ".$tmp[3]." ".$tmp[4]." ".$tmp[5]." ".$tmp[6];
  push @{$hash{$year}}, $str;
}
foreach my $key(sort keys %hash){
  my $file="file_".$key.".txt";
  open TFH,">$file";
  local $,="\n";
  print TFH @{$hash{$key}};
  close TFH;
}
7

Hello summer_cherry,

The files are generated correctly, just need to give them the name of the
original file and create them in a "year"-directory. For Eample:

Also, is it possible to specify the name of the file only when running the script? - i.e.

Thank you!

8

If you put this into a file, say for example 'script'

 
fileloc=$1
file=${fileloc##*/}
cut -d"." -f1 "${fileloc}" | sort -u | while read year
do
mkdir ${year}
cd ${year}
awk -F, -v yr="${year}" -v fl="${file}" '{if(yr==substr($0,1,4)){gsub(/\./,"-",$1);
gsub(/\./,",");print > fl}}' "${fileloc}"
cd ..
done

you can run it as..

 
./script /location/of/filename.txt
9

Hi malcomex999,

Previous scripts already saved me a lot of time. This one works Great!

Thank you!