How to add following decimal point to a CSV value?

hi there I being trying to figure out way to add " .0" to an integer value in a csv using sed or awk with out success. just as a work around for 2147483647 32 bit limitation that influxdb is currently having

the data base will accept values and work fine if it has the XXX.0 attached but if only and integer format it caps the database at 2147483647

I already have a shell script that reads the csv and converts the epoch to timestamp
ie

epoch,tx,rx
1534522909,6666,55555
1534522910,6667,55556
timestamp,tx,rx
17 -8- 2018 16:21:49,6666,55555
17 -8- 2018 16:21:49,6667,55556

I just want to add addition scripting to add a following decimal point and 0

timestamp,tx,rx
17 -8- 2018 16:21:49,6666.0,55555.0
17 -8- 2018 16:21:49,6667.0,55556.0

thank you for your time

Welcome to the forum.

Please show your script so we can discuss enhancements / improvements to satisfy your request.

  

#!/bin/bash
 usage()
{

  echo "usage: sysinfo_page [[-f file ] | [-h]]"
}
  while [ "$1" != "" ]; do
    case $1 in 
       -f | --file )           shift
                                filename1=$1
                                ;;
        -h | --help )           usage
                                exit
                                ;;
        * )                     usage
                                exit 1
    esac
    shift
done

cd /tmp/runfiles/tmp

awk 'BEGIN {FS=OFS=","}{$1=strftime("%Y-%m-%d %T",$1)}{print}' $filename1 > $filename1.tmp

sed 's/1970-01-01 00:00:00/timestamp/' $filename1.tmp  > /tmp/runfiles/CSV/$filename1
rm $filename1.tmp

currently to change my time I used this to change my timestamp it might not be perfect but it works well enough..

I suspect i can add in

sed "s/.*/&.0/" $filename1.tmp  to add in the .0 at the end

sed 's/rx.0/rx/' $filename1.tmp and to clean up the rx header

I am just not sure how to do the the TX value

Not sure I understand what the script is doing. All your efforts seem to be doable in one single awk script. Please post an input sample.

the whole script is a secondary processor for a another script that sends CSV to influx. this particular one just converts collectd CVS to the proper format to work with csv-to-influx ( found on github) collectd writes in epoch and csv-to-influx wants timestamp and human readable the primary script gathers the MAC of a openwrt router based on BPI-R1 read a config file for CSV to be processed, passing the name to this script which converts the CSV to the proper format and then once it is converted the primary script builds the format to send the information to an influx database . and then it repeats the process over and over again every 30 seconds or so

input sample CSV: (Collectd)

epoch,tx,rx 
1534522909,6666,55555 
1534522910,6667,55556
  

output sample CSV; ( for csv-to-influx)

timestamp,tx,rx 
2018-8-17 16:21:49,6666.0,55555.0
2018-8-17 16:21:49,6667.0,55556.0
 
  

I think I figured out the awk command

awk -F, '{ $2=$2".0";$3=$3".0";}1' OFS=, test.file

not sure how you would combine all into one awk command

would not one need to fix the csv header after the processing

is there a way to ignore the 3rd colum if it does not exist as some CSV only use 2 columns

How about

awk -F, '
NR == 1         {$1 = "timestamp"
                 print
                 next
                }
                {cmd = "date +\"%F %T\" -d@" $1
                 cmd | getline $1
                 close (cmd)
                 $2 = $2 ".0"
                 if ($3) $3 = $3 ".0"
                }
1
' OFS=, file

Hi RudiC

thank you kindly for the code it works nicely in normal linux arch but not in ARM arch

#!/bin/bash
 usage()
{

  echo "usage: sysinfo_page [[-f file ] | [-h]]"
}
  while [ "$1" != "" ]; do
    case $1 in 
       -f | --file )           shift
                                filename1=$1
                                ;;
        -h | --help )           usage
                                exit
                                ;;
        * )                     usage
                                exit 1
    esac
    shift
done
#cd /tmp/runfiles/tmp


awk -F, '
NR == 1         {$1 = "timestamp"
                 print
                 next
                }
                {cmd = "date +\"%F %T\" -d@" $1
                 cmd | getline $1
                 close (cmd)
                 $2 = $2 ".0"
                 if ($3) $3 = $3 ".0"
                }
1
' OFS=, $filename1

against and actual collectd CSV

epoch,rx,tx
1534477715.852,29429474,724680066
1534477720.854,29433801,724682823
1534477725.852,29437903,724685952
1534477730.852,29439120,724688043
1534477735.852,29446691,724691078
1534477740.852,29452624,724692719
1534477745.852,29457362,724696536
1534477750.852,29461215,724699053
1534477755.852,29465872,724701793
1534477760.853,29466906,724703284
1534477765.853,29471367,724705629
1534477770.852,29475009,724707836
1534477775.852,29480881,724711779
1534477780.852,29484515,724714005
1534477785.852,29488975,724716416
1534477790.854,29490008,724717907
1534477795.852,29495013,724720064
1534477800.852,29498611,724722541
1534477805.852,29502482,724724656
1534477810.852,29506624,724727067
1534477815.852,29510496,724729182
1534477820.852,29512315,724731298
1534477825.852,29516186,724733413
1534477830.852,29520328,724735824
1534477835.852,29525286,724739184
1534477840.852,29529670,724741993
1534477845.853,29533541,724744108
1534477850.852,29535210,724745937
1534477855.852,29539081,724748052
1534477860.852,29543353,724750463
1534477865.852,29549652,724752686
1534477870.852,29553840,724755247
1534477875.852,29557620,724756738
1534477880.852,29559465,724759244
1534477885.852,29566161,724762681
1534477890.852,29570349,724765242

under ARM I get invalid date whereas in X86_64 it works fine

how would I insert my above awk time shift into your code,, As I remember it was a bit of a bugger to get awk time to work correctly under ARM

awk 'BEGIN {FS=OFS=","}{$1=strftime("%Y-%m-%d %T",$1)

------ Post updated at 09:33 PM ------

hi RudiiC
I modified your code slightly

 awk -F, 

' NR == 1         {$1 = "timestamp" 

                  print    

              next  
 
               } 

                {$1=strftime("%Y-%m-%d %T",$1)  

                $2 = $2 ".0"  

                if ($3) $3 = $3 ".0" 

                } 1 ' OFS=, file


 

now it works in both ARM and X86_64

and again thank you kindly for your help

Brilliant, thanks for sharing. My mawk doesn't have the strftime function, so I had to use that clumsy date | getline construct