# Convert comma separated file to fix length

**URL:** <https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800>\
**Category:** Shell Programming and Scripting\
**Created:** [May 29, 2013, 9:33am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800 "2013-05-29T09:33:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![apenkov](https://community.unix.com/letter_avatar/apenkov/32/6_22734026db12803ebb3e059622ac3534.png) [@apenkov](https://community.unix.com/u/apenkov)\
**Post date:** [May 29, 2013, 9:33am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800/1 "2013-05-29T09:33:46Z")

</div>

Hi, I am converting a comma separated file to fixed field lenght and I am using that:

```nohighlight
COLUMNS="25 24 67 26 39 63 20 34 35 14 397"
(
cat $indir/input_file.dat | \
$AWK -v columns="$COLUMNS" '
  BEGIN {
    FS=",";
    OFS="";
    split(columns, arr, " ");
  }
  {
    for(i=1; i<=NF; i++)
      printf("%-*s%c", arr, $i, (i==NF) ? RS : OFS)
  }
') >> $outdir/output_file.dat

```

It is working fine, but most of the files are big and the performance is very slow. Any ideas how can I make it faster?

Thanks!

---

<div class="post-metadata">

**Author:** ![alister](https://community.unix.com/letter_avatar/alister/32/6_22734026db12803ebb3e059622ac3534.png) [@alister](https://community.unix.com/u/alister)\
**Post date:** [May 29, 2013, 9:54am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800/2 "2013-05-29T09:54:36Z")

</div>

Step 1: Cut the i/o in half by eliminating the unnecessary cat.  
Step 2: Eliminate the for-loop and replace it with a single call to printf.  
Step 3: Don't use gawk unless you must. It's the slowest awk implementation.

Regards,  
Alister

---

<div class="post-metadata">

**Author:** ![apenkov](https://community.unix.com/letter_avatar/apenkov/32/6_22734026db12803ebb3e059622ac3534.png) [@apenkov](https://community.unix.com/u/apenkov)\
**Post date:** [May 29, 2013, 10:02am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800/3 "2013-05-29T10:02:43Z")

</div>

Actually it is:

```nohighlight
AWK="/usr/xpg4/bin/awk" #extended awk for solaris

```

---

<div class="post-metadata">

**Author:** ![alister](https://community.unix.com/letter_avatar/alister/32/6_22734026db12803ebb3e059622ac3534.png) [@alister](https://community.unix.com/u/alister)\
**Post date:** [May 29, 2013, 10:10am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800/4 "2013-05-29T10:10:28Z")

</div>

> [@apenkov](#):
>
> Actually it is:
> 
> ```plaintext
> AWK="/usr/xpg4/bin/awk" #extended awk for solaris
> 
> ```

I wasn't asserting that you are using gawk. Your post did not specify the implementation, so I mentioned gawk's slow execution speed in case it was relevant.

Regards,  
Alister

---

<div class="post-metadata">

**Author:** ![elixir\_sinari](https://community.unix.com/user_avatar/community.unix.com/elixir_sinari/32/2563_2.png) [@elixir\_sinari](https://community.unix.com/u/elixir_sinari)\
**Post date:** [May 29, 2013, 10:23am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800/5 "2013-05-29T10:23:53Z")

</div>

Assuming ASCII data, try

```nohighlight
perl -F, -lape 'BEGIN { ($tmpl = shift) =~ s/(\d+)/A$1/g }
$_ = pack($tmpl, @F)' "$COLUMNS" "$indir/input_file.dat" > "$outdir/output_file.dat" 

```

---

<div class="post-metadata">

**Author:** ![apenkov](https://community.unix.com/letter_avatar/apenkov/32/6_22734026db12803ebb3e059622ac3534.png) [@apenkov](https://community.unix.com/u/apenkov)\
**Post date:** [June 3, 2013, 8:47am UTC](https://community.unix.com/t/convert-comma-separated-file-to-fix-length/329800/6 "2013-06-03T08:47:30Z")

</div>

> [@elixir\_sinari](#):
>
> Assuming ASCII data, try
> 
> ```plaintext
> perl -F, -lape 'BEGIN { ($tmpl = shift) =~ s/(\d+)/A$1/g }
> $_ = pack($tmpl, @F)' "$COLUMNS" "$indir/input_file.dat" > "$outdir/output_file.dat" 
> 
> ```

Hi, thanks. I tried it, but unfortunately it is the same ☹