Slow FFT in ksh93 and awk.

Well i set myself a challenge to have an FFT function using nothing but ksh93 and awk.
It took some serious jiggery pokery and concentration with all the brackets and '$' characters but here is the result. It is RADIX 2 only, but hey, show me another UNIX shell script that does it. It IS SLOW but is ideal for my needs.
It uses ksh's floating point arithmetic and awk to obtain the SINE and COSINE requirements although even awk is not needed for this as typical Taylor series for these could have been created.
This was a learning curve primarily because ksh has a few minor differences to bash; however this is the result.
(For those that know the AMIGA, I am now thinking of porting this to ARexx although it has no SINE, COSINE or SQRT. I have already done SINE and SQRT for ARexx so COSINE is just a simple SINE change.)
This is in the Public Domain, CC0, here and hopefully there will be some interest from outside our community.
The results of the code below:

Last login: Thu Aug 30 16:06:02 on ttys000
AMIGA:amiga~> cd Desktop/Code/Shell
AMIGA:amiga~/Desktop/Code/Shell> ksh
AMIGA:uw>   
AMIGA:uw> ./FFT_ksh

Real floating point values:
8.5 3.637821186735 -1.70710678119 0.051425210515 1.5 -0.17274555408 -0.292893218805 0.483499156825 0.5 0.483499156845 -0.292893218815 -0.17274555407 1.5 0.05142521051 -1.7071067812 3.637821186735

Imaginary floating point values:
0 -1.93071440555 -0.5 -0.2414680083 -1 0.465638772885 0.5 -1.223607624345 0 1.223607624355 -0.499999999995 -0.46563877289 1 0.24146800831 0.500000000005 1.930714405535

Absolute values of complex numbers:
8.500000 4.118420 1.778820 0.246883 1.802780 0.496649 0.579471 1.315670 0.500000 1.315670 0.579471 0.496649 1.802780 0.246883 1.778820 4.118420 

Done...
AMIGA:uw> exit
AMIGA:amiga~/Desktop/Code/Shell> _

The script:

#!/bin/ksh
# KSH Version: 'Version AJM 93u+ 2012-08-01'
# From both ${.sh.version} and ${KSH_VERSION}.
#
# Awk Version: 'awk version 20070501'
#
# OSX 10.13.6, default bash terminal running ksh.
# Darwin Barrys-MacBook-Pro.local 17.7.0 Darwin Kernel Version 17.7.0:
# Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64

# All of this code assumes functions can use global variables, arrays, etc...
# Only constant required, 11 decimal places.
PI=3.14159265359

fft()
{
	# Check for equal length real and imaginary arrays.
	if [ ${#REAL_ARRAY[@]} -ne ${#IMAG_ARRAY[@]} ] || [ ${#REAL_ARRAY[@]} -le 1 ] || [ ${#IMAG_ARRAY[@]} -le 1 ]
	then
		echo "ERROR 1:"
		echo "Number of elements, both real and imaginary, must be the same."
		exit 1
	fi
	# Check for powers of 2.
	N=${#REAL_ARRAY[@]}
	if [ $(( $N&(N-1) )) -ne 0 ]
	then
		echo "ERROR 2:"
		echo "Padding or cropping is required to the nearest power of 2 elements."
		exit 2
	fi
	REAL=()
	IMAG=()
	# For each element into REAL() and IMAG() arrays...
	for ((K=0; K<=$(( $N-1 )); K++))
	do
		SUMREAL=0.0
		SUMIMAG=0.0
		# For each element from REAL_ARRAY() and IMAG_ARRAY() arrays...
		for ((T=0; T<=$(( $N-1 )); T++))
		do
			ANGLE=$(( 2.0*PI*$T.0*$K.0/$N.0 ))
			# Use awk for the COSINE and SINE values...
			COS_ANGLE=$( awk 'BEGIN { printf("%.11f", cos('$ANGLE')); }' )
			SIN_ANGLE=$( awk 'BEGIN { printf("%.11f", sin('$ANGLE')); }' )
			SUMREAL=$(( $SUMREAL+${REAL_ARRAY[$T]}*$COS_ANGLE+${IMAG_ARRAY[$T]}*$SIN_ANGLE ))
			SUMIMAG=$(( $SUMIMAG-${REAL_ARRAY[$T]}*$SIN_ANGLE+${IMAG_ARRAY[$T]}*$COS_ANGLE ))
		done
		REAL[$K]=$SUMREAL
		IMAG[$K]=$SUMIMAG
	done
}

abs()
{
	SQUARE=$(( (${REAL[COUNT]}**2)+(${IMAG[COUNT]}**2) ))
	ABS=$( awk 'BEGIN { printf(sqrt('$SQUARE')); }' )
}

# 16 samples of single cycle square wave, floating point values must always be positive.
#
# REAL_ARRAY=( 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
# IMAG_ARRAY=( 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
#
# 8.000000 5.125830 0.000000 1.799950 0.000000 1.202690 0.000000 1.019590
# 0.000000 1.019590 0.000000 1.202690 0.000000 1.799950 0.000000 5.125830

# This has midway '0.5' value padding at the end to bring to the power of 2.
# This is the running default.
#
REAL_ARRAY=( 1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5 )
IMAG_ARRAY=( 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
#
# 8.500000 4.118420 1.778820 0.246880 1.802780 0.496650 0.579470 1.315670
# 0.500000 1.315670 0.579470 0.496650 1.802780 0.246880 1.778820 4.118420

# Standard test values.
#
# REAL_ARRAY=( 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 )
# IMAG_ARRAY=( 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
#
# 4.000000 2.613130 0.000000 1.082390 0.000000 1.082390 0.000000 2.613130

# This assumes all needed variables are global!
fft

# This lot can be discarded...
echo ""
echo "Real floating point values:"
echo "${REAL[@]}"
echo ""
echo "Imaginary floating point values:"
echo "${IMAG[@]}"
echo ""
echo "Absolute values of complex numbers:"

for COUNT in $( seq 0 1 $(( ${#REAL[@]}-1 )) )
do
	abs
	printf "%f " "$ABS"
done
echo ""
echo ""
echo "Done..."

Enjoy all.
Snippets like these are what keep me from going insane.

------ Post updated at 07:31 PM ------

Hi bakunin...

Thank you for the info but......
I do bizarre things like this to expand my knowledge on the limits of languages, not the limits of HW.
I even got my USB Arduino Diecimila board to talk to my AMIGA with minor HW mods, uploaded to AMINET, using BWBasic purely as a challenge, that's right BWBasic. It has had hundreds of DLs which shows people ARE interested in esoteric stuff like mine.
Real time applications are of no interest at the moment as AudioScope.sh is by definition slow as is its builtin Spec-An using Python 2.7.x with module imports of scipy and scipy.IO. As these ARE dependencies then I have written FFT code that works from Python 2.0.1 to the current 3.7.0. I now have since modified code to work from Python 1.4.0, the only version to work on a stock AMIGA A1200, also up to 3.7.0. Boy how, Python has changed. These mods can be seriously difficult to work around but now I have my building blocks to work with to adapt into AudioScope.sh for AudioScope.sh's Spec-An section without the need for dependencies.
ATM FFT on ksh was a new baby to see how easy it would be, difficult? YES; impossible? NO. I really do get enjoyment out of stretching languages to strange esoteric limits.
Can I do it in AWK? Dunno, I am not good with AWK but ARexx for the AMIGA WILL be serious fun as it does have basic floating point arithmetic but NO SINE, COSINE, SQRT, etc...

1 Like

I really like FFTs, did quite some more than 30 years ago...

Calling awk (or any external command) twice in a nested loop will inevitably be slow. Why don't you do all of it within one awk script?

Hi RudiC...

I just love trying near impossible stuff like this. I watch with awe when I see some of you guys code in the shell on things I know little or nothing about so the code above is the way I can get to learn strange subtleties in languages and apply it to things I do know something about.

so two reasons why I haven't tried awk yet.
1) I wanted to see if it was possible to do it in a shell script.
And 2) I don't know enough awk yet and would love to do it, so it IS on the books...

The code is slow but there is no other at all so it might come onto Google's search list bringing people here.

Hi all...

Well guys, it took a while because of the serious limitations of ARexx for a stock AMIGA A1200 but here is a working FFT for it. ARexx has no ARRAY facility but there is a workaround. ;o)
Boy oh boy, ARexx's arithmetic is so convoluted and floating point precision is not the best.
This was a challenge. I could have generated SIN, COS and SQRT in pure ARexx but decided to use an external ARexx math(s) library instead...
It would be just as easy to create SIN, COS and SQRT in ksh too eliminating awk entirely but awk just made it easier...
Just for the record the AMIGA shell/terminal has a large subset of xterm's escape codes.
An exercise in futility? Maybe but serious fun nevertheless...
This is through FS-UAE AMIGA emulation and my real A1200 setup for easy transfer from this Macbook Pro.
ARexx code:

/* Simple_FFT_DEMO.rexx */

/* This ARexx version needs this dependency: */
/* http://aminet.net/package/util/rexx/RexxMathLib */
CALL ADDLIB('rexxmathlib.library',0,-30,0)

NUMERIC DIGITS 14

SAY ""
SAY "An experimental method to do an FFT in ARexx using the standard test values."
SAY "It requires rexxmathlib.library for the SIN and COS."
SAY "$VER Simple_FFT_DEMO.rexx_(C)06-09-2018_B.Walker_issued_under_GPL2."
SAY ""

/* Create standard test _ARRAYS_ REAL and IMAGINARY. */
/* These MUST be powers of 2 and greater than 2 in size. */
/* Real values, '1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0' */
/* Imag values, '0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0' */
/* Not really ARRAYS but good enough for this DEMO! */
DO N=0 TO 3 BY 1
	REAL_ARRAY.N=1.0
END
DO N=4 TO 7 BY 1
	REAL_ARRAY.N=0.0
END
DO N=0 TO 7 BY 1
	IMAG_ARRAY.N=0.0
END

/* Display them. */
SAY "Input, REAL: IMAG:"
DO N=0 TO 7 BY 1
	SAY "        "||REAL_ARRAY.N||"   "||IMAG_ARRAY.N
END

/* Only constant required, 14 decimal places. */
PI=3.14159265358979 
/* N should be 8 by default in this DEMO. */
/* SAY "Number of elements = "||N||"..." */
N=N-1

CALL FFT

SAY ""
DO N=0 TO 7 BY 1
	SAY "REAL: "||+OUT_REAL.N||",    IMAG: "||+OUT_IMAG.N
END

SAY ""
SAY "Final FFT values to 5 decimal places:"
STR=""
DO N=0 TO 7 BY 1
	CALL ABS_COMPLEX
	NUMERIC DIGITS 14
	IF +ABS <= 0.000000000001
	THEN
		ABS=0
	ENDIF
	NUMERIC DIGITS 6
	STR=STR||+ABS||" "
END
SAY STR
SAY ""
EXIT

/* Create the subroutines required. */
/* FFT subroutine. */
FFT:
DO K=0 TO N BY 1
	SUMREAL=0.0
	SUMIMAG=0.0
	DO T=0 TO N BY 1
		ANGLE=(2.0*PI*T*K)/(N+1)
		SUMREAL=SUMREAL+(REAL_ARRAY.T*COS(ANGLE))+(IMAG_ARRAY.T*SIN(ANGLE))
		SUMIMAG=SUMIMAG-(REAL_ARRAY.T*SIN(ANGLE))+(IMAG_ARRAY.T*COS(ANGLE))
	END
	OUT_REAL.K=SUMREAL
	OUT_IMAG.K=SUMIMAG
END
RETURN
/* FFT subroutine end. */

/* Absolute value of complex number subroutine. */
ABS_COMPLEX:
	ABS=SQRT( (OUT_REAL.N**2)+(OUT_IMAG.N**2) )
RETURN
/* Absolute value of complex number subroutine end. */

People have wanted a good awk fft for years. From this example I'll try and write an awk version. That should help. It has all the facilities needed.

1 Like

awk version:

$ cat fft.awk

# Don't feed it  REAL/IMAG/K/T, those are local variables
function fft(REAL_ARRAY, IMAG_ARRAY, K, T, SUMREAL, SUMIMAG,
        ANGLE, COS_ANGLE, SIN_ANGLE)
{
        for(K=1; K in REAL_ARRAY; K++)
        {
                SUMREAL=SUMIMAG=0;

                for(T=1; T in REAL_ARRAY; T++)
                {
                        ANGLE=(2*3.14159*(T-1)*(K-1))/N;
                        COS_ANGLE=cos(ANGLE);
                        SIN_ANGLE=sin(ANGLE);

                        SUMREAL += REAL_ARRAY[T]*COS_ANGLE + IMAG_ARRAY[T]*SIN_ANGLE;
                        SUMIMAG += -REAL_ARRAY[T]*SIN_ANGLE + IMAG_ARRAY[T]*COS_ANGLE;
                }

                REAL[K]=SUMREAL
                IMAG[K]=SUMIMAG
        }
}

BEGIN {
#        REALSTR="1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5"
#        IMAGSTR="0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0"

        # Split into arrays.  INDEXES 1-16, not 0-15!
        N=split(REALSTR, REAL_ARRAY, " ");
        split(IMAGSTR, IMAG_ARRAY, " ");
        # Convert strings to integers
        for(X in REAL_ARRAY) REAL_ARRAY[X] += 0.0
        for(X in IMAG_ARRAY) IMAG_ARRAY[X] += 0.0
        # N=16    # Hardcoded 16 elements

        fft(REAL_ARRAY, IMAG_ARRAY);

        printf("Real:\n");
        for(X=1; X in REAL; X++)
                printf(" %.2f", REAL[X]);
        printf("\n");

        printf("Imaginary:\n");
        for(X=1; X in IMAG; X++)
                printf(" %.2f", IMAG[X]);
        printf("\n");

}

$ awk -f fft.awk -v N=16 -v REALSTR="1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5" IMAGSTR="0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0" /dev/null


Real:
 8.50 3.64 -1.71 0.05 1.50 -0.17 -0.29 0.48 0.50 0.48 -0.29 -0.17 1.50 0.05 -1.71 3.64
Imaginary:
 0.00 1.93 0.50 0.24 1.00 -0.47 -0.50 1.22 -0.00 -1.22 0.50 0.47 -1.00 -0.24 -0.50 -1.93

$

It is easily dozens to hundreds of times faster from removing 100% of all external calls.

1 Like

Hi Corona688...

I just knew you would create a translation and get it working on awk. ;o)
You could remove the SIN_ANGLE and COS_ANGLE and put sin(ANGLE) and cos(ANGLE) directly into the two lines below them, eliminating two lines...

Thanks for the translation, I will see if I can eliminate the Python version in AudioScope.sh and attach your name to it...

Another thing is that there is no need to issue the imaginary values for this basic FFT. As long as the real values are actually, cropped or padded to power(s) of 2 then the imaginary values of 0.0 can be internally generated to the same array length as the real values, eliminating the IMAGSTR initialisation on the command line call.

But nice to see an awk version that I can steal... <wink>

Another thing you can do to speed it up is keep it running - feed it input and read it output at need, rather than starting and killing it all the time. How to do that depends on what your input data would look like and what you'd want for your output data.

1 Like

There is one error and corrected for the correct results, note the minus sign in red:
SUMIMAG += -REAL_ARRAY[T]*SIN_ANGLE + IMAG_ARRAY[T]*COS_ANGLE;
Correct results:

Real:
 8.50 3.64 -1.71 0.05 1.50 -0.17 -0.29 0.48 0.50 0.48 -0.29 -0.17 1.50 0.05 -1.71 3.64
Imaginary:
 0.00 -1.93 -0.50 -0.24 -1.00 0.47 0.50 -1.22 0.00 1.22 -0.50 -0.47 1.00 0.24 0.50 1.93

Otherwise phenominal.
Thanks a lot...
(This has taught me more about awk than ANY tutorial or info from the mighty WWW and for that I can only say thank you.)

1 Like

Just as a side note - awka is an awk -> C translator

I think it was created out of similar frustrations we have all had with awk performance in major applications. I've tried it with varying success.

May be worth considering:
GitHub - noyesno/awka: Revive awka - AWK to C Conversion Tool

2 Likes

OK a modified version of Corona688's awk translation:

# FFT_Awk.awk
# A modified translation by Corona688, WAY COOL!
# Don't feed it REAL/IMAG/K/T, those are local variables!
function fft(REAL_ARRAY, IMAG_ARRAY, K, T, SUMREAL, SUMIMAG,
        ANGLE)
{
        # PI to 14 decimal places.
        PI=3.14159265358979;
        for(K=1; K in REAL_ARRAY; K++)
        {
                SUMREAL=SUMIMAG=0;
                for(T=1; T in REAL_ARRAY; T++)
                {
                        ANGLE=(2*PI*(T-1)*(K-1))/N;
                        SUMREAL += REAL_ARRAY[T]*cos(ANGLE) + IMAG_ARRAY[T]*sin(ANGLE);
                        SUMIMAG += -REAL_ARRAY[T]*sin(ANGLE) + IMAG_ARRAY[T]*cos(ANGLE);
                }
                REAL[K]=SUMREAL
                IMAG[K]=SUMIMAG
        }
}

BEGIN {
        # REALSTR="1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5"
        # IMAGSTR="0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0"
        # 'IMAGSTR' can be builtin but not done in this DEMO.

        # Split into arrays.  INDEXES 1-16, not 0-15!
        N=split(REALSTR, REAL_ARRAY, " ");
        split(IMAGSTR, IMAG_ARRAY, " ");
        # Convert strings to floats.
        for(X in REAL_ARRAY) REAL_ARRAY[X] += 0.0
        for(X in IMAG_ARRAY) IMAG_ARRAY[X] += 0.0
        # N=16;    # Hardcoded 16 elements WITH '0.5' mid point padding for this DEMO!

        fft(REAL_ARRAY, IMAG_ARRAY);

        printf("Real:\n");
        for(X=1; X in REAL; X++)
                printf(" %.5f", REAL[X]);
        printf("\n\n");

        printf("Imaginary:\n");
        for(X=1; X in IMAG; X++)
                printf(" %.5f", IMAG[X]);
        printf("\n\n");

        printf("Absolute Values:\n");
        for(X=1; X<=N; X++)
		printf(" %.5f", sqrt(((REAL[X])^2)+((IMAG[X])^2)));
        printf("\n");

# awk -f FFT_Awk.awk -v N=16 -v REALSTR="1.0 1.0 1.0 1.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.5 0.5 0.5" IMAGSTR="0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0" /dev/null
#
# Real:
#  8.50000 3.63782 -1.70711 0.05143 1.50000 -0.17275 -0.29289 0.48350 0.50000 0.48350 -0.29289 -0.17275 1.50000 0.05143 -1.70711 3.63782
#
# Imaginary:
#  0.00000 -1.93071 -0.50000 -0.24147 -1.00000 0.46564 0.50000 -1.22361 0.00000 1.22361 -0.50000 -0.46564 1.00000 0.24147 0.50000 1.93071
#
# Absolute Values:
#  8.50000 4.11842 1.77882 0.24688 1.80278 0.49665 0.57947 1.31567 0.50000 1.31567 0.57947 0.49665 1.80278 0.24688 1.77882 4.11842
}

Is this a first?
Could it even pull views to UNIX from search engines?
Who knows but one thing I do know my eventual aim was to do this.
I chose the ksh-awk and ARexx versions earlier in this thread to see the possibilities of FFT on languages not really designed to do it, then to dig deep into awk programming to succeed at this also...
The awk only version, courtesy of Corona688, makes FFT available for all the shells floating point or not.
Again C688, superb, now to ether remove import dependencies for python, (already done a python version), or modify this for the builtin 'SPECAN' in AudioScope.sh...
Lastly, I hope this has been an interesting break from all the usual stuff posted on here.

2 Likes

WOW!
The awk version works on an AMIGA A1200 with HDD and 4MB extra memory using ADE the UNIX _emulator_ for AMIGA OS-3.x running ksh88 on top of a default AMIGA A1200 shell.
Now that is pretty damn good!
Thanks to Corona688 for his original translation.

@Corona688 with your permission can I upload to AMINET for we AMIGA fanatics too?

Aminet - Latest packages, last 14 days

Absolutely. I don't post code for people to not use it.

It's a kind of naive implementation though. Usually better to read values from stdin, but knowing how audioscope works I knew you'd need batches. That'd obscure how it really works a bit though. Maybe better as this, people can populate the arrays as they please.

It's not truly an FFT either, just a DFT -- which gives the same result but takes much more time to do so. Notice how it must do N*N loops for N input. FFT accomplishes the same with N log N loops.

Still, I hope it has its uses. And I'm grateful for you writing one in shell. That's the first DFT I've seen written in a language I really understand.

2 Likes

Yes it is technically a DFT but most people don't know what the difference is and FFT is what they know most, so I used that term. I will upload to AMINET as a DFT however.
And naive it is too, but I could stay with python and use the builtin import cmath version I have got to eliminate the 'scipy' and 'scipyIO' dependencies. I really didn't want to use python although my code works from Python 1.4.0 to the current 3.7.0.
Thanks a lot matey, will post the URL for you to see when I have uploaded it...
Bazza...

------ Post updated 19-09-18 at 06:19 AM ------

Hi Corona688...

Uploaded to the AMINET site at around 9:30pm UK local time last night, 18-09-2018.

At around 6:00am today 19-09-2018 UK local time it was added and has had 2 downloads, dls, already. It will be on show for 14 days and I expect at least 150 dls in that time.

Aminet - dev/gcc/DFT-FFT.awk.txt

Thanks a lot...

Bazza.

(Consider this thread wrapped up.)

I have no idea why it added my separate post to the end of this one...

1 Like

Hi Corona688...

Well the Awk DFT has just dropped out of its 14 days viewing on AMINET and has reached 135 downloads.
Did not quite reach the 150 mark, but hey 135 is not bad for such an ancient platform, running the _ADE_ UNIX emulation for the AMIGA A1200; the Awk version for this platform is about 20 years old.

Aminet - dev/gcc/DFT-FFT.awk.txt

From tiny Acorns big trees grow...
There will be someone out there who will improve considerably on this _seed_ that has now been sown.

Thanks a lot matey...

2 Likes

A "small" machine with 4MB RAM isn't so different from the earlier UNIX machines.

Perhaps, but we have come a long way since the inception of the AMIGA A1200.
It is surprising how difficult it is to create code to work on multiple platforms with large time spans between platform types and differences in languages as they evolve over these time spans.
It took me a long time to do an FFT for Python 1.4.0 for the A1200, to the current, (04-10-2018), 3.7.0 for current machines, but I got there in the end. Everything is easy when code is written around current language versions but backwards compatibility is hard.

My next esoteric code is going to be DWTs, (Discrete Wavelet Transforms), and I think it can be done in pure ksh93, and maybe zsh.
It does look fairly straightforward and this site will be the first to have a basic DWT for ksh93 when done.
Once done I will leave the Awk version for you... <wink>

Depends what you're doing. Hardware I/O changes often, plain old data processing hasn't. awk in particular didn't burden itself with 100,000 obsolete expansion modules like Perl and Python did, or changed as much as shell has; it remains a special-purpose language just for data processing, quite portable. Mostly. You can find it on anything from wireless routers on up, but occasionally we must tell people 'use nawk on solaris' or else they'll get a crusty old version backwards-compatible with 1977.

Python is one of the worst languages for backwards compatibility, its basic syntax changes with every sneeze.

If you're curious, there's a lot more to learn about awk. It's actually a very fast, powerful, and low-footprint language for processing text. (It had to be, to operate on 1977 hardware.) It's not a do-everything language, it has a few really nice features and not a lot else. You wouldn't write a menu system or graphical UI with it. But for data processing it's almost as powerful as perl.

3 Likes