Multiple variables using awk and for loop for web form submission

Shell Programming and Scripting
1

Hi

My goal is to fill an HTML form and submit.

What I have managed to do:

  1. curl command to fill up the form and submit
  2. a file which has the input

curl command:

curl -v -b cookie.txt -d __CSRFToken__=dc23d5da47953b3b390ec68d972af10380908b14 -d do=create -d a=open -d email=test@test.net -d name=fullname -d source=Phone -d topicId=12 -d 198855e3047ff8a0=issuesummary1 -d message=details  http://localhost:8000/input.php

The input file looks like this;

57582	CVE-2011-3389	6.4	Medium	x.x.x.x	tcp	443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	14			5	2	3months	0:00	s3	6
57582	CVE-2011-3389	6.4	Medium	x.x.x.x	tcp	8443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	4			3	2	3months	0:00	s3	6
58751	CVE-2011-3389	4.3	High	x.x.x.x	tcp	8443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	5			5	3	2months	0:00	s3	3
65821	CVE-2013-2566	4.3	Medium	x.x.x.x	tcp	8443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	6			6	2	3months	0:00	s3	6
65821	CVE-2015-2808	4.3	Critical	x.x.x.x	tcp	8443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	78			7	4	1month	0:00	s3	5
78479	CVE-2014-3566	4.3	Medium	x.x.x.x	tcp	8443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	5			7	2	3months	0:00	s3	6
80035	CVE-2014-8730	4.3	Medium	x.x.x.x	tcp	8443	Bad vulnerability, fix ti	Full Name	email@xxxxxx.net	3			7	2	3months	0:00	s3	6

So every line of input will be assigned to the curl variable data (-d) for example:

curl -v -b cookie.txt -d __CSRFToken__=dc23d5da47953b3b390ec68d972af10380908b14 -d do=create -d a=open -d email=email@xxxxxx.net -d name=Full Name -d source=2months -d topicId=4 -d 198855e3047ff8a0=8443 -d message=Bad vulnerability, fix ti  http://localhost:8000/input.php

I was thinking

for i in $(awk -F "\t" '//{print $0}' inputfile)
do
"the curl command"
done

but I know it wont work.

I think I should assign the awk output to an array and use for loop to iterate the array. But how do i parse each field and then each line in the input file?

Please help. Thanks.

2

What be your OS and shell?

3

With bash , this snippet - although being far from complete and error proof - could point you in the right direction:

OLDIFS="$IFS"
IFS=$'\t'
while read LINE
  do    CUARR=($LINE)
        echo curl  -v -b cookie.txt -d __CSRFToken__=dc23d5da47953b3b390ec68d972af10380908b14 -d do=create -d a=open -d email=${CUARR[9]} -d name="'${CUARR[8]}'" -d source=${CUARR[13]} ...
  done < file
IFS="$OLDIFS"

Please complete to taste. The echo is there to prevent curl from running while testing and searching the correct array elements.

2 Likes
4

In bash, if you read straight into an array you do not need to set and unset IFS globally, so RudiC's suggestion modified, becomes:

while IFS=$'\t' read -a CU
do
  curl  -v -b cookie.txt -d __CSRFToken__=dc23d5da47953b3b390ec68d972af10380908b14 -d do=create -d a=open -d "email=${CU[9]}" -d "name='${CU[8]}'" -d "source=${CU[13]}" ...
done < file
2 Likes
5

Thank you to both of you.

Now, I am having problem when there is an empty item in the input. The subsequent item in the array does not take the empty item which causes the output of the script to be not accurate. For example the second line of input below has empty second item so the array ${CUARR[1]} will be the third item "6.4" instead of empty. How to make ${CUARR[1]} stays empty?

Thanks.

35291   CVE-2004-2761   4       Medium  xxxxxxxxxxxxxxx tcp     443     SSL Certificate Signed Using Weak Hashing Algorithm     14      4       2       12/8/2016       0:00    s3      6
51192           6.4     Medium  xxxxxxxxxxxxxxx tcp     443     SSL Certificate Cannot Be Trusted       14      4       2       24/9/2016       0:00    s3      6
6

Yes this is a problem for bash . If you have either ksh93 or zsh on your system (if not you can install it) then you can do this:

while IFS=$'\t\t' read -A CU
do
  curl  -v -b cookie.txt -d __CSRFToken__=dc23d5da47953b3b390ec68d972af10380908b14 -d do=create -d a=open -d "email=${CU[9]}" -d "name='${CU[8]}'" -d "source=${CU[13]}" ...
done < file
1 Like
7

Now comes the full beauty of the proposal in post#3 into play: if you insert

LINE=${LINE//$'\t\t'/$'\t'x$'\t'};

right after the do , it might fly...

1 Like
8

@RudiC, what if there are two consecutive empty fields?

9

You're pushing the limits, no? Admittedly that's been a band aid. Use awk or equivalent?

10

Thank you again.

Mr Scrutinizer codes have this error using Bash on Debian:

read: -A: invalid option
read: usage: read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [-t timeout] [-u fd] [name ...]

Mr RudiC codes work so far.

11

It was said that it would not work with bash, but with ksh93 or zsh

1 Like