I have a script which uses cli curl to download the source code of a webpage and then tests if a specific string exists in the source.
The problem is that the website has a slow response, so the eval expression hasn't completed when the test starts.
The test returns a negative, and the curl completes right after the test and changes the source.html. The test should have been positive, but ran too soon.
How do I make the test wait until curl has finished? Do I have to sleep for a few seconds, or is there another way?
string=$(echo "curl -d "'"'$datastring'"'" -b cookie.txt -c cookie.txt -L "$url)
eval $string > source.html
teststring=$(awk '/text/' source.html)
The curl progress line is still showing after the test has been completed.
---------- Post updated at 11:54 PM ---------- Previous update was at 11:50 PM ----------
If I put
> source.html
into the string it still doesn't work. The script has already completed when I receive the source a few seconds later.
string=$(echo "curl -d "'"'$datastring'"'" -b cookie.txt -c cookie.txt -L "$url" > source.html")
eval $string
teststring=$(awk '/text/' source.html)
Perhaps the URL contains a ampersand (&) character causing the curl to be run in the background.
Why do you need to do an eval instead of invoking curl directly?
For debuging purposes try printing the command before you eval it :
string=$(echo "curl -d "'"'$datastring'"'" -b cookie.txt -c cookie.txt -L "$url)
printf "%s\n" "$string"
eval $string > source.html
teststring=$(awk '/text/' source.html)
1 Like
I have to pass variables like post data to curl, and I don't know any other way than constructing a string and then eval it.
---------- Post updated at 12:14 AM ---------- Previous update was at 12:10 AM ----------
But yes, it does contain an ampersand. Does that make it run in the background?
So is it just $datastring that contains $variables that you need to expand?
Edit: Yes, the expanded $url variable will need to be quoted to ensure the shell dosn't intrepret anything within it. Preferablly with single quotes.
Try:
string=$(echo "curl -d "'"'$datastring'"'" -b cookie.txt -c cookie.txt -L "\'$url\')
1 Like
yes, just $datastring
Now I have found a solution with your help. I put double quotes around the url, to prevent it from running in the background. 
string=$(echo "curl -d "'"'$datastring'"'" -b cookie.txt -c cookie.txt -L "\'$url\')
eval $string > source.html
teststring=$(awk '/text/' source.html)
---------- Post updated at 12:21 AM ---------- Previous update was at 12:18 AM ----------
By the way, do you or anyone else know an easier way to pass variables to curl?
Double quotes aren't so good if URL contains a dollar sign ($), the shell will try and expand it.
1 Like
Try:
curl -d "$(eval echo \"$datestring\")" -b cookie.txt -c cookie.txt -L "$url"
1 Like
that's a bit better than my way