Formatting wget request within script

Shell Programming and Scripting
1

When using a browser and calling this url .. the data returns the proper range of information

ichart dot finance dot yahoo dot com/table.csv?s=YAHOO&a=3&b=14&c=2012&d=03&e=20&f=2012&g=d&ignore.csv

(geeze wont let me post url's sorry )

However in my script the formatting is messing up on this request and the range returns incorrect not sure of correct syntax. Please Help

while read s1 ; do
url=http://ichart.finance.yahoo.com/table.csv?s=
s="&a=3&b=14&c=2012&d=03&e=20&f=2012&g=d&ignore.csv"
  wget -a wget.log $url$s1$s ;
  sleep 5;
  cat header.txt > $s1.txt;
  chmod 777 $s1.txt;
  sed '1d' table.csv?s\=$s1 >> $s1.txt;
rm -Rf table*
done < nasdaqsymbollist
2

Try this. The sub-shell to run 'cat' and 'wget | sed' will allow all the output to redirect to $s1.txt

while read s1
do
    url=http://ichart.finance.yahoo.com/table.csv?s=
    s="&a=3&b=14&c=2012&d=03&e=20&f=2012&g=d&ignore.csv"
    (cat header.txt; wget -O - --quiet $url$s1$s | sed 1d) > $s1.txt
    sleep 5
done < nasdaqsymbollist
3

No need for a subshells , you can redirect the loop:

while read sl
do 
  ....
  cat header.txt
  wget   ...    "$url$s1$s" | ...
done < nasdaqsymbollist > "$s1.txt"

and it is better to use double quotes
---
*edit* oops this is incorrect you can't redirect to the variable in the loop of course...
---

4

Hi Scrutinizer,

I don't think your proposed redirection will work "$s1.txt" will be interpreted by shell before doing the read and this will just resulted in ".txt". Here is my little test.

$ cat in
x
y
z

$ while read s1
do
  cat /etc/hosts
  uname -a
done < in > "$s1.txt"

$ ls -lart
total 16
-rw-r--r--  1 user user    6 2012-05-02 08:15 in
drwxrwxrwt 14 root root 4096 2012-05-02 08:17 ..
drwxr-xr-x  2 user user 4096 2012-05-02 08:19 .
-rw-r--r--  1 user user  960 2012-05-02 08:19 .txt
1 Like
5

Hi Chihung, yes you are right, that was just silly :rolleyes: