I am trying to use wget to make a local copy of this website
accuscore.com/fantasy-sports/nfl-fantasy-sports/Current-Week-DEF-ST (i have the http:// in front, but the forum will not allow me to put it in at this time)
Whenever i try to use wget i receive an "Error 404: Not found". I tried the referer with no luck.
If anyone can help I would appreciate it.
Probably due to dynamic webpage generation. As far as I know, wget is not able to retrieve such webpages.
It's probably to disallow automatic downloading. wget doesn't hide the fact that it's a batch downloader. Try giving it a different user-agent with -U, and make accuscore.com the referer with --referer.
@fpmurphy do you know of anything to convert a dynamic page to static and store it locally? The reason i want to store this locally is because 1) the site causes timeouts with some scripts i have 2) I do no want to overload or be the cause of extra traffic on the server because of my scripts
@Corona688 I added the user agent, but already had a referer. It still fails. My query looks like this:
wget --directory-prefix=/Users/problemss/Desktop --proxy=off -Q0 --user-agent=Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) --passive-ftp --header=REFERER:http://accuscore.com -k -r -l2 --progress=dot:binary http://accuscore.com/fantasy-sports/nfl-fantasy-sports/Current-Week-DEF-ST
The response is:
Resolving accuscore.com... 184.106.172.20
Connecting to accuscore.com|184.106.172.20|:80... connected.
HTTP request sent, awaiting response... 404 NOT FOUND
2012-08-16 14:13:25 ERROR 404: NOT FOUND.
Do you have curl?
curl http://accuscore.com/fantasy-sports/nfl-fantasy-sports/Current-Week-DEF-ST > page.html
But you won't get recursive download this way...