Hi there,
I'm trying to extract automatically opening hours from a website.
The page displaying the schedules is
with xxx going from 101 to 174
I managed to get the following output :
le lundi de 10.30 � 19.30
le mardi de 9.30 � 19.30
blank
le jeudi de 9.30 � 19.30
le vendredi de 9.30 � 19.30
le samedi de 10.30 � 21.30
blank
There is one line per weekday (from monday to sunday)
blank is an actual blank line (not displaying anything)
How can I now get the final output:
"10:30 19:30|09:30 19:30| |09:30 19:30|09:30 19:30|10:30 21:30| "
Thanks for your help.
Santiago
awk '{ printf("%s\t%s%s",$4,$6,"|")}' input_file.txt
awk -F'[^0-9]+' '{b=b (NR>1?"|":"")($2?sprintf("%02d:%02d %02d:%02d",$2,$3,$4,$5):"")}END{print b}' yourfile.txt
If you really want 11 spaces instead of a blank field, then:
awk -F'[^0-9]+' '{b=b (NR>1?"|":"")($2?sprintf("%02d:%02d %02d:%02d",$2,$3,$4,$5):sprintf("%11s",""))}END{print b}' yourfile.txt
if you have Python, an almost full alternative solution
#!/usr/bin/env python
import urllib2,re
pat=re.compile(""".*<span class="tdBlancBold">(.*)<div align="center">.*""",re.M|re.DOTALL)
days=['lundi','mardi','mercredi','jeudi','vendredi','samedi','dimanche']
url="http://www.natureetdecouvertes.com/pages/gener/view_FO_STORE_corgen.asp?mag_cod=%s"
for num in range(101,174):
page=urllib2.urlopen(url % str(num))
data=page.read()
if not "Impossible" in data:
result = pat.findall(data)
store={}
for i in result:
for j in i.split("<br>"):
j=j.strip()
if j.startswith("le"):
j=j.split()
if j[1] in days:
t1,t2=j[-3],j[-1]
store.setdefault(j[1],[])
store[j[1]].extend([t1,t2])
for DAY in days:
try:
print "%s |" %( ' '.join(store[DAY])),
except:
print "\t\t|",
print ""
else:
print "Page not found ",url % str(num)
extract of output :
# python test.py
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 19.00 |
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | |
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | | 10.00 21.00 | 10.00 20.00 | |
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | |
10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | |
10.00 21.00 | 10.00 21.00 | 10.00 21.00 | 10.00 21.00 | 10.00 21.00 | 10.00 20.00 | |
10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | |
10.00 20.00 | 10.00 21.00 | 10.00 21.00 | 10.00 21.00 | 10.00 21.00 | 10.00 20.00 | |
9.30 19.30 | 9.30 19.30 | 9.30 19.30 | 9.30 19.30 | 9.30 19.30 | 9.30 19.30 | |
Page not found http://www.natureetdecouvertes.com/pages/gener/view_FO_STORE_corgen.asp?mag_cod=110
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | |
10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | 10.00 19.30 | |
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | |
10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | 10.00 20.00 | |
9.30 19.30 | 9.30 19.30 | 9.30 19.30 | 9.30 19.30 | 9.30 19.30 | 9.30 19.30 | |
Thanks everyone for your help,
It looks that all stores display their timetable using different format but I came up with the following solution that matches perfectly all needs :
wget -qO- 'http://www.natureetdecouvertes.com/pages/gener/view_FO_STORE_corgen.asp?mag_cod=118' |
sed -rn 's/\r//g; s/<br>//; /^[[:space:]]*([Ll]e )?[Ll]undi/,/([Ll]e )?[Dd]imanche/ { ; /^[[:space:]]*$/!p }' |
sed -r 's/^[^0-9]*//; s/.*clipse totale.*/ /; s/ h /h/g; s/(�|de|-) //; s/\.|h/:/g; s/^9:/09:/; s/:( |$)/:00\1/g' |
sed ':a; N; $!b a; s/\n/|/g')'