Python Script Issues

Applications Programming
1

Gents,

This is a follow up to my previous post:

Here the latest function that I am having issues with:

def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        #startpos = link.find("lpk4=") + 5
        #endpos   = link.find("&amp")
        vidnum   = re.findall("(?<==)\d{5,7}", link, re.U)
        vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

    for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
            flvs.append(hashval_url.text)

    return flvs

in that I cant understand why when manually looking at my truple

flvs = []

it should have all the urls that were scraped such as:

[u'http://blah.blah.com/secure/courses/94343/f4v_H264/94343_00_01_SC03_welcome.f4v?hashval=1368124621_35861c278ff5c700797b0bf8fc04c554', u'http://blah.blah.com/secure/courses/94343/f4v_H264/94343_00_01_SC03_welcome.f4v?hashval=1368124623_67da0c443fe1b220332f1bd9e50e5d6c'

it only adding a single url to the truple:

[u'http://blah.blah.com/secure/courses/94343/f4v_H264/94343_00_01_SC03_welcome.f4v?hashval=1368124621_35861c278ff5c700797b0bf8fc04c554'

I have a deep feeling it has to do with my list reference:

vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

Someone please make me feel stupid.

2

Hi,

Try printing out vidum, then proceed from there.

vidnum   = re.findall("(?<==)\d{5,7}", link, re.U) 
print vidnum
3

thank you for the reply. It actually turned out to be my nested "for" loop that appeared to be the issue. So from

def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        #startpos = link.find("lpk4=") + 5
        #endpos   = link.find("&amp")
        vidnum   = re.findall("(?<==)\d{5,7}", link, re.U)
        vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

    for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
            flvs.append(hashval_url.text)

    return flvs

to

def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        #startpos = link.find("lpk4=") + 5
        #endpos   = link.find("&amp")
        vidnum   = re.findall("(?<==)\d{5,7}", link, re.U)
        vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

         for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
            flvs.append(hashval_url.text)

    return flvs