Python (seleniumrequests)Class, Constructor, Method Not Working

Applications Programming
1

Newbie question. I created a class:

class WP(seleniumrequests.PhantomJS):
    
                def __init__(self, wpurl='https://blah.org/download/release-archive/', bwppurl='https://blah.org/plugins/browse/popular/'):
                                self.wp=wpurl
                                self.bwpp=bwppurl
                def wpscaper(self):
                                wpjfetch=seleniumrequests.PhantomJS('../ms-tuesday/phantomjs/bin/phantomjs')
                                wpjfetch.get(self.wp)
                                wpsoup=BeautifulSoup(wpjfetch.page_source, 'lxml')
                def wppscaper(self):
                                wppjfetch=seleniumrequests.PhantomJS('../ms-tuesday/phantomjs/bin/phantomjs')
                                wppjfetch.get(self.bwpp)
                                wppsoup=BeautifulSoup(wpjfetch.page_source, 'lxml')

I am trying to understand why I cannot see any of my methods in where I would expect it to be:

goodies=WP()
goodies.wpscaper()
goodies.wpscaper.wpjfetch.page_source
goodies.wpscaper.wpsoup

What I see is:

goodies.wpscaper.im_class  goodies.wpscaper.im_func   goodies.wpscaper.im_self

and the same applies for:

goodies=WP()
goodies.wppscaper()
goodies.wppscaper.im_class  goodies.wppscaper.im_func   goodies.wppscaper.im_self

What gives ??

---------- Post updated 08-26-15 at 03:35 PM ---------- Previous update was 08-25-15 at 05:37 PM ----------

I can see that the stuff works. For example, if I add a print statement to the functions such as:

class WP(seleniumrequests.PhantomJS):
    
                def __init__(self, wpurl='https://blah.org/download/release-archive/', bwppurl='https://blah.org/plugins/browse/popular/'):
                                self.wp=wpurl
                                self.bwpp=bwppurl
                def wpscaper(self):
                                wpjfetch=seleniumrequests.PhantomJS('../ms-tuesday/phantomjs/bin/phantomjs')
                                wpjfetch.get(self.wp)
                                wpsoup=BeautifulSoup(wpjfetch.page_source, 'lxml')
                                print (wpsoup)
                def wppscaper(self):
                                wppjfetch=seleniumrequests.PhantomJS('../ms-tuesday/phantomjs/bin/phantomjs')
                                wppjfetch.get(self.bwpp)
                                wppsoup=BeautifulSoup(wpjfetch.page_source, 'lxml')
                                print (wppsoup)

it prints exactly whats in wpsoup or wppsoup:

goodies=WP()
goodies.wpscaper()
<!DOCTYPE html>
<html dir="ltr" lang="en" xmlns="http://www.w3.org/1999/xhtml"><head profile="http://gmpg.org/xfn/11">
<meta charset="utf-8"/>
<!--
<meta property="fb:page_id" content="6427302910" />
-->
<meta content="7VWES_-rcHBcmaQis9mSYamPfNwE03f4vyTj4pfuAw0" name="google-site-verification"/>
blah blah blah blah..
goodies.wppscaper()
<!DOCTYPE html>
<html dir="ltr" lang="en" xmlns="http://www.w3.org/1999/xhtml"><head profile="http://gmpg.org/xfn/11">
<meta charset="utf-8"/>
<!--
<meta property="fb:page_id" content="6427302910" />
-->
<meta content="7VWES_-rcHBcmaQis9mSYamPfNwE03f4vyTj4pfuAw0" name="google-site-verification"/>
blah blah blah blah..

Additonal Info:

In [180]: dir(goodies.wpscaper)
Out[180]: 
['__call__',
 '__class__',
 '__cmp__',
 '__delattr__',
 '__doc__',
 '__format__',
 '__func__',
 '__get__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__self__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 'im_class',
 'im_func',
 'im_self']
In [181]: dir(goodies.wppscaper)
Out[181]: 
['__call__',
 '__class__',
 '__cmp__',
 '__delattr__',
 '__doc__',
 '__format__',
 '__func__',
 '__get__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__self__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 'im_class',
 'im_func',
 'im_self']

So I simple cant understand why I dont have access to:

goodies.wpscaper.wpjfetch
goodies.wpscaper.wpsoup

or

goodies.wppscaper.wppjfetch
goodies.wppscaper.wppsoup

why cant I see them?? I am truly confused.

2

Python does not work as such.

In relatation to it:

wpjfetch=seleniumrequests.PhantomJS('../ms-tuesday/phantomjs/bin/phantomjs')

wpjfetch is a local binding to whatever object is returned from calling the method seleniumrequests.PhantomJS() and it is out of scope as soon as the goodies.wpscaper() method is finished.
You can return it inside the method, or you can make it an instance class attribute and then you can access to it as goodies.wppjfetch , but that breaks OOP encapsulation.

3

awesome explanation. When you say return back to inside the method, can you show me an example? What would I return it back too?

4 
$ cat model.py
class ModelNumber(object):

    def serial(self):
        num = "X12631"
        return num

Now in the REPL

>>> from model import *
>>> product = ModelNumber()
>>> type(product)
<class 'model.ModelNumber'>
>>> type(product.serial)
<type 'instancemethod'>
>>> product.serial()
'X12631'
>>>

However, that's not very useful. The following might be a bit more interesting.

$ cat model.py
class ModelNumber(object):

    def __init__(self, num):
        self.num = num

    def serial(self):
        return self.num

    def change_serial(self, num):
        self.num = num

In the REPL

>>>
>>>
>>> from model import *
>>>
>>> product = ModelNumber("X12631")
>>> product.serial()
'X12631'
>>> product.change_serial("X12632")
>>> product.serial()
'X12632'
>>> product.num
'X12632'
>>>

As you see, product.num is accessible, which is nice for debugging purposes, but it should be manipulated and handled by a method in the class.

5

Aia, I still cant unhide the methods under wppscaper or wpscaper with any return or restructuring this in anyway:

class WP(seleniumrequests.PhantomJS):
        
                      def __init__(self):
                                                        self.wpurl='https://blah.org/download/release-archive/'
                                                        self.wppurl='https://blah.org/plugins/browse/popular/'
                      def wpscaper(self):
                                                        wpinit=seleniumrequests.PhantomJS(executable_path='phantomjs/bin/phantomjs')
                                                        wpinit.get(self.wpurl)
                                                        wpsoup=BeautifulSoup(wpinit.page_source, 'lxml')
                                                        print (wpsoup)
                                                        return wpsoup
                      def wppscaper(self):
                                                        wppinit=seleniumrequests.PhantomJS(executable_path = "phantomjs/bin/phantomjs")
                                                        wppinit.get(self.wppurl)
                                                        wppsoup=BeautifulSoup(wppinit.page_source, 'lxml')
                                                        print (wppsoup)
                                                        return wppsoup

In [85]: dir(WP)
Out[85]: 
['__class__',
 '__delattr__',
 '__dict__',
 '__doc__',
 '__format__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__module__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 '_unwrap_value',
 '_wrap_value',
 'add_cookie',
 'application_cache',
 'back',
 'close',
 'create_web_element',
 'current_url',
 'current_window_handle',
 'delete_all_cookies',
 'delete_cookie',
 'desired_capabilities',
 'execute',
 'execute_async_script',
 'execute_script',
 'file_detector',
 'find_element',
 'find_element_by_class_name',
 'find_element_by_css_selector',
 'find_element_by_id',
 'find_element_by_link_text',
 'find_element_by_name',
 'find_element_by_partial_link_text',
 'find_element_by_tag_name',
 'find_element_by_xpath',
 'find_elements',
 'find_elements_by_class_name',
 'find_elements_by_css_selector',
 'find_elements_by_id',
 'find_elements_by_link_text',
 'find_elements_by_name',
 'find_elements_by_partial_link_text',
 'find_elements_by_tag_name',
 'find_elements_by_xpath',
 'forward',
 'get',
 'get_cookie',
 'get_cookies',
 'get_log',
 'get_screenshot_as_base64',
 'get_screenshot_as_file',
 'get_screenshot_as_png',
 'get_window_position',
 'get_window_size',
 'implicitly_wait',
 'log_types',
 'maximize_window',
 'mobile',
 'name',
 'orientation',
 'page_source',
 'quit',
 'refresh',
 'request',
 'save_screenshot',
 'set_page_load_timeout',
 'set_script_timeout',
 'set_window_position',
 'set_window_size',
 'start_client',
 'start_session',
 'stop_client',
 'switch_to',
 'switch_to_active_element',
 'switch_to_alert',
 'switch_to_default_content',
 'switch_to_frame',
 'switch_to_window',
 'title',
 'window_handles',
 'wppscaper',
 'wpscaper']

In [86]: dir(WP.wpscaper)
Out[86]: 
['__call__',
 '__class__',
 '__cmp__',
 '__delattr__',
 '__doc__',
 '__format__',
 '__func__',
 '__get__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__self__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 'im_class',
 'im_func',
 'im_self']

In [87]: dir(WP.wppscaper)
Out[87]: 
['__call__',
 '__class__',
 '__cmp__',
 '__delattr__',
 '__doc__',
 '__format__',
 '__func__',
 '__get__',
 '__getattribute__',
 '__hash__',
 '__init__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__self__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 'im_class',
 'im_func',
 'im_self']

I think I understand code encapulation but in this instance I dont want it but for the life of me I cannot understand why I can see my methods and or anything under wpscaper and or wppscaper ? also why cannot I see wpsoup and or wppsoup under wpscaper or wppscaper? Please shed further light upon me:)

6

Since wpsoup is still local to wpscaper() the only way of access is to bind a name to it, outside the class.

wp_instance = WP()
result = wp_instance.wpscaper()
print(result)

you can make wpsoup a class instance attribute and then it can be manipulated as part of the class object.

def wpscaper(self):
    wpinit=seleniumrequests.PhantomJS(executable_path='phantomjs/bin/phantomjs')
    wpinit.get(self.wpurl)
    self.wpsoup=BeautifulSoup(wpinit.page_source, 'lxml')

Which will allow you to allow you to access the attribute directly for debugging.

wp_instance = WP()
wp_instance.wpscaper()
print(wp_instance.wpsoup)

Note that wpsoup does not exist until there's a call to wp_instance.wpscaper() , since it is not coded to be created in the __init__()