Login using perl LWP module

Shell Programming and Scripting
1

Hi,

Could some one tell me how to login to any web site and get that page using perl LWP. I heard that we can login to the site using LWP.

I dont want to use WWW:Mechanize as I dont have that module installed on the server.

Appreciate your early response.

Thanks...

2

Depends on the web site. What is it?

Some accept simple http://user:pass@hostname/ logins and those are all done the same way. But these days, this is rare.

Others require you to post user and password data into a form, then pass on the resulting cookie to future web requests, which is really annoying to have to do automatically.

3 
#!usr/bin/perl
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;
use HTTP::Cookies;
 
$user = xxxx;
$pass = xxxx;
 
my $ua = LWP::UserAgent->new;
 $ua->cookie_jar(HTTP::Cookies->new(file => "lwpcookies.txt",autosave => 1));
 my $req = HTTP::Request->new(GET => 'http://www.drivehq.com/Secure/LogonOption.aspx?gotoURL=');
 
 $req->authorization_basic($user, $pass);
 
 my $content = $ua->request($req)->as_string;
print "$content\n";

Above is the code I am using. I am trying to access "http://drivehq.com"

4

Obviously I can't test it given I don't have the username and pw but that site is definitely not using basic authentication. They've got a login form on their mainpage like so:

<form name="aspnetForm" method="post" action="index.aspx" onsubmit="javascript:return WebForm_OnSubmit();" id="aspnetForm">
<div>
<input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value="" />
<input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value="" />
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwUJODk3NjczNDQzZBgKBRdjdGwwMCRtdHZNYWluVGFiU..." />
</div>

...

      <div class="logonCont">      
      
                
                <i>Username:</i><input name="ctl00$ContentPlaceHolder1$LogonUserNameTextBox" type="text" maxlength="50" id="ctl00_ContentPlaceHolder1_LogonUserNameTextBox" class="unnamed1" />
...
                <i>Password:</i><input name="ctl00$ContentPlaceHolder1$LogonPasswordTextBox" type="password" maxlength="20" id="ctl00_ContentPlaceHolder1_LogonPasswordTextBox" class="unnamed1" />
...
                <input type="image" name="ctl00$ContentPlaceHolder1$LogonButtons" id="ctl00_ContentPlaceHolder1_LogonButtons" class="BtnLogon" 
...
</form>

I adore the glorious futility of putting hidden elements inside their own <div> :smiley: But anyway. You'll need to POST some data like

ctl00$ContentPlaceHolder1$LogonUserNameTextBox=xxxx&ctl00$ContentPlaceHolder1$LogonPasswordTextBox=xxxx

to http://drivehq.com/index.aspx , and you might need to throw all those hidden variables in along with it. See what I mean by annoying?

5

Thanks for your reply...

Actaully I have to login to this site "http://zip-codes.com" using LWP and after log into the site, need to go to "Database Downloads" link.

Username: Matt123
Password: usermatt

Can you pls paste the code and if possible explanation...

Thanks.....

6

Is this real username and password? If so, please dont post them

7

If you want us to write and test every last scrap of code for you this is the wrong place to look. Services like vworker are plentiful.

As already said: It's a pain. You have to emulate every last scrap of whatever convoluted login system they're using, which may involve hacks like javascript redirects.

This is what I tried and it just doesn't work and only zip-code.com can tell you why.

TMP=`mktemp`
wget --save-cookies "$TMP" --keep-session-cookies -U netscape --referer=http://zip-codes.com/ "http://zip-codes.com/account_login.asp" --post-data="loginUsername=xxxx&loginPassword=xxxx&redir=account_home.asp&Action=Login&Submit=Login"
8 
#!/usr/bin/perl
use HTTP::Request::Common qw(POST);
use LWP;
use HTTP::Cookies;
#use LWP;
$browser = LWP::UserAgent->new();
my $cookie_jar = HTTP::Cookies->new( file=>'cookies.txt', autosave=>1);
my $response = POST 'http://www.zip-codes.com/account_login.asp?msg=10&redir=account_home.asp',
   [ username => 'Matt123', password => 'usermatt' ];
$content = $browser->request($response)->as_string;
#$req = $ua->request($req);
$a = $content;
print "$content" if $content =~ m/Copyright/g;

I am able to login to the "zip-codes.com" site...but unable to search the "Copyright" string..

When i am trying to search "Copyright" string...script is retrieving all the content of the site...

Pls help..

9

I even posted the login form for you to look at, and suggest you go back and do so: The relevant form entries are not "username" and "password".