i have to point out that i'm a avid shell script lover and i have next to zero interest in perl which is why i dont know the first thing about it. however, just need to do this one thing on it..
i need to do something in perl but dont know how. I have a list of email addresses in the following format = firstname_lastname@company.com
now, i need to parse these emailaddresses to a webpage where i will search for the firstnames and lastnames of each user to see if they exist on the database of that webpage. its sorta like writing a perl script to automatically pull up the www.google.com page and search for first names and last names of any given people.
The search or request parameters are as follows,
Request parameters
search = phone
search text = first and last name
someone suggested i use the below but i have no clue how to do it.:
Please rephrase your question, say with some illustration. I am not totally sure the exact question you have.
LWP is okay, but more complicated with its API. If you are just interested in simply access an HTTP URL and get the results, LWP::Simple is a simpler choice.
ok. i have a list of names (first and last name) that i need to automatically search for on a site called, for instance, www.finestar.com.
on this site called finestar.com are search boxes where you can input the first name or last name of the person your looking for.
now, the list of names i have is quite enormous so manually searching for them on this site is virtually impossible or basically impractical.
enter perl
so i need to use perl to parse the list of names into the search boxes on the site called www.finestar.com and pull out any reference to the names being searched for.
Then, first of all you need to somehow reverse engineer the search form on the target site you are trying to connect to. Does it use POST or GET for the form submission? And investigate how the form is actually submitted, such as the parameters and the like, and whether any cookies may be required.
Actually for this, you can get a lot of help by using a packet sniffer to capture the HTTP request messages to the server. You simply use LWP to generate similar HTTP messages, and voila, just capture the response and parse it yourself. I suppose you know how to and what to parse yourself.
If you just read the documentation for LWP set of classes, you will see something similar to what I post below. In fact, the example below is nearly identical to one of the examples quoted in the manual:
#!/usr/bin/perl -w
use strict;
use LWP;
my $site = 'http://localhost/~bernardchan/test.php';
my $req = HTTP::Request->new(POST => $site);
$req->content_type('application/x-www-form-urlencoded');
$req->content('lastname=Green&firstname=John');
my $http = LWP::UserAgent->new();
my $res = $http->request($req);
if ($res->is_success) {
my $body = $res->content;
printf STDERR ("POST to %s successful\nContent goes below:\n", $site);
print $body;
} else {
printf STDERR ("ERROR: Cannot POST to %s (Code: %d)\n", $site, $res->code);
}
[bernardchan@allan shm]$ perl -Mstrict -w testlwp.pl
POST to http://127.0.0.1/~bernardchan/test.php successful
Content goes below:
<html>
<body>
<h1>Search Results</h1>
<p>This is a demo only.</p>
</body>
</html>
[bernardchan@allan shm]$ perl -Mstrict -w testlwp.pl
ERROR: Cannot POST to http://lfgjhdfkjhgldfjhgdfkg.com/~bernardchan/test.php (Code: 500)
Well, I think I need not clarify further, that mass POSTing for data mining is prohibited on most sites and you may be subject to bans by the system administrators should they find out. And it's just easy to find out, because everything is logged and I don't think any real sysadmin is dumb enough not to read the server logs.