Ok. But how do I get my server program to run on the apache server? How do I go about this?
I think you misunderstand my suggestion. Simply install and configure Apache. Once you have it up and running and serving static pages, you can add a simple CGI script to accept custom POST queries from your clients. If that's fast enough for you, you're done. If it's too slow, look into normal CGI optimization techniques. Only if that fails, try to write any server code of your own.
Another thing: Before you delve into that, do check whether the simple forking server in man perlipc is fast enough for you. Apache comes with a tool called apachebench (ab) which lets you stress-test servers under extreme loads.
Era,
Thanks for all your suggestions. But I am a little confused at the end of all this, I am sorry. Let me try explaining my real problem again.
I will be running processes simultaneously on MANY machines. Each of these processes has an output that I want to look at, at one central location. I cant have them write to seperate files in one location to prevent load on the file server. So what do you think is the best way to achieve this?
Regards,
garric
I don't think I can tell you anything you don't already know, but let's review the situation.
You want a client/server architecture: a server accepts incoming connections from the clients, and stores the information they submit when they connect. The server can be a simple program which merely accepts incoming connections, reads whatever the clients submit, and stores it to a file, then disconnects. Similarly, the clients can simply open a connection, dump a report, and disconnect.
The sample code you find in the perlipc manual page can do this, but if you have performance demands, it might not be enough. Then it's probably better to go look for a server architecture which is designed for heavy concurrency. Apache has a good reputation in this regard, although you can look for others, too. You'll just need to figure out a way to make it do what you want. A simple CGI script is a good way to make Apache do something different than just serve up static web pages.
Are you following so far? So my suggestions at this point would be to test the "simple forking server" you find in man perlipc and measure whether it performs well enough. If it doesn't, see if you can make Apache suit your needs.
Ok thanks. So looking at my needs, I have decided that making Apache suit our needs is the way to go. Now, I needed help on exactly how to go about this? If you lead me to specific docs on the same, I will try taking this forward.
Regards,
garric
Do you have basic familiarity with CGI? Write a simple CGI script to accept a POST from a client.
Yes, I am familiar with the basics. But all my CGI programming has been related to the web. How do I pass and accept parameters when there is no HTML involved?
Regards,
garric
Pretend you have a web form, and that the clients have downloaded it and filled it in. Now what you will be receiving from them is the POST action from that form. If it helps, actually code up a real form and play around with it; but the clients will not need the actual form, they just need to know how to submit the final POST query.
I get this so far. But how can I submit the final post query without the submit button.
And how will my CGI script deal with multiple client connections?
Regards,
garric
The server takes care of any concurrency issues. And the POST query can basically be submitted by a telnet to port 80 of the server, there is no HTML involved any longer at this point.
It should be revealing to create a simple form with a POST action to http://localhost:4343/ and submit it from your browser, while running netcat -l -p 4343 on the same computer. Now imagine you create a simple client which just submits the same bytes and then disconnects. No browser, no HTML, no form, just a TCP connection to port 80 (or 4343, or where-ever).
I understand what you are trying to explain..
I will give you an example of how my client code shud look like.. Let me know if this is what you were trying to tell
#!/usr/bin/perl
use IO::Socket;
if (@ARGV < 1) { print "usage: perl send_to_server.pl [host]\n"; exit; }
$host = $ARGV[0];
$var = IO::Socket::INET->new( PeerAddr=>"$host",
PeerPort=>"80",
Proto=>"tcp") || die "Connection failed.";
print $var "POST /cgi-bin/server_code.pl?data=abc";
Yes, that's exactly the idea (although that's not really a valid HTTP transaction, nor a useful POST -- but if a GET form is good enough for you, then by all means use that).
A valid GET would be
print $var "GET /cgi-bin/server_code.pl?data=abc HTTP/1.0\r\n\r\n";
Another thing came to mind. What if your clients would simply send an email; would that work? That would leave even less for you to reinvent.
Thats looks like an interesting idea 
But I will finish working on the one you helped me so much, first.
So if my client code is this simple, will my server code just look like this
$client_data=param(data);
open a file
print $client_data
close a file
My concern now is to handle the concurrent writes into a common area. How do I tackle that?
Regards,
garric
A few possible strategies:
- Create a unique file for each connection, mktemp or similar, maybe key by IP address and/or date
- Use a proper database back end
- Just write to standard error and it will end up in Apache's log file, which you can post-process periodically (very simple, very stupid ...)
This problem is the reason I came up with the email idea, actually (-:
era,
This is what I wrote
client side
#!/usr/bin/perl
use CGI;
use IO::Socket;
if (@ARGV < 1) { print "usage: perl send_to_server.pl [host]\n"; exit; }
$host = $ARGV[0];
$var = IO::Socket::INET->new( PeerAddr=>"$host",
PeerPort=>"80",
Proto=>"tcp") || die " $! Connection failed.";
print $var "GET server_code.pl?data=abc HTTP/1.0\r\n\r\n";
Server side
#!/usr/bin/perl
open (FL,"/tmp/garric");
print FL "abcdefgh";
close (FL);
Then I run the client code like this
#>./client_code.pl <web server>
This should create a file on the webserver running apache, right? But it doesnt seem to be doing it. Can you see why?
Regards,
garric
You are opening a file for reading, not writing; and not checking the exit code. You want open (FL, ">/tmp/garric") or die "Anguish: $!";
Before I noticed that, I wrote the following. I'm leaving it just in case it might come in useful ...
How are you invoking the server-side script? You have it in (your moral equivalent of) /var/www/cgi-bin/server_code.pl and it's marked executable? You have Perl available on the box? No chroot or other trickery?
You don't really need to use CGI in the client script if you intend to keep it that simple.
Oh, and another quick tip: run the server-side thingy from the command line before you try to run it from within the server context. Nothing beats finding a syntax error only after several hours of figuring out what "500 Internal Server Error" really means ...
The reason I did not write a "die " on fail is because this is going to run on the web server and since I am running the client code on another machine, I wont be able to see the standard output there (or std err)
Yes, the server code does have executable permissions.
How do I exactly go about doing that? I mean checking the server code on the command line.