scp Commands not executing via cgi script

Hi,

I have a CGI script, which includes a python custom header file. The cgi script calls a few functions.

Py file.

#resourcemanager creation

def make_rm(rmip,nip,nport):
    fp = open("temp1.txt",mode="w")
    fp.write('<?xml version="1.0"?>\n<!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.-->\n\n<configuration>\n\n<!-- Site specific YARN configuration properties -->\n\n<property>\n<name>yarn.resourcemanager.resource-tracker.address</name>\n<value>'+rmip+':8025</value>\n</property>\n\n<property>\n<name>yarn.resourcemanager.scheduler.address</name>\n<value>'+rmip+':8030</value>\n</property>\n\n</configuration>')
    fp.close()
    commands.getstatusoutput("sshpass -p redhat scp temp1.txt "+rmip+":/hadoop2/etc/hadoop/yarn-site.xml")

    fp = open("temp2.txt",mode="w")
    fp.write('<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?>\n<!--  Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file.-->\n\n<!-- Put site-specific property overrides in this file. -->\n\n <configuration>\n\n<property>\n <name>fs.default.name</name>\n<value>'+nip+':' +nport+ '</value>\n</property>\n</configuration>\n')
    fp.close()
    commands.getstatusoutput("sshpass -p redhat scp temp2.txt "+rmip+":/hadoop2/etc/hadoop/core-site.xml")
    # print("Resourcemanager setup.\n")

    commands.getstatusoutput("sshpass -p redhat ssh -o StrictHostKeyChecking=no root@"+rmip+" yarn-daemon.sh start resourcemanager")

    a = commands.getstatusoutput("sshpass -p redhat ssh -o StrictHostKeyChecking=no root@"+rmip+" /usr/java/jdk1.7.0_79/bin/jps | grep 'ResourceManager' | awk '{print $2}'")

    if a[1] == "ResourceManager":
        print("ResourceManager setup complete!")

Now, when i call this function throught the cgi script via the browser, the scp command doesn't work. But it works fine if i run the cgi script through the terminal as " python autoip.cgi ". This executes the scp command and everything works well.

I would appreciate any help available on this.

This might be a permissions issue. Remember that the webserver account (often Apache) will be the one that calls your code. What do you have in the error logs?

Can you run any cgi-script through your webserver, e.g. just a simple Hello display. That way we can check that you have the code in the right place too.

Robin

I'm not sure about that. It runs all the other commands in the function apart from the one that involves scp.

Yes, I can run cgi-script through the browser. It does give me an output, but it doesn't copy the temp file into the appropriate location on the other VM. So i can't setup my cluster.

The cgi-sript if it helps:

#!/usr/bin/python

import cgi
import cgitb
import os
import commands
import base2
cgitb.enable()

print "content-type:text/html"
print ""

dn_iplist=[]

base2.ipscanner();
base2.freemem();
base2.procno();
base2.freehd();

dn_iplist = sorted(base2.spacedict, key=lambda x: float(base2.spacedict[x][:-1]), reverse=True)[:2]

masterdict = base2.memdict.copy()

for key in dn_iplist:
	if key in masterdict: 	
		del masterdict[key]

#print masterdict

#####################Resource Manager IP
rmip = max(masterdict, key=lambda key: int(masterdict[key]))
del masterdict [rmip]
#####################Namenode IP
nnip = max(masterdict, key=lambda key: int(masterdict[key]))
del masterdict[nnip]
#####################Client IP
cip = max(masterdict, key=lambda key: int(masterdict[key]))

print """
<html>
<head>
	<title>Hadoop Cluster Services - Create and Manage your Hadoop clusters</title>
	<link rel="stylesheet" type="text/css" href="/css/style1.css">
</head>

<body>
		<br><b>
		<div class="drop">
		<ul class="drop_menu">
			<li><a href="home.html">HOME</a></li>
			<li><a href="create.cgi">CLUSTER</a>
				<ul>
					<li><a href="/loader.html">Create Cluster</a></li>
					<li><a href="#">Manage Cluster</a></li>	
				</ul>
			</li>
			<li><a href="#">SERVICES</a>
				<ul>
					<li><a href="#">Shell</a></li>
					<li><a href="#">Create Cluster</a></li>	
					<li><a href="#">System Admin</a></li>	
				</ul>
			</li>
		</ul>	
		</div>

	<br><br><br><br><br>
	<div id="autoips">
		Resource Manager IP: """ 
print rmip
print "<br><br>Namenode IP: "
nport = "9001"
print nnip + ":" + nport 
print "<br><br>Client IP: "
print cip
print "<br><br>Datanodes/NodeManagers: "
for k in dn_iplist:
	print k + ","
#print """<br><br><input type="submit" value="Accept"> """

print "<br><br><br> BUILDING CLUSTER"

print"""		
	
	</div>

</body>

</html>
"""

base2.make_rm(rmip,nnip,nport)
print "<br>"
base2.make_nn(nnip,nport)
print "<br>"
base2.make_client(nnip,nport,rmip,cip)
print "<br>"
for dnip in dn_iplist:
	print dnip	
	base2.make_dnnm(nnip,nport,rmip,dnip)
	print "<br>"

I've attached a screenshot of the output.
The last line should be similar to the one above it. But it isn't cuz the file isn't copied. The one above displays that because it still has the files from a previous manual execution.