Feed python with read line in shell

Hello,
I am running under ubuntu 14.04.
I have a python file bringing requested data. When I run, it asks me to enter query info. After that, it gives the data if anything matched can be found.

What I need to accomplish is, instead of entering the requested data from keyboard, to put all query keywords into a file and to run the python inside the shell script.
What I tested:

#!/bin/bash
while read -r line
do
python test.py
sleep 2 #enter what you read from keywords file 2seconds later
echo "$inputline" #this command is not working as a part of `python test.py` command
###sed -i "/^echo.*mpu_opps$/s/\"/$inputline\"/2 /system/etc/99oc #
done<keywords

Thank you
Boris

I have no idea what you are trying to do.

You do three things every time you read a line from a file:

  1. run a python script that reads no input,
  2. sleep for two seconds, and
  3. echo the contents of a variable that has never been defined to the standard output of your script.

In a more normal sequence of operations, one might expect to see something more like:

python test.py < keywords

or, if the data in keywords needs to be massaged in some way before being passed on to python :

while read -r line
do	# do whatever needs to be done to $line before passing it on to python
	... ... ...
	# pass on the modified line to python
	printf '%s\n' "$line"
done < keywords | python test.py
1 Like

Thank you Don,
Works perfect. I was not hoping that pipe can pass the output of the previous previous process like this way.

while read -r line
do	# do whatever needs to be done to $line before passing it on to python
	... ... ...
	# pass on the modified line to python
	printf '%s\n' "$line"
done < keywords | python test.py

Thank you again
Boris

What previous process?

Did you try using:

python test.py < keywords

like I suggested in post #2? If so, did it work? If not, in what way did it fail?

If the above works and you have some other process that is creating the contents of the file named keywords , then piping the output of that command directly into python test.py is highly likely to work if that process writes the data that is stored into keywords to its standard output.

With very few exceptions, if the command sequence:

producer > temp_file
consumer < temp_file

works, then those two commands be replaced by:

producer | consumer

and get the same results. Or, if you need to keep the data that is being passed between producer and consumer , you can use:

producer | tee temp_file | consumer

and often still get faster results that running the two processes synchronously.

1 Like

Thank you Don,
I am trying to gather imdb details of some movies.
Regarding, your first answer, it works for the first line of keywords file but no problem..
Regarding your next answer, python test.py > keywords also reads only first line but no problem.
I can solve it.

PS: Changed like this way:

count_raw=$(wc -l < keywords)
for (( i=1; i<=$count_raw; i++ ))
do
python test.py < keywords
sed -i '1d' keywords
done<keyword

If I do not delete active line after the process, script reads only one line and stops. That's why I used sed
Thank you so much
Boris

That still doesn't sound quite right, manipulating the input file while "reading" from it. Why should python test.py read just one line from a file, and the stop reading? What happens if you run it interactively and supply input from the keyboard? Does it take several keywords separated by <new line> chars, i.e. individual lines, or do you have to run it once for every single keyword you enter?

Hello Rudic,
Yes, Python code works for just one entry.

python test.py

keywords

Irma La Douce
As Good As It Gets
Shawshank Redemption
..
..

Now, in shell, it gives like this:

./test.sh

Output:

{"rating": "7.4", "genres": ["Comedy", "Romance"], "description": "Irma la Douce is a movie starring Jack Lemmon, S$
{"rating": "7.7", "genres": ["Comedy", "Drama", "Romance"], "description": "As Good as It Gets is a movie starring $
{"rating": "9.3", "genres": "Drama", "description": "The Shawshank Redemption is a movie starring Tim Robbins, Morg$

Now it's nice with your helps.

Thank you
Boris

Frankly speaking, baris35,

The information you are pulling from that movie review site is a JSON object.

The great majority of web developers process JSON objects with JSON object processing tools.

For example, this is tutorial for parsing JSON when using Python:

Python JSON

Reference: Parse JSON - Convert from JSON to Python

import json

# some JSON:
x =  '{ "name":"John", "age":30, "city":"New York"}'

# parse x:
y = json.loads(x)

# the result is a Python dictionary:
print(y["age"])
1 Like

And here is a more complete Python reference for processing JSON, in general:

Current version as of today:

json - JSON encoder and decoder - Python 3.7.3 documentation

If you need only one keyword per script run, try (given your shell provides "here strings")

while read KW; do python test.py  <<< "$KW"; done < keywords
1 Like

I prefer, generally, Javascript to Python, but FYI, here is an example Python program to read a JSON object via a URL using urllib2 and simplejson .

There are many ways to to this, this is only one quick example:

import urllib2
import simplejson
 
response = urllib2.urlopen("https://myurl/custom/get/")
data = simplejson.load(response)
print data
# => {'content': 'Hello World!', 'success': True}
 

If you want to get the value of a specific key in the JSON object:

import urllib2
import simplejson
 
response = urllib2.urlopen("https://myurl/custom/get/")
data = simplejson.load(response)
print data["content"]
# => Hello World!

These kind of web-based JSON operations are nearly trivial to do in Python and Javascript.

1 Like

Thank you Rudic & Neo,

while read KW; do python test.py  <<< "$KW"; done < keywords

It's very good. One line and that's it :slight_smile:

Dear Neo,
I need more time to read and understand.

Thank you
Boris