Curl Loop help

ETC Homework & Coursework Questions
1
  1. The problem statement, all variables and given/known data:
    Here is my assignment.
    Parse the /home/access_log and /home/access_.xz files
    Create a sorted list of ip address with no duplicates
    Pass each ip address to ipinfo.io with curl
    Create a logfile that has the ip address,lat,long
    Save the data file and script to the inclass dir
    Submit your script to canvas email_first_part_lab8.txt

I am trying to figure out how to create a loop to cat all the access logs then curl all the ips in that log to the site ive been provided and then redirect that ouput to a text file. I am still rather ignorant to shell scripting.

  1. The attempts at a solution (include all code and scripts):

Here are my scripts to cat the files

cat /home/access_log | awk '{ print $1 }' | sort | uniq -c | sort -n

xzcat /home/access_log-20160917.xz | awk '{ print $1 }' | sort | uniq -c | sort -n

xzcat /home/access_log-20161005.xz | awk '{ print $1 }' | sort | uniq -c | sort -n

And my sample curl with a single ip

curl --header "X-Forwarded-For: 137.226.113.7" ipinfoio

(cant post urls yet)

  1. Modesto junior college, Manetca (CA), US, dale Phillips, and CSCI-210 (Link to Course):

Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).

2
  1. To loop over the files matching a particular pattern in a given directory, you can use the for loop construct. For e.g.,
for file in *.log
do
    sort file | do_something_else >> newfile ## Append output to newfile. >> is the redirection operator (performs an append)
done
  1. Also utilities like sort, awk (well, technically awk is not a utility, it's a programming language) can take a filename as argument. So it's a useless use of cat.

Let us know your progress and where you're stuck.

1 Like