Using sed to repeat pattern

Shell Programming and Scripting
1

I want to create a script that performs a mass download using formatted URLs:

https://example.com/macro?rev=11
 https://example.com/script?rev=18

Assuming all software is delivered as a zip file, I want to turn this into:

 wget https://example.com/macro?rev=11 -O macro-11.zip
 wget https://example.com/script?rev=18 -O script-18.zip

If all URLs are given in a plain text file url.txt, how would I add the output file specification by repeating file name and revision number using sed?

sed -e 's/^/wget /' -e 's/$/ -O /' url.data
2

You need capture groups and their references.
Knowing the & catch-all you can have
sed -E 's/.*\/(.*)\?rev=([0-9]+)/wget & -o \1-\2\.zip/' url.data
Not knowing it you would need a 3rd capture group
sed -E '(s/.*\/(.*)\?rev=([0-9]+))/wget \1 -o \2-\3\.zip/' url.data
The capture groups are numbered 1 2 3 at their opening (
This ist extended RE, ERE.
Basic RE, BRE is
sed 's/.*\/\(.*\)?rev=\([0-9]\{1,\}\)/wget & -o \1-\2\.zip/' url.data
where GNU sed allows
sed 's/.*\/\(.*\)?rev=\([0-9]\+\)/wget & -o \1-\2\.zip/' url.data
For ASCII-incompatible locales (languages, do they exist at all?) you better replace [0-9] by [[:digit:]]
The .* (zero or more characters) are a bit tricky, because they are greedy and want to catch as much as possible. And the left .* is greedier than the following one.

3

Hi @technossomy,

assuming that your input file has one entry url zip per line, you could also use xargs:

xargs -n2 sh -c 'wget $0 -O $1' < infile
4

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.