Use "sed" to copy lines in range

I have list of url in temp.txt as follows:

https://www.example.com/files/AE2345.txt
https://www.example.com/files/AE2347.txt
https://www.example.com/files/AE2350.txt
https://www.example.com/files/AE2369.txt
https://www.example.com/files/AE2387.txt
https://www.example.com/files/AE2388.txt
https://www.example.com/files/AE2392.txt
https://www.example.com/files/AE2440.txt
https://www.example.com/files/AE3111.txt
https://www.example.com/files/AE3123.txt

and so on.

the above list are not in sequence like 1,2,3 ...

I have tried with following sed which works perfectly:

#!/bin/bash
sed '/https:/\/\www.example.com/\files\/AE2350.txt\/,/https://www.example.com/files/AE2440.txt/!d' temp.txt > temp1.txt

output in temp1.txt are as follows:

https://www.example.com/files/AE2350.txt
https://www.example.com/files/AE2369.txt
https://www.example.com/files/AE2387.txt
https://www.example.com/files/AE2388.txt
https://www.example.com/files/AE2392.txt
https://www.example.com/files/AE2440.txt`

I wanted the script to use variables to hold those starting url and end url and use that varibles in "sed" but it does not work.
my script now looks like this:

#!/bin/bash
a="https://www.example.com/files/AE2350.txt"
b="https://www.example.com/files/AE2440.txt"

cat temp.txt | sed '/$a/,/$b/!d' > temp1.txt

output file temp1.txt shows empty.

Am I missing something in sed?

Please help fix the syntax.

Thanks

Hi
What for? It is enough to make out the digital values

a=2350
b=2440
sed '\%https://www\.example\.com/files/AE'"$a"'\.txt%,\%https://www\.example\.com/files/AE'"$b"'\.txt%!d'
a='https://www.example.com/files/AE2350.txt'
b='https://www.example.com/files/AE2440.txt'
sed '\%'$a'%,\%'$b'%!d' file

In the last version, I can’t escape points through variables :thinking:

I was mistaken, everything is ok

a='https://www\.example\.com/files/AE'
b='2350\.txt'
c='2440\.txt'
sed '\%'$a$b'%,\%'$a$c'%!d' file
1 Like

Made an unordered list of URL

$ cat temp.txt
https://www.example.com/files/AE2347.txt
https://www.example.com/files/AE2369.txt
https://www.example.com/files/AE2350.txt
https://www.example.com/files/AE2387.txt
https://www.example.com/files/AE2345.txt
https://www.example.com/files/AE2388.txt
https://www.example.com/files/AE2392.txt
https://www.example.com/files/AE2440.txt
https://www.example.com/files/AE3111.txt
https://www.example.com/files/AE3123.txt

Sorting is important for you to achieve your objective

a=2350
b=2388
sort -t '/' -k 5,5nr temp.txt | sed '\%https://www\.example\.com/files/AE'"$a"'\.txt%,\%https://www\.example\.com/files/AE'"$b"'\.txt%!d'

https://www.example.com/files/AE2350.txt
https://www.example.com/files/AE2369.txt
https://www.example.com/files/AE2387.txt
https://www.example.com/files/AE2388.txt
sort -t '/' -k5,5nr --debug file
...
https://www.example.com/files/AE2350.txt
                              ^ no match for key

sort -t\/ -k5,5
or
sort -t\/ -k5.3,5.6n

Exactly, thanks for pointing out @nezabudka

1 Like

Thanks for the solution but unfortunately the output are showing different. when I run the above said sed the result which I get are as follows:

https://www.example.com/files/AE2350.txt (it is $a)
https://www.example.com/files/AE2351.txt  (this line does not present in source file but added after running sed)
https://www.example.com/files/AE2352.txt  (this line does not present in source file but added after running sed)
...
https://www.example.com/files/AE2369.txt (this URL present in source file)
https://www.example.com/files/AE2370.txt (this line does not present in source file but added after running sed)
https://www.example.com/files/AE2371.txt  (this line does not present in source file but added after running sed)
...
https://www.example.com/files/AE2387.txt (this URL present in source file)
https://www.example.com/files/AE2388.txt(it is $b)

the above result shows that, sed is adding additional sequence of files between $a and $b.
I just want the sed to copy exact same url which is present between $a and $a. Like

https://www.example.com/files/AE2350.txt (this is $a)
https://www.example.com/files/AE2369.txt
https://www.example.com/files/AE2387.txt
https://www.example.com/files/AE2388.txt
https://www.example.com/files/AE2392.txt
https://www.example.com/files/AE2440.txt(this is $b)

I tried this too but the result are showing same like I described in "Mannu25251" reply.

awk -F'/AE|\\.txt' '$2>=2350 && $2<=2440' file

Find the difference

?

sed '/2350/,/2440/!d' file

I haven’t changed anything in your algorithm. Just put the template into a variable

The sed does not have variables, so you fight with escaping of the RE special characters.
Not so with shell builtins. Create a speaking function, and the main program is quite simple:

#!/bin/bash
function sift_out(){
  local begin=$1 end=$2
  local line prt=0
  while read line
  do
    case $line in (*"$begin"*) prt=1;; esac
    [ $prt -eq 1 ] && printf "%s\n" "$line"
    case $line in (*"$end"*) prt=0;; esac
  done
}

a="https://protect2.fireeye.com/v1/url?k=4ae50b7e-1445ebe0-4ae54be5-86ee86bd5107-93ba947c1d9f7dbf&q=1&e=cc7ff68e-e191-4790-a487-47afb74e7fab&u=https%3A%2F%2Fwww.exampl.com%2Ffiles%2FAE2350.txt"
b="https://protect2.fireeye.com/v1/url?k=98cce59c-c66c0502-98cca507-86ee86bd5107-b23623b84f1ae9bf&q=1&e=cc7ff68e-e191-4790-a487-47afb74e7fab&u=https%3A%2F%2Fwww.exampl.com%2Ffiles%2FAE2440.txt"
sift_out "$a" "$b" <temp.txt
3 Likes

I got the point. now my last question:

will the above sed work for the url like https://secure.example.co.in/srv-01/2009/files/111/AE2392.txt? sorry for asking this because I just wanted this script to be work in any kind of URLs.
and another question is why \ is not been used around "files" like \%https://www\.example\.com/\files/\AE'"$a"'\.txt%

Sorry, I don't know English very well. I would be grateful if someone would help or will you give an example as you see this

LESS='+/\\c' man sed

what I mean to say is.. if I change the web URL from sed '\%https://www\.example\.com/files/AE'"$a"'\.txt% to sed '\%https://secure\.example\.co\.in/srv-01/2009/files/111/AE'"$a"'\.txt%
will it still work? or do I have to change the sed pattern?

my another question was, in sed you have sed '\%https://www\.example\.com/files/AE'"$a"'\.txt%, ..., so why \ is not used with around "files" like sed '\%https://www\.example\.com\/files\/AE'"$a"'\.txt%,....
maybe my question is wrong but I am just curious to know.

Certainly.
For this, it seems to me that this script is best suited

In addition, if you do not specify the anchor of the end of line $ in the template, you can simplify the template a little without damage

a='https://www\.example\.com/files/AE'
b='2350'
c='2440'
sed '\%'$a$b'%,\%'$a$c'%!d' file

run in terminal

LESS='+/\\c' man sed

I applied a special design so as not to escape direct slashes. Instead /pattern/
\%pattern% or \|pattern|

I also strongly advise you to pay attention for the post from @MadeInGermany
Because passing variables to sed is essentially a bad practice

ok

Thank you so much for keeping me on right track. Finally my script could work.

I also thank Mannu25251 and MadeInGermany

You guys are great! :heart_eyes: