SED Search in YouTube file not working.

Shell Programming and Scripting
1

Hi

I've succesfully downloaded a HTML file from youtube (for a specific video id)
I'm trying to use the SED command to search for double and single quotes and convert them to a line break (new line).
However i'm getting end of file errors when I run this script, can someone help?

I've included the whole script below, with comments for each line.

#!/bin/bash

echo ENTER THE YOUTUBE URL TO COPY:
read VIDEOID
#The above 2 lines read in the full URL link from the end user.
echo VIDEO ID TO  DOWNLOAD IS $VIDEOID #This line echos back to the user, what he just entered.
#YOURLINK=$FILENAME
file="youtube.html"
wget -q -O - $VIDEOID > $file #gets the file from youtube.
vidid=`cat $file | grep "fmt_url_map" | sed "s/'/\n"  | sed "s/"/\n" > youtube2.html
# this line is not working first grep is a search for the text within the file, second 2 sed's should convert single and double spaces to a new line.
2

Hi,

to download youtube-videos from the commandline use cclive.

Your line:

vidid=`cat $file | grep "fmt_url_map" | sed "s/'/\n"  | sed "s/"/\n" > youtube2.html

can be simplified to:

vidid=$(sed -n "/fmt_url_map/{s/[\'\"]/\n/g;p}"  $file > youtube2.html)

HTH Chris

3 
# sed -n '/fmt_url_map/p'  $file | sed -e 's/"/\n/g' -e "s/'/\n/g" > youtube2.html
4

Hi

Thanks for your response, it worked, and broke the quotes up for me.
The below is what I ultimatley want to end up, i've tried to add the cache line, but it broke the search, can u have a look please?

Search for "fmt_url_map"
Break new lines at single-quotes, double-quotes and pipe (DONE)
grep for http addresses with the string "cache" (Added but not working)
Get the last such line. This is the http address of the actual video, which can be downloaded with wget.

Thanks

5

if you can give me sample input file and desired output , i can help you :slight_smile:

6

Hi Ygemici

Basically I start by logging onto YouTube, find a video clip i want, then copy the URL of that video clip such as
YouTube - RC-Heli-Action: Tag der offenen T�r bei Vario
I then download this through a wget to my PC.
What I want to do is then search the downloaded HTML file.

Search for "fmt_url_map"
Break new lines at single-quotes, double-quotes and pipe.
grep for http addresses with the string "cache"
Get the last such line. This is the http address of the actual video, which can be downloaded with wget.
End Result should be a downloaded FLV file of the choosen video clip from YouTube.

C.

7

hi

i download youtube video page url as html but i cant find "flv" in source code..
flv informations should be located server side and not client side..
but i write something :slight_smile: but i cant find flv address :frowning:

# sed -n '/fmt_url_map/p'  YouTube\ -\ RC-Heli-Action\ Tag\ der\ offenen\ T�r\ bei\ Vario.htm | sed -e 's/"/\n/g' -e "s/'/\n/g" -e 's:|:/n:g' | grep  "cache" | sed -n -e '$p' |sed 's/.*\(http.*\)http.*/\1/'
 
http://v23.lscache1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dWTVZPVl9FSkNNNl9OSVpB&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1278648000&key=yt1&signature=2C7FF8A961208A9536EDC5F7BDE3B90937A51BEE.80DEA95F5B5DF092C5A3CDA92BA599CDC73814C8&factor=1.25&id=fba0731f5dc7af9b/n/ntc.v23.cache1.c.youtube.com,5/n

# sed -n '/fmt_url_map/p'  YouTube\ -\ RC-Heli-Action\ Tag\ der\ offenen\ T�r\ bei\ Vario.htm | sed -e 's/"/\n/g' -e "s/'/\n/g" -e 's:|:/n:g' | grep  "video_id" | sed -e 's/.*\(http.*\)http.*/\1/' -e '$d'
 
http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dv1RmD8lnDUc&rv.7.url=
http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dv1RmD8lnDUc&rv.7.url=
8

HI Guys

If I wanted to find the last instance of the word 'cache' in a HTML file, would the below syntax work ? (bash script)
tail -f grep "cache"