Batch Remove Line From Files Recursively

Good day,

I have no experience in BASH scripting, so I'm hoping you all can help me find a solution to my problem. I've never been able to wrap my head around this stuff, so I really need help from the guru's.

I use UnRAID as a NAS (Linux based), and it has a useful plugin called "User Scripts" which let you easily run BASH scipts on a schedule. I'm using this plugin to have simple copy and remove scripts for my media server, which lets me move content in/out of my media server dynamically, like adding Christmas movies into the media server December 1st, and delete them on December 31st. Works great, with one issue...

The first time I add a movie, it creates an ".nfo" file and writes metadata which includes a "date added" tag. If I delete the file and recopy it again, it uses the original ".nfo" file and thus, the original date added. This keeps it from showing as "recently added".

In a nutshell, I need a bash script that I can run periodically, which will do the following.

1. Locate every file in the "All Movies" folder ending in ".nfo" recursively through all subfolders...
2. Open each ".nfo" and remove the line containing

<dateadded>SOMEDATETIME</dateadded>

3. Resave the modified ".nfo" file...

Any help would be greatly appreciated.

Thanks!

A sample .nfo file - before and after - would be helpful.
Anything you tried on your own, @HouseOfCards ?
Might want to start with man find.

@HouseOfCards , welcome, we hope you find the forum helpful and friendly.

firstly, we don't as a rule write solutions without you collaborating .
the task you describe is a trival one, and with a little effort on your part achievable .

what 'linux' distribution is being used ?

as for examples, a search online 'finding files in linux' will bring back 000's of examples.

additionally, please provide an example .nfo would be appreciated.

if there is 1 .nfo filename for each movie then why not simply delete the movie and .nfo file at the same time, why do you need to remove a line , if you 'recopy' it surely the .nfo file will be recreated (or am I missing some detail?)

thks

Hi,

Thanks for the help. I've always had a hard time learning this stuff without seeing the thing in action, and the "options" in a search are overwhelming. I'm more the type who learns from seeing something in action, and I branch out from there. I'm not a guy who can read a book on bash scripts and take away much, is what I'm saying.

But here is the context. I have two servers set up. One strictly for EVERYTHING, and I use that server to curate metadata, such as descriptions, artwork, etc... This server is like a staging area where all the media resides, and I edit details the way I want them. This process creates an ".nfo" file with the information... but it adds its "date added" line to the ".nfo"...

My UnRAID script will copy a movie's folder to the second production server which is only showing SELECTED content on a schedule, and since the curation was done, the correct artwork and metadata are just the way I want the data to be presented. But because of the way the timestamp is automatically added, it uses the date added to the ORIGINAL staging server. This means that I can't change that date on each copy, and if I delete the ".nfo" file, I lose the other customization.

So I need that "date added" line stripped from all the files every so often, so it re-adds just that snippet when I add to the other server. Here is an example ".nfo" file... I can't add attachment, so it's copied inline... I'm not sure if the date added line is always in the same spot in each file.

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<movie>
  <plot><![CDATA[For decades, next-door neighbors and former friends John and Max have feuded, trading insults and wicked pranks. When an attractive widow moves in nearby, their bad blood erupts into a high-stakes rivalry full of naughty jokes and adolescent hijinks.]]></plot>
  <outline><![CDATA[The best of enemies until something came between them.]]></outline>
  <lockdata>false</lockdata>
  <dateadded>2023-04-12 20:30:43</dateadded>
  <title>Grumpy Old Men</title>
  <originaltitle>Grumpy Old Men</originaltitle>
  <actor>
    <name>Jack Lemmon</name>
    <role>John Gustafson</role>
    <type>Actor</type>
    <tmdbid>3151</tmdbid>
    <tvdbid>253869</tvdbid>
  </actor>
  <actor>
    <name>Walter Matthau</name>
    <role>Max Goldman</role>
    <type>Actor</type>
    <tmdbid>6837</tmdbid>
    <tvdbid>255031</tvdbid>
  </actor>
  <actor>
    <name>Ann-Margret</name>
    <role>Ariel Truax</role>
    <type>Actor</type>
    <tmdbid>13567</tmdbid>
    <tvdbid>254337</tvdbid>
  </actor>
  <actor>
    <name>Burgess Meredith</name>
    <role>Grandpa Gustafson</role>
    <type>Actor</type>
    <tmdbid>16523</tmdbid>
    <tvdbid>257416</tvdbid>
  </actor>
  <actor>
    <name>Daryl Hannah</name>
    <role>Melanie</role>
    <type>Actor</type>
    <tmdbid>589</tmdbid>
    <tvdbid>351037</tvdbid>
  </actor>
  <actor>
    <name>Kevin Pollak</name>
    <role>Jacob Goldman</role>
    <type>Actor</type>
    <tmdbid>7166</tmdbid>
    <tvdbid>253764</tvdbid>
  </actor>
  <actor>
    <name>Ossie Davis</name>
    <role>Chuck</role>
    <type>Actor</type>
    <tmdbid>15531</tmdbid>
    <tvdbid>257376</tvdbid>
  </actor>
  <actor>
    <name>Buck Henry</name>
    <role>Snyder</role>
    <type>Actor</type>
    <tmdbid>7795</tmdbid>
    <tvdbid>254654</tvdbid>
  </actor>
  <actor>
    <name>Christopher McDonald</name>
    <role>Mike</role>
    <type>Actor</type>
    <tmdbid>4443</tmdbid>
    <tvdbid>296566</tvdbid>
  </actor>
  <actor>
    <name>John Carroll Lynch</name>
    <role>Moving Man</role>
    <type>Actor</type>
    <tmdbid>3911</tmdbid>
    <tvdbid>279111</tvdbid>
  </actor>
  <actor>
    <name>Steve Cochran</name>
    <role>Weatherman</role>
    <type>Actor</type>
    <tmdbid>1994261</tmdbid>
    <tvdbid>9096821</tvdbid>
  </actor>
  <actor>
    <name>Joe Howard</name>
    <role>Pharmacist</role>
    <type>Actor</type>
    <tmdbid>82581</tmdbid>
    <tvdbid>252430</tvdbid>
  </actor>
  <actor>
    <name>Isabell O'Connor</name>
    <role>Nurse</role>
    <type>Actor</type>
    <tmdbid>94627</tmdbid>
    <tvdbid>333953</tvdbid>
  </actor>
  <actor>
    <name>Charles Brin</name>
    <role>Fisherman</role>
    <type>Actor</type>
    <tmdbid>1994264</tmdbid>
  </actor>
  <actor>
    <name>Oliver Osterberg</name>
    <role>Fisherman</role>
    <type>Actor</type>
    <tmdbid>1574263</tmdbid>
  </actor>
  <director tmdbid="18281" tvdbid="258047">Donald Petrie</director>
  <trailer>plugin://plugin.video.youtube/?action=play_video&amp;videoid=2A-SOVQvPyg</trailer>
  <trailer>plugin://plugin.video.youtube/?action=play_video&amp;videoid=i_WM-YknHaU</trailer>
  <trailer>plugin://plugin.video.youtube/?action=play_video&amp;videoid=64MH2i5gr_I</trailer>
  <rating>7</rating>
  <year>1993</year>
  <sorttitle>Grumpy Old Men</sorttitle>
  <mpaa>PG-13</mpaa>
  <imdbid>tt0107050</imdbid>
  <tvdbid>4342</tvdbid>
  <tmdbid>11520</tmdbid>
  <premiered>1993-12-25</premiered>
  <releasedate>1993-12-25</releasedate>
  <criticrating>64</criticrating>
  <runtime>103</runtime>
  <tagline>The best of enemies until something came between them.</tagline>
  <country>United States of America</country>
  <genre>Comedy</genre>
  <studio>John Davis</studio>
  <studio>Lancaster Gate</studio>
  <studio>Warner Bros. Pictures</studio>
  <uniqueid type="tmdb">11520</uniqueid>
  <uniqueid type="imdb">tt0107050</uniqueid>
  <uniqueid type="tvdb">4342</uniqueid>
  <id>tt0107050</id>
  <fileinfo>
    <streamdetails>
      <video>
        <codec>h264</codec>
        <micodec>h264</micodec>
        <bitrate>4498696</bitrate>
        <width>1920</width>
        <height>1080</height>
        <aspect>16:9</aspect>
        <aspectratio>16:9</aspectratio>
        <framerate>23.976025</framerate>
        <language>und</language>
        <scantype>progressive</scantype>
        <default>True</default>
        <forced>False</forced>
        <duration>103</duration>
        <durationinseconds>6208</durationinseconds>
      </video>
      <audio>
        <codec>aac</codec>
        <micodec>aac</micodec>
        <bitrate>160104</bitrate>
        <language>eng</language>
        <scantype>progressive</scantype>
        <channels>2</channels>
        <samplingrate>48000</samplingrate>
        <default>True</default>
        <forced>False</forced>
      </audio>
      <embeddedimage>
        <codec>mjpeg</codec>
        <micodec>mjpeg</micodec>
        <width>151</width>
        <height>227</height>
        <aspect>151:227</aspect>
        <aspectratio>151:227</aspectratio>
        <framerate>90000</framerate>
        <scantype>progressive</scantype>
        <default>False</default>
        <forced>False</forced>
      </embeddedimage>
    </streamdetails>
  </fileinfo>
</movie>

As for the Linux version, it's UnRAID with kernel 5.19.17-UNRAID from UNAME command.

Let me know if you have more questions. I'll look into the FIND command, but this is something which can cause havoc if I do it wrong. That's why I need some help.

Thanks again...

So this should find all the ".nfo" files and pass them to an external program? Where from here?

find /mnt/user/"All Movies" -name “*.nfo” -type f -exec 

@HouseOfCards , kudo's for coming back with something.

next step is to confirm that the file list returned is the ones you expect, that check can either be exhaustive or a few diligent spot checks - up to you.

if/when you are confident its time to choose an option on how to make the appropriate amendment. below are 3 potential solutions. make some copies of actual files and then test each of them against those , pick any of them. the first one does not handle case differences so any variant of 'dateadded' wrt to upper/lower case will not work 'Dateadded' for example is a no no.
the 2nd (ex editor) and 3rd (sed editor) will work for mixed case 'DateAddED' is fine.
However, YOU need to test these and be confident the result is as expected, a simple diff original copy will suffice.

# using the traditional *nix editor ed
cat ed.cmds 
/<dateadded>/d
wq

# excute using 
ed <(ed.cmds) fileToProcess.nfo

# catering for case insensitive matches using the ex editor
cat ex.cmds 
1
set ic
/<DateAddeD/d
wq

# execute using
ex <(ex.cmds) fileToProcess.nfo

# case insensitive using sed editor
sed -i '/<dateADDED>/Id'  

# execute using
sed -i '/<dateADDED>/Id'  fileToProcess.nfo

PS: As I don't know your flavour of *nix is why I've shown more than 1 potential solution

Test on an example file:

sed "/<dateadded>/d" example.nfo

Redirect it to a new file and compare

sed "/<dateadded>/d" example.nfo >newfile
diff example.nfo newfile

Looks good? Then consider an "inplace" change on the original file(s), via sed -i

find /mnt/user/"All Movies" -name “*.nfo” -type f -exec echo sed -i "/<dateadded>/d" {} +

The echo only lists the following command(s)
Looks good? Then omit the echo to really run the sed command(s).

Okay, thanks for the direction. I was sidetracked for a bit. I'll give this some attention and post back.

So this command, if I specify the file name, works perfectly...

sed -i "/<dateadded>/d" "Original File.nfo"

However, and perhaps it's a syntax issue, when I execute this, no files get changed...

find /home/username/Desktop/Testing -name “*.nfo” -type f -exec sed -i "/<dateadded>/d" {} +

In the specified "testing" folder, I've added numerous items with varying file names ending in ".nfo" some in subdirectories below it. What I need from here is to have the command start in "Testing", and in-place modify all the ".nfo" files recursively from there.

Is there something missing from the command I'm using?

Thanks!

Hmm, your command is correct, should work exactly like that.
Have a -print before the -exec ... to also print the files.

ditto, this works for me .

@HouseOfCards , can you provide (a sample of the output generated by ) the command below.

find /home/username/Desktop/Testing -name “*.nfo” -type f -print -exec grep -i  "<dateadded>" {} +

Well, just like with either command, there is no output. At the prompt, I execute that command, and it brings me right back to a prompt. No errors, no output, no nadda.

It's as though it runs, but it doesn't do anything. I get no errors about commands not found, or anything. Just a shell prompt.

I'm testing on MX Linux right now...

Is your "find" the correct one?

which find
type find

Hi @HouseOfCards,

your soft quotes are not correct, see the difference:

$ echo '""' | xxd
00000000: 2222 0a                                  "".
$ echo '“”' | xxd
00000000: e280 9ce2 809d 0a                        .......

So find tries to find files named “*.nfo”, including the asterisk and the quotes.

@HouseOfCards , hmm, if the 'find...-print -exec grep -i ....' doesn't produce any output then there are no files matching the "*.nfo" name (the -print guarantees to print a filename match regardless of '-exec grep .... '

below , a snippet from my own attempt.
(OS:linux mint 20.2, utilities: find (GNU findutils) 4.7.0, grep (GNU grep) 3.4)

find /media/ -name "*.nfo" -exec grep -i Audio {} +
/media/movies/Antonine-wall.nfo:[b][color=orange]Audio info[/color][/b]
/media/movies/Brazil-rio.nfo:Audio
/media/movies/Brazil-rio.nfo:Format/Info                              : Advanced Audio Codec
/media/movies/danger.nfo:Audio Format: E-AC-3 (Dolby Digital Plus) 
/media/movies/danger.nfo:  Audio Lang: English
/media/movies/danger.nfo:  Audio Rate: 640 kb/s @ 6 channels          
/media/movies/Drive.2021.nfo:Audio
/media/movies/Drive.2021.nfo:Format/Info                              : Advanced Audio Codec
/media/movies/ecosse.nfo:[b][color=orange]Audio info[/color][/b] 
/media/movies/ecosse.nfo:Format: AVI (Audio Video Interleaved)
/media/movies/gold.nfo:[b][color=orange]Audio info[/color][/b]
/media/movies/gold.nfo:Format: AVI (Audio Video Interleaved) 
/media/movies/n500-scotland.nfo:Audio
/media/movies/n500-scotland.nfo:[b][color=orange]Audio info[/color][/b]
/media/movies/portugal-2022.nfo:Audio Channels...: 6
/media/movies/portugal-2022.nfo:Audio Codec......: AC-3
/media/movies/portugal-2022.nfo:Audio Bitrate....: 384 Kbps
/media/movies/reptiles.nfo:Audio Format: E-AC-3 (E-AC-3)
/media/movies/reptiles.nfo:  Audio Lang: English
/media/movies/reptiles.nfo:  Audio Rate: 768 kb/s @ 6 channels
/media/movies/stuff/S07E03.nfo: Audio                  : 128 kbps
/media/movies/Trekking.Worlds.nfo:Audio Format: E-AC-3 (Dolby Digital Plus)
/media/movies/Trekking.Worlds.nfo:  Audio Lang: English
/media/movies/Trekking.Worlds.nfo:  Audio Rate: 640 kb/s @ 6 channels 
/media/movies/Wild Africa.nfo:Audio 1 English.
/media/movies/Wild Africa.nfo:Audio 3 Italian.
/media/movies/Wild Africa.nfo:=>-Codec.............................: Digital Audio Compression AC-3.
...

shellcheck catches this too (which surprised me some).

SC1110: This is a unicode quote. Delete and retype it (or quote to make literal).

chapeau , excellent catch !

shellcheck -s bash zz

find /home/username/Desktop/Testing -name “*.nfo” -type f -exec sed -i "/<dateadded>/d" {} +
                                          ^-- SC1110 (warning): This is a unicode quote. Delete and retype it (or quote to make literal).
                                          ^-----^ SC2061 (warning): Quote the parameter to -name so the shell won't interpret it.
                                                ^-- SC1110 (warning): This is a unicode quote. Delete and retype it (or quote to make literal).

So... This works perfectly!

find /home/username/Desktop/Testing -name '*.nfo' -type f -exec sed -i "/<dateadded>/d" {} +

I'm going to test something later before applying this to all my files, but one last question... As far as the path on my server... Does the quote matter there as well? I have spaces in the path so it needs quotes, so which to use? I'm thinking...

find /mnt/user/"All Movies" -name '*.nfo' -type f -exec sed -i "/<dateadded>/d" {} +

Sound right?

The shell syntax only understands the ASCII quote.
Other "soft" quotes are multi-byte UTF-8 characters. They can be printed but are not part of the syntax.
In sed -i "/<dateadded>/d" the (ASCII-)quotes tell the shell to not try filename generation and word splitting, and then hand it unquoted to sed.
sed gets
ARG1: -i
ARG2: /<dateadded>/

I'm confused. It looks like you put "soft" quotes in that command. But it works? Or are you just putting two individual ASCII characters there? This is breaking my brain. LOL