Add unique identifier from file to filetype in directory

I am trying to add a unique identifier to two file extensions .bam and .vcf in a directory located at /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome .

The identifier is in $2 of the input file. What the code below is attempting to do is strip off the last portion of the path in the for .... in the below bash that is R_2016_09_21_14_01_15_user_S5-00580-9-Medexome .

That string appears in the input file and is unique and has 3 lines above it with identifiers in it.

strings in bold are identifiers

IonXpress_007 MEV21
IonXpress_008 MEV22
IonXpress_009 MEV23
R_2016_09_21_14_01_15_user_S5-00580-9-Medexome   ---- this line is matched from the path in the directory

There are 3 .bam files in /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome , each on will match one $1 value in input . The corresponding $2 value is what is used as the identifier to update the file in /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome .

My actual data is several hundreds of lines but I have included a sample dataset:

input (file to update from) located at /home/cmccabe/s5_files/identifier

IonXpress_004 MEV49
IonXpress_005 MEV50
IonXpress_006 MEV51
R_2016_10_21_12_39_06_user_S5-00580-11-Medexome

IonXpress_001 MEC2
IonXpress_002 MEC3
IonXpress_003 MEV48
R_2016_10_21_09_52_37_user_S5-00580-10-Medexome

IonXpress_007 MEV21
IonXpress_008 MEV22
IonXpress_009 MEV23
R_2016_09_21_14_01_15_user_S5-00580-9-Medexome

files in /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome

MEV21.bam
MEV21.vcf
MEV22.bam
MEV22.vcf
MEV23.bam
MEV23.vcf

desired output

IonXpress_007.bam
IonXpress_007.vcf
IonXpress_008.bam
IonXpress_008.vcf
IonXpress_009.bam
IonXpress_009.vcf

bash

for file in /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/*.(bam,vcf}; do
   f="${file##*/}"
   path="${file%/*}"
   dt="${path##*/}"
   mv "$file" "$path/$(awk -v dt="$dt" -v f="$f" 'NF==1 {
               p=$0==dt ? 1 : 0; next} p && $1==f{print $2}' /home/cmccabe/s5_files/identifier/input)"
done

Currently the code does run, but the files do not update with the identifier. There will always be a match between the path and input . Thank you :).

Are you trying to rename *.bam and *.vcf files? Or, only *.bam files? There is nothing in your current script that does anything with *.vcf files, is there?

Do you only want to rename the files in the directory /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome ? Or do you want to rename the files in all of the directories in /home/cmccabe/Desktop/index/ that are named in your input file?

Do you need to look at the directory name to determine which name change needs to occur, or will the "unique identifiers" found in your input file be unique across all directories?

I updated the script to rename both .bam and .vcf , change is in bold. Hopefully, that is what I need to do.

Only the current directory files are renamed in this case /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-9-Medexome .

The path or current directory is uniques in the input file and the 3 lines above it have the identifiers in it.

So if, R_2016_09_21_14_01_15_user_S5-00580-9-Medexome is the path or current directory

in the input file
IonXpress_007 MEV21
IonXpress_008 MEV22
IonXpress_009 MEV23
R_2016_09_21_14_01_15_user_S5-00580-9-Medexome  --- path ----

Thank you :).

I am fully aware that your current code only processes the contents of one directory. What I asked was whether or not you wanted your script to only process one directory. Since you didn't answer that question, the following script only processes one subdirectory of the directory /home/cmccabe/Desktop/index . (With changes to two lines and removal of a third line in this script, you can make it process files in all of the subdirectories instead of just processing files in one subdirectory.) The subdirectory is specified by an operand passed to your script (which defaults to R_2016_09_21_14_01_15_user_S5-00580-9-Medexome if no operand is given when you invoke this script).

#!/bin/bash
BaseDir=/home/cmccabe/Desktop/index
TranslationFile=/home/cmccabe/s5_files/identifier/input
SubDir=${1:-R_2016_09_21_14_01_15_user_S5-00580-9-Medexome}

cd "$BaseDir/$SubDir"
printf '%s\n' *.bam *.vcf | awk '
FNR == NR  {
	if(NF == 2)
		old[$2] = $1
	next
}
{	prefix = substr($0, 1, length($0) - 4)
	suffix = substr($0, length($0) - 3)
	if(prefix in old)
		printf("mv \"%s\" \"%s%s\"\n", $0, old[prefix], suffix)
	else	printf("# No translation found for \"%s\"\n", $0)
}' "$TranslationFile" -

which, with the sample data you provided, produces the output:

mv "MEV21.bam" "IonXpress_007.bam"
mv "MEV22.bam" "IonXpress_008.bam"
mv "MEV23.bam" "IonXpress_009.bam"
mv "MEV21.vcf" "IonXpress_007.vcf"
mv "MEV22.vcf" "IonXpress_008.vcf"
mv "MEV23.vcf" "IonXpress_009.vcf"

If you like the list of command produced by this script, run it again and pipe the output to a shell.

And, as always, if you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk .

The full code is below, since I want to process only 1 directory at a time, as you knew, I use the first portion to ensure this.

#!/bin/bash

# get oldest folder
dir=/home/cmccabe/Desktop/index
{
  read -r -d $'\t' time && read -r -d '' filename
} < <(find "$dir" -maxdepth 1 -mindepth 1 -printf '%T+\t%P\0' | sort -z )
printf "The oldest folder is $filename and was created on $time, analysis was performed using v1.3 of the medex pipeline by $USER at $(date "+%D %r")\n" >> /home/cmccabe/Desktop/index/log

# rename bam
cd /home/cmccabe/Desktop/index/$filename
   rename 's/^([^_]+_[^_]+)_.+$/$1.bam/' *.bam
   
# rename vcf files
cd /home/cmccabe/Desktop/index/$filename
   rename 's/^([^_]+_[^_]+)_.+$/$1.vcf/' *.vcf
   
# rename .bam.bai files
cd /home/cmccabe/Desktop/index/$filename
   rename 's/^([^_]+_[^_]+)_.+$/$1.bam.bai/' *.bam.bai

# add identifier to bam and vcf
BaseDir=/home/cmccabe/Desktop/index  # search dir
TranslationFile=/home/cmccabe/s5_files/identifier/input #input
SubDir=${1:-$filename} # specific subdir

cd "$BaseDir/$SubDir" # look in this folder
printf '%s\n' *.bam *.vcf *.bam.bai | awk '
FNR == NR  {  # process all rows and columns
    if(NF == 2) # 2 columns in input
        old[$2] = $1  # old identifier
    next  # next line
}
{    prefix = substr($0, 1, length($0) - 4)
    if(prefix in old)
        printf("mv \"%s\" \"%s%s\"\n", $0, old[prefix])
    else    printf("# No translation found for \"%s\"\n", $0) #not found
}' "$TranslationFile" - # update from

since there is no suffix in $1 of input I get:

I removed them from the code and added a third file to search .bam.bai

input format

IonXpress_001 MEC2
IonXpress_002 MEC3
IonXpress_003 MEV48
R_2016_10_21_09_52_37_user_S5-00580-10-Medexome

IonXpress_007 MEV21
IonXpress_008 MEV22
IonXpress_009 MEV23
R_2016_09_21_14_01_15_user_S5-00580-9-Medexome
# No translation found for "IonXpress_007.bam"
# No translation found for "IonXpress_007.bam"
# No translation found for "IonXpress_007.bam"
# No translation found for "IonXpress_008.vcf"
# No translation found for "IonXpress_008.vcf"
# No translation found for "IonXpress_008.vcf"
# No translation found for "IonXpress_009.bam.bai"
# No translation found for "IonXpress_009.bam.bai"
# No translation found for "IonXpress_009.bam.bai"

Also I am not sure what you mean by output to a shell, as the files in the subdirectory, should be updated with the name from input . I tried to follow your code aand made comments that I hope are correct, but do not quite understand the portion in bold. I think that is what updates the identifiers, but not quite sure.

Example

IonXpress_007.bam  >>> MEV21.bam   ---- since the IonXpress_007 in the .bam located in the subdir matches $1 of input that .bam file is updated with $2 of input
IonXpress_007.vcf >>> MEV21.vcf   ---- since the IonXpress_007 in the .vcf  located in the subdir matches $1 of input  that .vcf file is updated with $2 of  input
IonXpress_007.bam.bai >>> MEV21.bam.bai  ---- since the IonXpress_007 in the .bam.bai  located in the subdir matches $1 of input  that .bam.bai file is updated with $2 of  input
IonXpress_008.bam  >>> MEV22.bam   ---- since the IonXpress_008 in the .bam  located in the subdir matches $1 of input  that .bam file is updated with $2 of  input
IonXpress_008.vcf >>> MEV22.vcf  ---- since the IonXpress_008 in the .vcf  located in the subdir matches $1 of input  that .vcf file is updated with $2 of  input
IonXpress_008.bam.bai >>> MEV22.bam.bai   ---- since the IonXpress_008 in the .bam.bai  located in the subdir matches $1 of input  that .bam.bai file is updated with $2 of  input
IonXpress_009.bam  >>> MEV23.bam   ---  since the IonXpress_009 in the .bam  located in the subdir matches $1 of input  that .bam file is updated with $2 of  input
IonXpress_009.vcf >>> MEV23.vcf   ---  since the IonXpress_009 in the .vcf  located in the subdir matches $1 of input  that .vcf file is updated with $2 of  input
IonXpress_009.bam.bai >>> MEV23.bam.bai  ---  since the IonXpress_009 in the .bam.bai  located in the subdir matches $1 of input  that .bam.bai file is updated with $2 of  input

Thank you for your help :).

Yes. I was fully aware that you only wanted to process one directory at a time. I gave you a script that processed one directory at a time (and told you that changing two lines and removing one line from that script would make it process all directories in a single run).

Did you even try running my suggested script with one operand (the name of the directory under /home/cmccabe/Desktop/index that you wanted to process)? Or, did you just decide to make my code fail by changing the names of all of the files you said you wanted my script to process before you invoked my script AND by deciding that some file suffixes to be processed will now be eight characters long instead of the four characters that you originally specified ( .bam and .vcf )???

What do you mean there is no suffix in $1 so you fixed my code??? My code extracted the existing suffix from the names of files being processed into an awk variable named suffix and looked for the prefix in $2 . When it renamed the file it was processing, it replaced the prefix found in $2 with the prefix in $1 (as you requested) and retained whatever four character suffix was on the existing filename. You gave no indication that there were other suffixes to be processed and you never gave a description of the format of the prefixes that could be present in your input file (so I had to assume that some of the hundreds of prefixes in your input file might contain a <period> character and that I couldn't be guaranteed that the first <period> found in a filename was the start of that filename's suffix). Therefore, my code assumed that the suffix was always four characters (as it was in all of your examples until you decided to change everything in post #5).

I don't have a rename utility on the system I'm using, so I can't verify what I think your code is doing, but I'm guessing that your code is renaming files with names of the form s1_s2_s3.bam to s1_s2.bam (and the same thing for the suffixes .vcf and .bam.bai ) where s1 and s2 are arbitrary strings that contain no underscore characters and do contain at least one character that is not an underscore and s3 is an arbitrary string of any zero or more characters. I have no idea what your original filenames were before this transformation, but I do know that in the data you showed us, there are no underscores in any name prefix in $2 in your input file. And, the output you got from my code stated that no prefix in $2 matched the prefix in the filename IonXpress_007.bam and the other eight files it listed. You will find those three prefixes in $1 in input , but none of them appear in $2 !

I fully admit that my original code doesn't stand a chance of working with your new (still incomplete) specification, but the changes you made made it much less likely that the code I suggested will ever work. And, I have absolutely no idea what you expect my script to do with the nine files for which it reported that it found no translation in your input .

It does seem extremely inefficient to rename files using rename and then use awk to rename them again using mv commands. And with the data you showed us in post #1 and post #5, I have absolutely no idea how you got the output you showed us above. I would have expected something much more like:

# No translation found for "IonXpress_007.bam"
# No translation found for "IonXpress_008.bam"
# No translation found for "IonXpress_009.bam"
# No translation found for "IonXpress_007.vcf"
# No translation found for "IonXpress_008.vcf"
# No translation found for "IonXpress_009.vcf"
# No translation found for "IonXpress_007.bam.bai"
# No translation found for "IonXpress_008.bam.bai"
# No translation found for "IonXpress_009.bam.bai"

If you want a script that does something completely different from what you have specified in all of your posts in this thread, you obviously need code that is different from what I wrote that tried to do what you specified in posts #1 and #3. You have yet to show us anywhere where my script behaved differently than you requested with input that matched what you described. And you have yet to show us that the directory you are trying to process contains any files that you said you wanted to rename.

I admit I did not fully understand your code and did run it as you had had it. That is how I got the No Translation Found . I only removed the suffix lines as I though that was I thought that was referring to $1 in input , and since it didn't match No Translation Found . I was mistaken and hope the below helps.

Files in directory being updated in dir: R_2016_09_21_14_01_15_user_S5-00580-9-Medexome

IonXpress_007.bam
IonXpress_007.vcf
IonXpress_007.bam.bai
IonXpress_008.bam
IonXpress_008.vcf
IonXpress_008.bam.bai
IonXpress_009.bam
IonXpress_009.vcf
IonXpress_009.bam.bai

input

IonXpress_001 MEC2
IonXpress_002 MEC3
IonXpress_003 MEV48
R_2016_10_21_09_52_37_user_S5-00580-10-Medexome

IonXpress_007 MEV21
IonXpress_008 MEV22
IonXpress_009 MEV23
R_2016_09_21_14_01_15_user_S5-00580-9-Medexome  --- line matches dir

The identifier is in $2 of the input file. That is what the file in dir should be updated with. The $1 value will match the file name (before it is renamed).

So using the dir in the example:

MEV21.bam
MEV21.vcf
MEV21.bam.bai
MEV22.bam
MEV22.vcf
MEV22.bam.bai
MEV23.bam
MEV23.vcf
MEV23.bam.bai

Each filename in dir IonXpress_007,IonXpress_008,IonXpress_009 will match a $1 value in input . The corresponding $2 value is what the filename in dir is renamed to. The dir will match only one line string appears in the input file and the 3 lines above it have the identifiers in it. Thank you very much :).

Please, try the following. If you like the output, comment the line that print and remove the comment on the line that says move.
This script parses the input file and works on each directory identified as the last line in each paragraph.

#!/usr/bin/perl
#
use strict;
use warnings;
use File::Copy;

{
    $/ = "\n\n";
    while(<>) {
        work_space();
    }
}

sub work_space{
    my @lines = split /\n/;
    my $curr_dir = pop @lines;
    for my $info (@lines) {
        my ($pattern, $new_name) = split /\s+/, $info;
        change_filename($curr_dir, $pattern, $new_name);
    }
}

sub change_filename {
    my ($c_dir, $pat, $new) = @_;
    my $base_path = "/home/cmccabe/Desktop/index"; # change me if needed.
    my $working_path = "$base_path/$c_dir";

    opendir my $dir, "$working_path" || return;
    my @files = grep { /$pat.*\.(bam|vcf)/ && -f "$working_path/$_" } readdir $dir;
    for my $f (@files) {
        my ($ext) = $f =~ /(\..*)$/;
        # For testing purposes. Comment this line and remove the # on the one after.
        print "$working_path/$f => GETS RENAMED TO => $working_path/$new$ext\n";
        #move "$working_path/$f", "$working_path/$new$ext";
    }
}
perl identifier.pl input
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_007.bam => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV21.bam
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_007.vcf => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV21.vcf
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_007.bam.bai => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV21.bam.bai
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_008.bam => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV22.bam
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_008.vcf => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV22.vcf
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_008.bam.bai => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV22.bam.bai
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_009.bam => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV23.bam
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_009.vcf => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV23.vcf
./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/IonXpress_009.bam.bai => GETS RENAMED TO => ./R_2016_09_21_14_01_15_user_S5-00580-9-Medexome/MEV23.bam.bai

You might note that in post #1 you said:

while in post #7 you said:

So, your original request was to rename MEV21.bam to IonXpress_007.bam and MEV21.vcf to IonXpress_007.vcf .

Now the request is to rename IonXpress_007.bam to MEV21.bam , IonXpress_007.vcf to MEV21.vcf , and IonXpress_007.bam.bai to MEV21.bam.bai .

In other words the request has changed from moving two files with fixed length suffixes per prefix to moving three files with varying length suffixes per prefix and the direction of movement has changed from moving from $2 to $1 to moving from $1 to $2 .

There is absolutely no reason given for adding all of the rename commands into your script which seem to be complete no-ops (assuming that the files that are in your folder have the names you say they have with just a prefix found in $1 in input and one of the three suffixes above).

Is there a reason why you need those rename commands in your script? Do your existing filenames contain additional characters between the prefixes in input and the three suffixes you want to process? If so, can a <period> ever be one of those additional characters?

Why is it so important that only one directory be processed at a time instead of renaming the files in all of the subdirectories in one run of your script?

Does Aia's suggestion do what you need? Or is something else still required?

My file names do contain many other additional characters that are removed in the rename . A <period> is not one of them however.

There are a few reasons to process only one directory at a time.

  1. Since I am a clinical scientist it is important to only process and log one directory
  2. Only the oldest directory is processed by the bash in the beginning
  3. Since many of the lines repeat duplicates may exist, however the current directory is unique in input

Aia's code works except I am getting:

readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 1.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 1.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 1.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 2.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 2.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 2.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 3.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 3.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 3.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 4.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 4.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 4.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 5.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 5.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 5.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 6.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 6.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 6.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 7.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 7.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 7.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 8.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 8.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 8.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 9.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 9.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 9.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 10.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 10.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 10.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 11.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 11.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 11.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 12.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 12.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 12.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 13.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 13.
readdir() attempted on invalid dirhandle $dir at /home/cmccabe/Desktop/NGS/scripts/identifier.pl line 29, <> chunk 13.

line 29 is the grep . Thank you very much :slight_smile:

Below is the bash used to run the code and ensure the current directory is used:

#!/bin/bash

# get oldest folder
dir=/home/cmccabe/Desktop/index
{
  read -r -d $'\t' time && read -r -d '' filename
} < <(find "$dir" -maxdepth 1 -mindepth 1 -printf '%T+\t%P\0' | sort -z )
printf "The oldest folder is $filename and was created on $time, analysis was performed using v1.3 of the medex pipeline by $USER at $(date "+%D %r")\n" >> /home/cmccabe/Desktop/index/log

# rename bam
cd /home/cmccabe/Desktop/index/$filename
   rename 's/^([^_]+_[^_]+)_.+$/$1.bam/' *.bam
   
# rename vcf files
cd /home/cmccabe/Desktop/index/$filename
   rename 's/^([^_]+_[^_]+)_.+$/$1.vcf/' *.vcf
   
# rename .bam.bai files
cd /home/cmccabe/Desktop/index/$filename
   rename 's/^([^_]+_[^_]+)_.+$/$1.bam.bai/' *.bam.bai

# add patient identifier to bam bam.bai and vcf
cd /home/cmccabe/Desktop/index/$filename
perl /home/cmccabe/Desktop/NGS/scripts/identifier.pl /home/cmccabe/s5_files/identifier/input

Those messages are for the directories in the input file that it can not read. I did not want to stop the program at the first time that entries in the input file does not contain an available path.

Please, replace || for or and they will go away.
Instead of:

opendir my $dir, "$working_path" || return;

This:

opendir my $dir, "$working_path" or return;

[/CODE]

Please, use the code outside the bash file. I read your other threads and the program does not require for you to rename them. It will work even if they are originally as:

IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_009_MEVxz_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.bam
IonXpress_007_MEVxx_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf
IonXpress_008_MEVxy_R_2016_11_18_10_45_10_user_S5-00580-14-Medexome.vcf

The program does not require to be cd'ed into a particular directory nor does depend of any transformation. Just make sure that the directory lines are correct in the input file. And that these directories live under the path: /home/cmccabe/Desktop/index or you need to change that where is says "change as needed."

The script does run without error now, however the files do not update in the directory. Below is the code I use the perl is run before the bash

Is this correct or am I missing something? Thank you for all of your help :).

#!/bin/bash

# add identifier to bam bam.bai and vcf
perl /home/cmccabe/Desktop/NGS/scripts/identifier.pl /home/cmccabe/s5_files/identifier/input

# get oldest folder
dir=/home/cmccabe/Desktop/index
{
  read -r -d $'\t' time && read -r -d '' filename
} < <(find "$dir" -maxdepth 1 -mindepth 1 -printf '%T+\t%P\0' | sort -z )
printf "The oldest folder is $filename and was created on $time, analysis was performed using v1.3 of the medex pipeline by $USER at $(date "+%D %r")\n" >> /home/cmccabe/Desktop/index/log

/home/cmccabe/s5_files/identifier/input

IonXpress_001 MEC2
IonXpress_002 MEC3
IonXpress_003 MEV53
R_2016_11_10_10_37_08_user_S5-00580-12-Medexome

IonXpress_004 MEV49
IonXpress_005 MEV50
IonXpress_006 MEV51
R_2016_10_21_12_39_06_user_S5-00580-11-Medexome

IonXpress_001 MEC2
IonXpress_002 MEC3
IonXpress_003 MEV48
R_2016_10_21_09_52_37_user_S5-00580-10-Medexome

IonXpress_007 MEV21
IonXpress_008 MEV22
IonXpress_009 MEV23
R_2016_09_21_14_01_15_user_S5-00580-9-Medexome

IonXpress_001 MEC1
IonXpress_002 MEC32
IonXpress_003 MEC33
R_2016_09_21_11_26_19_user_S5-00580-8-Medexome

current directory /home/cmccabe/Desktop/index/R_2016_09_21_11_26_19_user_S5-00580-8-Medexome

IonXpress_001_MEVxx_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_001_MEVxx_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_001_MEVxx_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_002_MEVxy_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_002_MEVxy_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_002_MEVxy_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_003_MEVxw_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_003_MEVxw_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_003_MEVxw_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai

after script is run files are renamed to:

MEC1.bam
MEC1.vcf
MEC1.bam.bai
MEV32.bam
MEV32.vcf
MEV32.bam.bai
MEV33.bam
MEV33.vcf
MEV33.bam.bai

You have shown at the end of the post #12 some files renamed. Is not that what you want?

But previously you said:

Are you referring that there are no logs in /home/cmccabe/Desktop/index/log ?

Otherwise, I am not understanding you.

after script is run files are renamed to: ---- desired output

MEC1.bam
MEC1.vcf
MEC1.bam.bai
MEV32.bam
MEV32.vcf
MEV32.bam.bai
MEV33.bam
MEV33.vcf
MEV33.bam.bai

but what the directory looks like currently after the script is run.

IonXpress_001_MEVxx_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_001_MEVxx_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_001_MEVxx_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_002_MEVxy_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_002_MEVxy_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_002_MEVxy_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_003_MEVxw_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_003_MEVxw_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_003_MEVxw_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai

Maybe I typed something wrong, but I do get a log in the index directory. Thank you :).

Where are you getting that list from? Is this a ls of the directory?

Are you saying that when you do ls /home/cmccabe/Desktop/index/R_2016_09_21_11_26_19_user_S5-00580-8-Medexome , this is what you see in it?

Did you make the change in identifier.pl as suggested in the comment? Did you comment the print and remove the # from move?

# For testing purposes. Comment this line and remove the # on the one after.
print "$working_path/$f => GETS RENAMED TO => $working_path/$new$ext\n";
#move "$working_path/$f", "$working_path/$new$ext";

Here is the information: the code as well as the directory after the execution of it. Thank you very much :).

perl

#!/usr/bin/perl
#
use strict;
use warnings;
use File::Copy;

{
    $/ = "\n\n";
    while(<>) {
        work_space();
    }
}

sub work_space{
    my @lines = split /\n/;
    my $curr_dir = pop @lines;
    for my $info (@lines) {
        my ($pattern, $new_name) = split /\s+/, $info;
        change_filename($curr_dir, $pattern, $new_name);
    }
}

sub change_filename {
    my ($c_dir, $pat, $new) = @_;
    my $base_path = "/home/cmccabe/Desktop/index"; # path to directory
    my $working_path = "$base_path/$c_dir";

    opendir my $dir, "$working_path" or return;
    my @files = grep { /$pat.*\.(bam|vcf)/ && -f "$working_path/$_" } readdir $dir;
    for my $f (@files) {
        my ($ext) = $f =~ /(\..*)$/;
        # For testing purposes. Comment this line and remove the # on the one after.
        #print "$working_path/$f => GETS RENAMED TO => $working_path/$new$ext\n";
        move "$working_path/$f", "$working_path/$new$ext";
    }
}

after the script executes:

ls /home/cmccabe/Desktop/index

/R_2016_09_21_14_01_15_user_S5-00580-8-Medexome
IonXpress_001_MEV21_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_001_MEV21_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_001_MEV21_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_002_MEV22_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_002_MEV22_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_002_MEV22_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf
IonXpress_003_MEV23_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam
IonXpress_003_MEV23_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.bam.bai
IonXpress_003_MEV23_R_2016_09_21_14_01_15_user_S5-00580-9-Medexome.vcf

Thank you, but where did you get this list from and what did you mean by "files are renamed to"?

Please, do some troubleshooting.
Place back the # to the move command. Remove the # from the print command.
Create a new_input file with just:

IonXpress_001 MEC1
IonXpress_002 MEC32
IonXpress_003 MEC33
R_2016_09_21_11_26_19_user_S5-00580-8-Medexome

Issue the command perl /home/cmccabe/Desktop/NGS/scripts/identifier.pl /home/cmccabe/s5_files/identifier/new_input

After that, please, post just the unmodified output showing the source => GETS RENAMED TO => destination .

Also, cd into R_2016_09_21_11_26_19_user_S5-00580-8-Medexome and post the result of the command: pwd

I tried the suggestions and the comand runs but there is source => GETS RENAMED TO => destination

Thank you :).

Did you mean "there is NOT" source => GETS RENAMED TO => destination? No output?
If there is output, please post it as displayed.
cd into R_2016_09_21_11_26_19_user_S5-00580-8-Medexome and post the result of the command: pwd

There is no output from the command though it does execute, I will post the pwd soon as well. Thank you very much:).

---------- Post updated 11-29-16 at 06:27 AM ---------- Previous update was 11-28-16 at 06:14 PM ----------

Here is the pwd

~/Desktop/index$ cd /home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-8-Medexome

~/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-8-Medexome$ pwd

/home/cmccabe/Desktop/index/R_2016_09_21_14_01_15_user_S5-00580-8-Medexome

Thank you :).