Parse file name out of UNC path

bytesnoop · May 17, 2012, 1:07pm

Hello,

I searched the forums and didn't see a situation like this:

I cannot figure out how to parse out just the file name from the full path. The path looks like this:

\\foo\bar\filename.ext

I don't think something like 'cut' will work so I tried to whip up a regex but couldn't get it quite right. Any ideas? Thanks in advance.

monalsinha · May 17, 2012, 1:10pm

try basename

Scrutinizer · May 17, 2012, 1:12pm

Try:

sed 's/.*\\//'

or

${var##*\\}

$ echo '\\foo\bar\filename.ext' | sed 's/.*\\//'
filename.ext

$ var='\\foo\bar\filename.ext'
$ echo "${var##*\\}"
filename.ext

The first is more suited for filtering stdin or files, the second is more suited for single strings..

bytesnoop · May 23, 2012, 12:14pm

Thanks for the quick feedback. How would I parse out everything BUT the file name? In other words, how can I parse our just the folder path?

Corona688 · May 23, 2012, 12:43pm

Advanced Bash Scripting Guide -- string operations

bytesnoop · May 23, 2012, 1:17pm

Thanks Corona, that index will prove useful someday. Let me run this scenario by you guys and maybe I can get a more elegant solution than what I've been rolling with thus far.

I want to create a script that will parse out a specific string (UNC or regular folder paths) from a Robocopy log. The string searching, parsing and de-duping has all been taken care of with this one liner:

grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >$1.tmp

colrm removes all the junk I don't need up to the beggining of the path. Uniq de-dupes it because I have Robocopy set to retry so I generally have 2 of the same error and tr cleans up the carriage returns (if there are any) so sed can work on the strings later.

What I am left with is a file containing 1 string per line: \\foo\bar\filename.ext

The end goal is to have the folder path and file name seperated into either temprary files or into seperate array's so that I can drop them into a new robocopy batch file with the following format:

RCXP026.exe  <path>  <destination>  <filename.ext> /R:1 /W:10 /TEE /Log+:�<Report Path with File Name>

The only other piece to the puzzle is to make sure each path and file name is wrapped in quotes. To this point I have everything figured out except for separating the folder and file name.

Thanks again for your time and consideration.

Corona688 · May 25, 2012, 6:52pm

LINE="\\foo\bar\filename.ext"
FILE="${LINE##*\\}"
FOLDER= "${LINE%\\*}"

bytesnoop · May 28, 2012, 3:43pm

OK, I see where you were going with your initial post Corona. Thanks for teaching me that, I never knew about those string features in bash.

I've run into a new problem with my code that I can't figure out. The while loop below successfully iterates through the array and fills the temporary files with the values I need. However, when I attempt to use sed to process these files after the loop completes, sed reports that there is "No such file or directory".

After the program exits I can open the two files I created and filled in the loop and they have the appropriate values, so why is sed telling me they don't exist? I even tried to use cat after the loop and I get the same error message. Any ideas? I'm at a loss and so close to completing this script.

# parse out, de-dupe and clean up failed source UNC paths. 
grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path

# create array of failed source UNC paths.
mapfile -t PATH <path

# parse out and save the source folder path and filenames from PATH array.
x=0
while [ $x -lt ${#PATH[@]} ]; do
	echo "${PATH["$x"]##*\\}" >>file
	echo "${PATH["$x"]%\\*}" >>folder
	x=$(( $x + 1 ))
done

# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed

Corona688 · May 28, 2012, 4:15pm

Can you show the exact error message this makes? Paste it word for word, letter for letter, keystroke for keystroke.

bytesnoop · May 28, 2012, 4:36pm

Copied straight off the terminal:

$ ./NEW-rc_logparser-denied.sh GS_Data3.txt
./NEW-rc_logparser-denied.sh: line 26: sed: No such file or directory
./NEW-rc_logparser-denied.sh: line 27: sed: No such file or directory

---------- Post updated at 04:36 PM ---------- Previous update was at 04:19 PM ----------

I know I'll eventually get this sorted out so I started looking at my code and seeing what I could do to make it more efficient. Since I have to duplicate the folder path from my source location to my destination location, I have decided to try out 'tee'. I am getting the same issues as I previously stated, take a look at code and STDERR:

# parse out and save the source folder path and filenames from 
# PATH array.
x=0
while [ $x -lt ${#PATH[@]} ]; do
	echo "${PATH["$x"]##*\\}" >>file
	echo "${PATH["$x"]%\\*}" |tee -a folder >dest
	x=$(( $x + 1 ))
done

# wrap each line from temp files in double quotes and assign temporary drive letter to destination path.
sed 's/^/"/; s/$/"/' file >file.sed
sed 's/^/"/; s/$/"/' folder >folder.sed
sed -r 's/^(.{1,2})/"B:\\/; s/$/"/' dest >dest.sed

$ ./NEW-rc_logparser-denied.sh GS_Data3.txt
./NEW-rc_logparser-denied.sh: line 22: tee: No such file or directory
./NEW-rc_logparser-denied.sh: line 27: sed: No such file or directory
./NEW-rc_logparser-denied.sh: line 28: sed: No such file or directory
./NEW-rc_logparser-denied.sh: line 29: sed: No such file or directory

The temporary drive letter will be replaced by the computer forensics data collection engineer after the batch file is created using WordPad or something similar. That is, if I can ever get this damn script to work.

bakunin · May 28, 2012, 5:09pm

bytesnoop:

# parse out, de-dupe and clean up failed source UNC paths. 
grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path

# create array of failed source UNC paths.
mapfile -t PATH <path

# parse out and save the source folder path and filenames from PATH array.
x=0
while [ $x -lt ${#PATH[@]} ]; do
	echo "${PATH["$x"]##*\\}" >>file
	echo "${PATH["$x"]%\\*}" >>folder
	x=$(( $x + 1 ))
done

# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed

I have no idea what does go wrong with your script because i can't see a (syntactical) error and a first test worked for me. I can tell you, though, that you are going in circles.

You "cook up" a file <path> in your first line, which should be a single sed (awk, text-filter-of-your-choice) statement, while it is a whole pipeline of different commands. That could and should be optimized, but let us put that aside for the moment.

After creating file <path> you parse it into an array PATH (which is at least dangerous, because PATH is a special variable to the shell) and then circle through this array writing two new files. What is the array for when you already have the same information in your file?

grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path

typeset pathname=""
while read pathname ; do
	echo "${pathname##*\\}" >>file
	echo "${pathname%\\*}" >>folder
done < path

# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed

Then you use two sed-invocations to add double quotes. Why don't you add them already in the echo-statements? It is possible to escape characters and use them literally instead of their special meaning to the shell:

grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path

typeset pathname=""
while read pathname ; do
	echo "\"${pathname##*\\}\"" >>file
	echo "\"${pathname%\\*}\"" >>folder
done < path

I won't even mention using full-qualified pathnames ("/path/to/file" instead of "file", etc.) inside of scripts because otherwise the result files are going to go wherever the script is called from - ever thought about putting this script in cron? You're guaranteed to have fun.

I hope this helps.

bakunin

Corona688 · May 28, 2012, 5:18pm

It's staring me in the face.

# create array of failed source UNC paths.
mapfile -t PATH <path

PATH is a special variable.

Once you overwrite it with this garbage, your shell will no longer be able to find external commands like sed.

bytesnoop · May 28, 2012, 6:54pm

corona688:

It's staring me in the face.
# create array of failed source UNC paths.
mapfile -t PATH <path
PATH is a special variable.

Once you overwrite it with this garbage, your shell will no longer be able to find external commands like sed.

:wall: Wow, that was the culprit?! Sorry to have waisted your time. I appreciate the help and thanks for helping me learn.

---------- Post updated at 06:54 PM ---------- Previous update was at 06:42 PM ----------

Definitely the least of my worries but I will definetly look into sed for this. I know I can make sed "grep" out the string and since if I don't use the global option I take care of de-duplication. My only issue with employing sed would be, how do I delete the first 54 characters of each line it finds? I got it to work before in another part of my code by haven't been able to do it since. I'll figure it out.

My initial program didn't use array's but I couldn't get a while loop to read through the <path> file and then process the strings with the string operations for folder and filename. I found that by passing the elements of the array in to the while loop the string operations worked and achieved my goal.

bakunin:

grep "ERROR 5" $1 |colrm 1 54 |uniq |tr -d '\r' >path

typeset pathname=""
while read pathname ; do
	echo "${pathname##*\\}" >>file
	echo "${pathname%\\*}" >>folder
done < path

# wrap each string from temp files in double quotes.
sed 's/^/"/; s/$/"/' file >>file.sed
sed 's/^/"/; s/$/"/' folder >>folder.sed

I love this idea, and it was something I was trying to do and failed at (see previous excuse). While this looks sound, it does not populate the files correctly. The file <file> contains the entire string from <path> and <folder> contains just <"">. For now I am sticking with my array.

I guess at some point I got too carried away with sed and didn't even thing to escape the quotes. Thanks, this makes it much cleaner.

Thanks a million for all the help. You made my day, I hope I can repay the favor to someone on this forum some day.