Help parse comma separated list

Shell Programming and Scripting
1

I have a list of files with the same name, but they have a different date stamp in the name. I can find the first file, but I need to find the second file. I am using this information to create a variable I use later. Here is a example of how I find the first file.
"ls -mr /HOME/Daily_????????.txt | cut -d "," -f1 | head -1"
files are Daily_20100101.txt, Daily_20100102.txt, Daily_20100103.txt, etc.
this list get converted to Daily_20100103.txt,Daily_20100102.txt,Daily_2010101.txt
The variable1 gets set to "/HOME/Daily_20100103.txt"

I can grab the most current day, but I also need the information for the prior day and I can not seem to grab the second file in the list.
I would like variable2 to be "/HOME/Daily_20100102.txt"
Thanks in advance for your assistance.

2 
i=0
ls -t /HOME/Daily_*.txt | while read a
do
var[$i]=$a                                      or ${a##*/} if you want to get rid off the PATH
echo "var[$i] = ${var[$i]}"
let i++
done
3

You don't need the "-m" to ls. Use "ls -1" instead.

# Current
ls -1 /HOME/Daily_????????.txt | tail -1
# Previous
ls -1 /HOME/Daily_????????.txt | tail -2 | head -1
4

BTW, if these are true MS Comma Separated Value (CSV) files, they can have double quoted fields with comma, doubled double quotes for any double quote in a field and a carriage return as well as a line feed. (Excel always honors this, but not some old Access versions.)

haha,hoho,"Baked, AK","He said ""Get out of my head!"" and shook his head violently.",975^M
5

"ls -1 /HOME/Daily_????????.txt | tail -2 | head -1" seems to work like a charm..

Thanks
NMB

6

@DGPickett
This post has nothing to do with CSV files. The O/P had generated a comma-separated list of filenames using "ls -m" but this made processing of the list fiddly.
Btw. CSV pre-dates Microsoft.

7

It seems to me that you want the two most recent files. Have you considered:-

ls -1rt|tail -2|while read var2 # -1rt is number one (for one file name per line) rt (for reverse time order)
do
   read var1
done

Of course, this relies on the files update timestamp being in the correct sequence. if you would prefer to work on the file name being the determining factor, then:-

ls -1 Daily_*|tail -2|while read var2
do
   read var1
done

....or if there are a large number of files, you might be better with a find, such as:-

find . -name "Daily_*.txt"|sort|while read var2
do
   read var1
done

The sort is required because the order of find will probably read in i-node sequence, which is not necessarilly the order you want.

8

@rbatte1
Unfortunately none of your scripts actually produce the correct result.
On my tests $var2 is always blank.

9

If you want time, skip the tail and the r of the ls!

ls -lt | (
 read x                 # if you have a header to discard, but read below
 while read x
 do
 . . .
 done
 )

No need for -1 on ls, that is automatic on stdout=pipe. No total to discard if you provide a file name list Daily*.

The problem with 'ls Daily_*' and sort is that they sort alpha-numeric binary, so for many files, Daily11 will be after Daily1, Daily2 will be between Daily199 and Daily20. That is why I used sed to spin off the number as a separate sort field, and used sort -n.

The order of find will probably be in directory order! A directory is just a pile of entry name strings and inode numbers in whatever order occurs due to creation, deletion, and further creation.

10 
ls -t | head -1
11

@ctsgnb
Sorry for hijacking your thread for debate.

That may be true for a completely different filenames format.
The suffix here appears to be yyyymmdd which will come out in the chronological order with "ls" (which is why it is such a good choice of suffix).
(It doesn't work with American style yyyyddmm).
I can understand why he O/P has used a full pattern match too. We have very similar scripts for working in a central logs directory.

12

Sorry, cross-thread head leak! :smiley:

Still, no need for -1 when there is a pipe.