How to find all files for same month and year?

Shell Programming and Scripting
1

Hi All,
I find all files for same month and year
lets say there are following files in directory

-rwxr-xr-x   1 user  userg         1596 Mar 19 2012 c.txt
-rwxr-xr-x   1 user  userg         1596 Mar 21 2012 d.txt
-rwxr-xr-x   1 user  userg         1596 Mar 22 2012 f.txt
-rwxr-xr-x   1 user  userg         1596 Mar 24 2012 h.txt
-rwxr-xr-x   1 user  userg         1596 Mar 25 2012 w.txt
-rwxr-xr-x   1 user  userg         1596 Feb 12 2012 q.txt
-rwxr-xr-x   1 user  userg         1596 Feb 21 2012 a.txt
-rwxr-xr-x   1 user  userg         1596 Feb 22 2012 s.txt
-rwxr-xr-x   1 user  userg         1596 Feb 23 2012 k.txt
-rwxr-xr-x   1 user  userg         1596 Feb 27 2012 j.txt
-rwxr-xr-x   1 user  userg         1596 Apr 12 2014 q.txt
-rwxr-xr-x   1 user  userg         1596 Apr 21 2014 a.txt
-rwxr-xr-x   1 user  userg         1596 Apr 22 2014 s.txt
-rwxr-xr-x   1 user  userg         1596 Apr 23 2014 k.txt
-rwxr-xr-x   1 user  userg         1596 Apr 27 2014 j.txt

output should be:
first set should be

-rwxr-xr-x   1 user  userg         1596 Mar 19 2012 c.txt
-rwxr-xr-x   1 user  userg         1596 Mar 21 2012 d.txt
-rwxr-xr-x   1 user  userg         1596 Mar 22 2012 f.txt
-rwxr-xr-x   1 user  userg         1596 Mar 24 2012 h.txt
-rwxr-xr-x   1 user  userg         1596 Mar 25 2012 w.txt

and second set should be

-rwxr-xr-x   1 user  userg         1596 Feb 12 2012 q.txt
-rwxr-xr-x   1 user  userg         1596 Feb 21 2012 a.txt
-rwxr-xr-x   1 user  userg         1596 Feb 22 2012 s.txt
-rwxr-xr-x   1 user  userg         1596 Feb 23 2012 k.txt
-rwxr-xr-x   1 user  userg         1596 Feb 27 2012 j.txt

third set should be

-rwxr-xr-x   1 user  userg         1596 Apr 12 2014 q.txt
-rwxr-xr-x   1 user  userg         1596 Apr 21 2014 a.txt
-rwxr-xr-x   1 user  userg         1596 Apr 22 2014 s.txt
-rwxr-xr-x   1 user  userg         1596 Apr 23 2014 k.txt
-rwxr-xr-x   1 user  userg         1596 Apr 27 2014 j.txt
2

You could use ls -rlt to have ls order the listing by date/time.
How do you want the month sets separated? Blank line between each; output to filename with month in it; or something else?

3

basically i want to itearte though each of this sets in

for loop

.
first time for loop should itearet though first set ...
second time second set & so on

4

An oddly sorted output from ls -l indeed.

It might be easier to get the tools to do the work for you rather than to search and manipulate the output yourself. Using the touch command, you can create files with specific timestamps:-

$ touch -mt 201203010000 /tmp/my_begin
$ touch -mt 201204010000 /tmp/my_ended
$ ls -l /tmp/my_*
-rw-r--r--    1 RBATTE1  staff             0 01 Mar 2012  /tmp/my_begin
-rw-r--r--    1 RBATTE1  staff             0 01 Apr 2012  /tmp/my_ended

find /path/to/files -type f -newer /tmp/my_begin ! -newer /tmp/my_ended

You should be able to build a for loop around this to set the start and end dates.

I hope that this helps,
Robin
Liverpool/Blackburn
UK

5

You could also increment a set# variable every time the month changes like this (ksh/bash code):

ls -rlt | while read line
do
   mth=$(echo "$line" | awk '{print $6}')
   if [ "$mth" != "$last" ]
   then
      ((set++))
      last="$mth"
   fi
   echo $set ":" $line
done
6 
ls -l | awk '{idx=$(NF-3) FS $(NF-1);a[idx]=a[idx]?a[idx] ORS $0:$0}END {for (i in a) print i ORS a ORS}

Feb 2012
-rwxr-xr-x   1 user  userg         1596 Feb 12 2012 q.txt
-rwxr-xr-x   1 user  userg         1596 Feb 21 2012 a.txt
-rwxr-xr-x   1 user  userg         1596 Feb 22 2012 s.txt
-rwxr-xr-x   1 user  userg         1596 Feb 23 2012 k.txt
-rwxr-xr-x   1 user  userg         1596 Feb 27 2012 j.txt

Mar 2012
-rwxr-xr-x   1 user  userg         1596 Mar 19 2012 c.txt
-rwxr-xr-x   1 user  userg         1596 Mar 21 2012 d.txt
-rwxr-xr-x   1 user  userg         1596 Mar 22 2012 f.txt
-rwxr-xr-x   1 user  userg         1596 Mar 24 2012 h.txt
-rwxr-xr-x   1 user  userg         1596 Mar 25 2012 w.txt

Apr 2014
-rwxr-xr-x   1 user  userg         1596 Apr 12 2014 q.txt
-rwxr-xr-x   1 user  userg         1596 Apr 21 2014 a.txt
-rwxr-xr-x   1 user  userg         1596 Apr 22 2014 s.txt
-rwxr-xr-x   1 user  userg         1596 Apr 23 2014 k.txt
-rwxr-xr-x   1 user  userg         1596 Apr 27 2014 j.txt
7

There are some very strange things going on in this thread:
Makarand Dodmis tells us that he has the following files in a directory:

-rwxr-xr-x   1 user  userg         1596 Mar 19 2012 c.txt
-rwxr-xr-x   1 user  userg         1596 Mar 21 2012 d.txt
-rwxr-xr-x   1 user  userg         1596 Mar 22 2012 f.txt
-rwxr-xr-x   1 user  userg         1596 Mar 24 2012 h.txt
-rwxr-xr-x   1 user  userg         1596 Mar 25 2012 w.txt
-rwxr-xr-x   1 user  userg         1596 Feb 12 2012 q.txt
-rwxr-xr-x   1 user  userg         1596 Feb 21 2012 a.txt
-rwxr-xr-x   1 user  userg         1596 Feb 22 2012 s.txt
-rwxr-xr-x   1 user  userg         1596 Feb 23 2012 k.txt
-rwxr-xr-x   1 user  userg         1596 Feb 27 2012 j.txt
-rwxr-xr-x   1 user  userg         1596 Apr 12 2014 q.txt
-rwxr-xr-x   1 user  userg         1596 Apr 21 2014 a.txt
-rwxr-xr-x   1 user  userg         1596 Apr 22 2014 s.txt
-rwxr-xr-x   1 user  userg         1596 Apr 23 2014 k.txt
-rwxr-xr-x   1 user  userg         1596 Apr 27 2014 j.txt

Note that there are several pairs of files with the same names, but different dates. Obviously this can' t happen in a single directory!

Standard ls -lt or ls -ltr output would sort the dates from oldest to newest or newest to oldest; not in the order shown above with the oldest files in the middle of the listings.

And, files with dates in the past six months would be shown with a timestamp rather than a year in the date display. (This means that some files from October 2013 will be displayed with a timestamp and some will be presented with a year if ls -l is run in April 2014.

And, finally, ls -l with or without other options will also print a line something like:

total 75

at the start of the output.

Chubler_XL's script ignores the year and assumes that grouping by month will be sufficient to get what is desired. (And, it works for the sample data given. Although it also prints the "total" line.)

vgersh99's script gets a syntax error on the "total" line and treats each timestamp on files in the previous six months as a different year.

In a directory where ls -lrt (when run on April 22, 2014 after 8pm) produces the output:

total 40
-rw-r--r--  1 dwc  staff     0 Feb 12  2012 q.txt
-rw-r--r--  1 dwc  staff     0 Feb 21  2012 a.txt
-rw-r--r--  1 dwc  staff     0 Feb 22  2012 s.txt
-rw-r--r--  1 dwc  staff     0 Feb 23  2012 k.txt
-rw-r--r--  1 dwc  staff     0 Feb 27  2012 j.txt
-rw-r--r--  1 dwc  staff     0 Mar 19  2012 c.txt
-rw-r--r--  1 dwc  staff     0 Mar 21  2012 d.txt
-rw-r--r--  1 dwc  staff     0 Mar 22  2012 f.txt
-rw-r--r--  1 dwc  staff     0 Mar 24  2012 h.txt
-rw-r--r--  1 dwc  staff     0 Mar  1  2013 y.txt
-rw-r--r--  1 dwc  staff     0 Oct  1  2013 b.txt
-rw-r--r--  1 dwc  staff     0 Oct 31 12:00 z.txt
-rw-r--r--  1 dwc  staff     0 Apr 12 01:02 e.txt
-rw-r--r--  1 dwc  staff     0 Apr 21 03:04 g.txt
-rw-r--r--  1 dwc  staff     0 Apr 22 05:06 i.txt
-rw-r--r--  1 dwc  staff  2266 Apr 22 08:22 problem
-rwxr-xr-x  1 dwc  staff   186 Apr 22 18:12 Chubler
-rwxr-xr-x  1 dwc  staff   135 Apr 22 18:23 vgersh99
-rwxr-xr-x  1 dwc  staff   579 Apr 22 19:32 tester.ksh
-rwxr-xr-x  1 dwc  staff   410 Apr 22 19:32 tester
-rw-r--r--  1 dwc  staff     0 Apr 23  2014 m.txt
-rw-r--r--  1 dwc  staff     0 Apr 27  2014 n.txt

the awk script:

#!/bin/ksh
ls -lrt | awk -v cy=$(date +%Y) '
BEGIN {	y["Jan"] = y["Feb"] = y["Mar"] = y["Apr"] = y["May"] = y["Jun"] = " " cy
	y["Jul"] = y["Aug"] = y["Sep"] = y["Oct"] = y["Nov"] = y["Dec"] = " " cy - 1
}
NF > 8 {if(length($8) == 4)	# Do we have a year or a timestamp?
		my = $6 " " $8	#   year
	else	my = $6 y[$6]	#   timestamp
	if(my != last) {
		set++
		last = my
	}
	printf("%3d:%s:%s\n", set, my, $0)
}'

(change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk on Solaris/SunOS systems) and the following Korn shell script (when using a 1993 or later version of ksh ):

#!/bin/ksh
cy=$(date +%Y)
last=""
ly=$((cy - 1))
set=""
typeset -A y
y["Jan"]=$cy
y["Feb"]=$cy
y["Mar"]=$cy
y["Apr"]=$cy
y["May"]=$cy
y["Jun"]=$cy
y["Jul"]=$ly
y["Aug"]=$ly
y["Sep"]=$ly
y["Oct"]=$ly
y["Nov"]=$ly
y["Dec"]=$ly
ls -lrt | while read -r line
do	if [ "$set" == "" ]
	then	# skip "total" line from ls
		set=0
		continue
	fi
	set -- $line
	if [ ${#8} -eq 4 ]		# Do we have a year or a timestamp?
	then	my="$6 $8"		#   year
	else	my="$6 ${y[$6]}"	#   timestamp
	fi
	if [ "$my" != "$last" ]
	then
		((set++))
		last="$my"
	fi
	printf "%3d:%s:%s\n" $set "$my" "$line"
done

both produce the output:

  1:Feb 2012:-rw-r--r--  1 dwc  staff     0 Feb 12  2012 q.txt
  1:Feb 2012:-rw-r--r--  1 dwc  staff     0 Feb 21  2012 a.txt
  1:Feb 2012:-rw-r--r--  1 dwc  staff     0 Feb 22  2012 s.txt
  1:Feb 2012:-rw-r--r--  1 dwc  staff     0 Feb 23  2012 k.txt
  1:Feb 2012:-rw-r--r--  1 dwc  staff     0 Feb 27  2012 j.txt
  2:Mar 2012:-rw-r--r--  1 dwc  staff     0 Mar 19  2012 c.txt
  2:Mar 2012:-rw-r--r--  1 dwc  staff     0 Mar 21  2012 d.txt
  2:Mar 2012:-rw-r--r--  1 dwc  staff     0 Mar 22  2012 f.txt
  2:Mar 2012:-rw-r--r--  1 dwc  staff     0 Mar 24  2012 h.txt
  3:Mar 2013:-rw-r--r--  1 dwc  staff     0 Mar  1  2013 y.txt
  4:Oct 2013:-rw-r--r--  1 dwc  staff     0 Oct  1  2013 b.txt
  4:Oct 2013:-rw-r--r--  1 dwc  staff     0 Oct 31 12:00 z.txt
  5:Apr 2014:-rw-r--r--  1 dwc  staff     0 Apr 12 01:02 e.txt
  5:Apr 2014:-rw-r--r--  1 dwc  staff     0 Apr 21 03:04 g.txt
  5:Apr 2014:-rw-r--r--  1 dwc  staff     0 Apr 22 05:06 i.txt
  5:Apr 2014:-rw-r--r--  1 dwc  staff  2266 Apr 22 08:22 problem
  5:Apr 2014:-rwxr-xr-x  1 dwc  staff   186 Apr 22 18:12 Chubler
  5:Apr 2014:-rwxr-xr-x  1 dwc  staff   135 Apr 22 18:23 vgersh99
  5:Apr 2014:-rwxr-xr-x  1 dwc  staff   579 Apr 22 19:32 tester.ksh
  5:Apr 2014:-rwxr-xr-x  1 dwc  staff   410 Apr 22 19:32 tester
  5:Apr 2014:-rw-r--r--  1 dwc  staff     0 Apr 23  2014 m.txt
  5:Apr 2014:-rw-r--r--  1 dwc  staff     0 Apr 27  2014 n.txt
1 Like
8

Thanks Don for this useful post

1) The data i have provided is just a dummy data because i am not allowed to provide real data.i agree with your concerns

2) I just want set of files per month/year to iterate through.

3) i want to find files with first two dates & last date for that month/year

---------- Post updated at 03:38 AM ---------- Previous update was at 03:11 AM ----------

Hi Don
I am getting below error.Any help

./tunning.sh
./tunning.sh[6]: typeset: bad option(s)

my unix version

SunOS abc100 5.8 Generic_117350-61 sun4u sparc SUNW,Sun-Fire-V440
9

Dummy data is fine. You can change the files' owners to "user" (or if there are multiple users, "user1", "user2", ...) and the files' group names to "group" (or "group1", "group2", ...) and the files' names to "x1.txt", "x2.txt", ... "xn.txt", but changing output like:

-rw-r--r--   1 dwc  staff    8095 Apr 22 22:11 d.txt

to:

-rw-r--r--   1 dwc  staff    8095 Apr 22  2014 d.txt

hides information that is crucial to getting the results you want. You should easily be able to the save output from ls -lt in a file, edit the file to sanitize private data, and post the sanitized output without hiding data that your script is going to need to use to get the job done.

I explicitly said "the following Korn shell script (when using a 1993 or later version of ksh)". /bin/ksh on Solaris 8 is a 1988 version of the Korn shell. The 1988 version of the Korn shell doesn't have associative arrays. The awk script I provided should work fine for you if you change awk to /usr/xpg4/bin/awk or nawk . (I don't remember the path, but there might also be a 1993 version of ksh on Solaris 8 with a pathname something like /usr/dtbin/dtksh. You might be able to find it with:
ls /usr/*/dtksh /usr/*/*/dtksh

With the output that script provides you can easily grep for ":Oct 2013:" (or any other month and year you want) to find the files for that month and year. I assume you can pick the 1st two and the last one out of the matching grep output to get the files you want to process for that month. Of course it would also be pretty easy to just modify the awk script or a shell script to pass in the month and year you want and have it only print the names of the three (or all, if there are less than three) files that you want for that month.

Try to build on what we've given you here to get what you want. If it doesn't work, start another thread showing us what you have done, what (sanitized) input you're processing, and what output you;re hoping to get.

1 Like