Script to determine Date,TotalFile,total size of file based on date

Shell Programming and Scripting
1

I have file listed like below

-rw-r--r--+ 1 test test  17M Nov 26 14:43 test1.gz
-rw-r--r--+ 1 test test    0 Nov 26 14:44 test2.gz
-rw-r--r--+ 1 test test    0 Nov 27 10:41 test3.gz
-rw-r--r--+ 1 test test 244K Nov 27 10:41 test4.gz
-rw-r--r--+ 1 test test  17M Nov 27 10:41 test5.gz

I need output as below

DATE TOTALFILE SIZE
Nov26   2       17M
Nov27   3       17.244M

Able to determine the DATE and TOTALFILE using below script

echo " "
echo  "DATE TOTALFILE"
cd /Mydir
ls -lrth | grep "^-" | awk '{
key=$6$7
freq[key]++
}
END {
for (date in freq)
       
        printf "%s\t%d\n", date, freq[date]
}'

Can someone help me to get the size as well based on date.

2

Hello,

If I am correct, you are doing the file listing using "ls -lh", instead you can do just "ls -l" which make sure that file sizes are listed in bytes.

if you do so, it would be easy to add the bytes and convert it into MB or KB.

Something like this:

ls -lrt | grep "^-" | awk '{
key=$6$7
freq[key]++
a[freq[key]]=a[freq[key]]+$5;
}
END {
for (date in freq)

        printf "%s\t%d\t%f\n", date, freq[date],a[freq[date]]/(1024*1024)
}'
3

Yes you are correct Panyam. i am doing

ls -lrth

. In my script you can see that.

4

Pls, re read my previous post again.

5

This is similar to your script adding the size (as long as you don't mind actual byte counts instead of sums of "human readable" approximations):

printf '\nDATE\tFILES\tSIZE\n'
cd /Mydir
ls -lrt | grep "^-" | awk '
{	key=$6$7
	freq[key]++
	size[key] += $5
}
END {	for (date in freq)
		printf("%s\t%d\t%d\n", date, freq[date], size[date])
}'

Like your script, the output is not necessarily in date order. If you'd like the output in date order, you could try something more like:

printf '\nDATE\tFILES\tSIZE\n'
cd /Mydir
ls -lrt | grep "^-" | awk '
{	nkey = $6$7
	if(key != nkey && key != "") {
		printf("%s\t%d\t%d\n", key, freq, size)
		freq = 1;
		size = $5
	} else {freq++
		size += $5
	}
	key = nkey
}
END {	printf("%s\t%d\t%d\n", key, freq, size)
}'

Neither of these have been tested, but they should come close to what you need. I will leave the conversion of actual bytes to human readable form as an exercise for the reader. (I prefer the actual byte counts.)

If you want to use panyam's code I think you would need to change all occurrences of a[freq[key]] to a[key] and change a[freq[date]] to a[date] .

6

Thanks Don!!!

Tested and getting expected result

DATE TOTALFILE TOTALSIZE
Nov26   2       17506702
Nov27   26      2429166955
Dec1    1        133