How to add missing date and time in a bash script?

Shell Programming and Scripting
1

Hi Again,

I have a file that contains date and time for the past 2 hours. What i need is add missing date and time in a file.

INPUT

2016-01-13 01:33 10
2016-01-13 01:31 10
2016-01-13 01:30 10
2016-01-13 01:29 10
2016-01-13 01:28 10
2016-01-13 01:27 10
2016-01-13 01:26 10
2016-01-13 01:06 10
2016-01-13 01:05 10
2016-01-13 01:04 10
2016-01-13 01:03 10
2016-01-13 01:02 10
2016-01-13 01:00 10
2016-01-13 00:59 10
2016-01-13 00:58 10
2016-01-13 00:57 10
2016-01-13 00:56 10
2016-01-13 00:55 10
2016-01-13 00:54 10
2016-01-13 00:53 10
2016-01-13 00:52 10
2016-01-13 00:51 10
2016-01-13 00:50 10
2016-01-13 00:49 10
2016-01-13 00:48 10
2016-01-13 00:47 10
2016-01-13 00:46 10
2016-01-13 00:45 10
2016-01-13 00:44 10
2016-01-13 00:43 10
2016-01-13 00:33 10
2016-01-13 00:30 10
2016-01-13 00:29 10
2016-01-13 00:28 10
2016-01-13 00:27 10
2016-01-13 00:26 10
2016-01-13 00:25 10
2016-01-13 00:24 10
2016-01-13 00:23 10
2016-01-13 00:22 10
2016-01-13 00:21 10
2016-01-13 00:20 10
2016-01-12 21:33 10

OUTPUT

2016-01-13 01:33 10
2016-01-13 01:31 10
2016-01-13 01:30 10
2016-01-13 01:29 10
2016-01-13 01:28 10
2016-01-13 01:27 10
2016-01-13 01:26 10
2016-01-13 01:25 0
2016-01-13 01:24 0
2016-01-13 01:23 0
2016-01-13 01:22 0
2016-01-13 01:21 0
2016-01-13 01:20 0
2016-01-13 01:19 0
2016-01-13 01:18 0
2016-01-13 01:17 0
2016-01-13 01:16 0
2016-01-13 01:15 0
2016-01-13 01:14 0
2016-01-13 01:13 0
2016-01-13 01:12 0
2016-01-13 01:11 0
2016-01-13 01:10 0
2016-01-13 01:09 0
2016-01-13 01:08 0
2016-01-13 01:07 0
2016-01-13 01:06 10
2016-01-13 01:05 10
2016-01-13 01:04 10
2016-01-13 01:03 10
2016-01-13 01:02 10
2016-01-13 01:00 10
2016-01-13 00:59 10
2016-01-13 00:58 10
2016-01-13 00:57 10
2016-01-13 00:56 10
2016-01-13 00:55 10
2016-01-13 00:54 10
2016-01-13 00:53 10
2016-01-13 00:52 10
2016-01-13 00:51 10
2016-01-13 00:50 10
2016-01-13 00:49 10
2016-01-13 00:48 10
2016-01-13 00:47 10
2016-01-13 00:46 10
2016-01-13 00:45 10
2016-01-13 00:44 10
2016-01-13 00:43 10
2016-01-13 00:42 0
2016-01-13 00:41 0
2016-01-13 00:40 0
2016-01-13 00:39 0
2016-01-13 00:37 0
2016-01-13 00:36 0
2016-01-13 00:35 0
2016-01-13 00:34 0
2016-01-13 00:33 10
2016-01-13 00:32 0
2016-01-13 00:31 0
2016-01-13 00:30 10
2016-01-13 00:29 10
2016-01-13 00:28 10
2016-01-13 00:27 10
2016-01-13 00:26 10
2016-01-13 00:25 10
2016-01-13 00:24 10
2016-01-13 00:23 10
2016-01-13 00:22 10
2016-01-13 00:21 10
2016-01-13 00:20 10
2016-01-13 00:19 0
2016-01-13 00:18 0
2016-01-13 00:17 0
2016-01-13 00:16 0
2016-01-13 00:15 0
2016-01-13 00:14 0
2016-01-13 00:13 0
2016-01-13 00:12 0
2016-01-13 00:11 0
2016-01-13 00:10 0
2016-01-13 00:09 0
2016-01-13 00:08 0
2016-01-13 00:07 0
2016-01-13 00:06 0
2016-01-13 00:05 0
2016-01-13 00:04 0
2016-01-13 00:03 0
2016-01-13 00:02 0
2016-01-13 00:01 0
2016-01-13 00:00 0
2016-01-12 23:59 0
2016-01-12 23:58 0
2016-01-12 23:57 0
2016-01-12 23:56 0
2016-01-12 23:55 0
2016-01-12 23:54 0
2016-01-12 23:53 0
2016-01-12 23:52 0
2016-01-12 23:51 0
2016-01-12 23:50 0
2016-01-12 23:49 0
2016-01-12 23:48 0
2016-01-12 23:47 0
2016-01-12 23:46 0
2016-01-12 23:45 0
2016-01-12 23:44 0
2016-01-12 23:43 0
2016-01-12 23:42 0
2016-01-12 23:41 0
2016-01-12 23:42 0
2016-01-12 23:41 0
2016-01-12 23:40 0
2016-01-12 23:39 0
2016-01-12 23:38 0
2016-01-12 23:37 0
2016-01-12 23:36 0
2016-01-12 23:35 0
2016-01-12 23:34 0
2016-01-12 21:33 10

Thanks.

2

Any attempts from your side?

3

Hi Rudic,

Yes, have tried this one

awk '{split($1,N,":"); while (++L < N[1]) printf "%02d:00\t0 <--\n", L; L=N[1]} 1'

but it is not what i am looking for.

OUTPUT

[dev:~]$ awk '{split($1,N,":"); while (++L < N[1]) printf "%02d:00\t0 <--\n", L; L=N[1]} 1' test123.txt 

01:00	0 <--
02:00	0 <--
2016-01-13 11:59 0
2016-01-13 09:59 0

Need help.

4

Try (requires a recent shell):

read FD <file
FS=$(date -d"${FD% *}" +%s)
for DT in {0..7200..60}; do printf "%(%F %H:%M)T 0\n" $(( FS - DT)); done |
awk '
NR == FNR       {T[$1,$2] = $3
                 next
                }
($1,$2) in T    {$3 = T[$1,$2]  
                 delete T[$1,$2]
                }
1
END             {for (t in T) print t, T[t]
                }   
' SUBSEP=" " file -
2016-01-13 01:33 10
2016-01-13 01:32 0
2016-01-13 01:31 10
2016-01-13 01:30 10
.
.
.
2016-01-12 23:33 0
2016-01-12 21:33 10
5

Hi,

I am getting error.

bash: printf: `(': invalid format character
bash: printf: `(': invalid format character
bash: printf: `(': invalid format character
bash: printf: `(': invalid format character
6

As noted: recent! shell... What be your bash version?

7

Hi,

Here is the bash version

[dev:~]$ echo $BASH_VERSION
4.1.2(1)-release

8

That version should provide the %(...)T format specifier. Did your copy/paste the proposal as given?

9

yes, but i'm still getting the same error. Please see debug mode details below:

+ for DT in '{0..7200..60}'
+ printf '%(%F %H:%M)T 0\n' 1452611280
test.sh: line 7: printf: `(': invalid format character
+ for DT in '{0..7200..60}'
+ printf '%(%F %H:%M)T 0\n' 1452611220
test.sh: line 7: printf: `(': invalid format character
+ for DT in '{0..7200..60}'
+ printf '%(%F %H:%M)T 0\n' 1452611160
test.sh: line 7: printf: `(': invalid format character
+ for DT in '{0..7200..60}'
+ printf '%(%F %H:%M)T 0\n' 1452611100
test.sh: line 7: printf: `(': invalid format character
10

Too bad ... what's said in man bash ?

---------- Post updated at 10:20 ---------- Previous update was at 10:15 ----------

Well, try

read FD <file
FS=$(date -d"${FD% *}" +%s)
seq -f"@%.0f" $FS -60 $((FS-7200)) | date -f- +"%F %H:%M 0" | awk '
NR == FNR       {T[$1,$2] = $3
                 next
                }
($1,$2) in T    {$3 = T[$1,$2]
                 delete T[$1,$2]
                }
1
END             {for (t in T) print t, T[t]
                }
' SUBSEP=" " file -
2016-01-13 01:33 10
2016-01-13 01:32 0
2016-01-13 01:31 10
2016-01-13 01:30 10
2016-01-13 01:29 10
2016-01-13 01:28 10
2016-01-13 01:27 10
2016-01-13 01:26 10
2016-01-13 01:25 0
2016-01-13 01:24 0
2016-01-13 01:23 0
.
.
.
2016-01-12 23:34 0
2016-01-12 23:33 0
2016-01-12 21:33 10
2 Likes
11

Hi RudiC,

thanks for the info. but this is what i am getting, it replaces all those in third column with 0.

2016-01-13 01:33 0
2016-01-13 01:32 0
2016-01-13 01:31 0
2016-01-13 01:30 0
2016-01-13 01:29 0
2016-01-13 01:28 0
2016-01-13 01:27 0
2016-01-13 01:26 0
2016-01-13 01:25 0
2016-01-13 01:24 0
2016-01-13 01:23 0
2016-01-13 01:22 0
2016-01-13 01:21 0
2016-01-13 01:20 0
2016-01-13 01:19 0
2016-01-13 01:18 0
2016-01-13 01:17 0
2016-01-13 01:16 0
2016-01-13 01:15 0
2016-01-13 01:14 0
2016-01-13 01:13 0
2016-01-13 01:12 0
2016-01-13 01:11 0
2016-01-13 01:10 0
2016-01-13 01:09 0
2016-01-13 01:08 0
2016-01-13 01:07 0
2016-01-13 01:06 0
2016-01-13 01:05 0
2016-01-13 01:04 0
2016-01-13 01:03 0
2016-01-13 01:02 0
2016-01-13 01:01 0
2016-01-13 01:00 0
2016-01-13 00:59 0
2016-01-13 00:58 0
2016-01-13 00:57 0
2016-01-13 00:56 0
2016-01-13 00:55 0
2016-01-13 00:54 0
2016-01-13 00:53 0
2016-01-13 00:52 0
2016-01-13 00:51 0
2016-01-13 00:50 0
2016-01-13 00:49 0
2016-01-13 00:48 0
2016-01-13 00:47 0
2016-01-13 00:46 0
2016-01-13 00:45 0
2016-01-13 00:44 0
2016-01-13 00:43 0
2016-01-13 00:42 0
2016-01-13 00:41 0
2016-01-13 00:40 0
2016-01-13 00:39 0
2016-01-13 00:38 0
2016-01-13 00:37 0
2016-01-13 00:36 0
2016-01-13 00:35 0
2016-01-13 00:34 0
2016-01-13 00:33 0
2016-01-13 00:32 0
2016-01-13 00:31 0
2016-01-13 00:30 0
2016-01-13 00:29 0
2016-01-13 00:28 0
2016-01-13 00:27 0
2016-01-13 00:26 0
2016-01-13 00:25 0
2016-01-13 00:24 0
2016-01-13 00:23 0
2016-01-13 00:22 0
2016-01-13 00:21 0
2016-01-13 00:20 0
2016-01-13 00:19 0
2016-01-13 00:18 0
2016-01-13 00:17 0
2016-01-13 00:16 0
2016-01-13 00:15 0
2016-01-13 00:14 0
2016-01-13 00:13 0
2016-01-13 00:12 0
2016-01-13 00:11 0
2016-01-13 00:10 0
2016-01-13 00:09 0
2016-01-13 00:08 0
2016-01-13 00:07 0
2016-01-13 00:06 0
2016-01-13 00:05 0
2016-01-13 00:04 0
2016-01-13 00:03 0
2016-01-13 00:02 0
2016-01-13 00:01 0
2016-01-13 00:00 0
2016-01-12 23:59 0
2016-01-12 23:58 0
2016-01-12 23:57 0
2016-01-12 23:56 0
2016-01-12 23:55 0
2016-01-12 23:54 0
2016-01-12 23:53 0
2016-01-12 23:52 0
2016-01-12 23:51 0
2016-01-12 23:50 0
2016-01-12 23:49 0
2016-01-12 23:48 0
2016-01-12 23:47 0
2016-01-12 23:46 0
2016-01-12 23:45 0
2016-01-12 23:44 0
2016-01-12 23:43 0
2016-01-12 23:42 0
2016-01-12 23:41 0
2016-01-12 23:40 0
2016-01-12 23:39 0
2016-01-12 23:38 0
2016-01-12 23:37 0
2016-01-12 23:36 0
2016-01-12 23:35 0
2016-01-12 23:34 0

is there a way for it to insert only those records that are not in the input file?

Thanks

---------- Post updated at 07:43 AM ---------- Previous update was at 07:40 AM ----------

sorry for asking too much.

seq -f"@%.0f" $FS -60 $((FS-7200)) | date -f- +"%F %H:%M 0"

this is the code that changes all value to "0"

12

... which then is modified by the last ( awk ) step in the pipe ...

13

Hi RudiC,

None, i have commented the awk part on the script. Please see details below.

SCIPT

[dev:Desktop]$ cat test.sh 
#!/bin/sh
read FD <test1.txt
FS=$(date -d "${FD% *}" +%s)

seq -f"@%.0f" $FS -60 $((FS-7200)) | date -f- +"%F %H:%M 0"  > ernie.txt
#awk 'NR == FNR {T[$1,$2] = $3
#                 next
#               }
#l
#END            { for (t in T) print t, T[t]
#               }
#' SUBSEP=" "

OUTPUT

[dev:Desktop]$ cat ernie.txt
2016-01-13 01:33 0
2016-01-13 01:32 0
2016-01-13 01:31 0
2016-01-13 01:30 0
2016-01-13 01:29 0
2016-01-13 01:28 0
2016-01-13 01:27 0
2016-01-13 01:26 0
2016-01-13 01:25 0
2016-01-13 01:24 0
2016-01-13 01:23 0
2016-01-13 01:22 0
2016-01-13 01:21 0
2016-01-13 01:20 0
2016-01-13 01:19 0
2016-01-13 01:18 0
2016-01-13 01:17 0
2016-01-13 01:16 0
2016-01-13 01:15 0
2016-01-13 01:14 0
2016-01-13 01:13 0
2016-01-13 01:12 0
2016-01-13 01:11 0
2016-01-13 01:10 0
2016-01-13 01:09 0
2016-01-13 01:08 0
2016-01-13 01:07 0
2016-01-13 01:06 0
2016-01-13 01:05 0
2016-01-13 01:04 0
2016-01-13 01:03 0
2016-01-13 01:02 0
2016-01-13 01:01 0
2016-01-13 01:00 0
2016-01-13 00:59 0
2016-01-13 00:58 0
2016-01-13 00:57 0
2016-01-13 00:56 0
2016-01-13 00:55 0
2016-01-13 00:54 0
2016-01-13 00:53 0
2016-01-13 00:52 0
2016-01-13 00:51 0
2016-01-13 00:50 0
2016-01-13 00:49 0
2016-01-13 00:48 0
2016-01-13 00:47 0
2016-01-13 00:46 0
2016-01-13 00:45 0
2016-01-13 00:44 0
2016-01-13 00:43 0
2016-01-13 00:42 0
2016-01-13 00:41 0
2016-01-13 00:40 0
2016-01-13 00:39 0
2016-01-13 00:38 0
2016-01-13 00:37 0
2016-01-13 00:36 0
2016-01-13 00:35 0
2016-01-13 00:34 0
2016-01-13 00:33 0
2016-01-13 00:32 0
2016-01-13 00:31 0
2016-01-13 00:30 0
2016-01-13 00:29 0
2016-01-13 00:28 0
2016-01-13 00:27 0
2016-01-13 00:26 0
2016-01-13 00:25 0
2016-01-13 00:24 0
2016-01-13 00:23 0
2016-01-13 00:22 0
2016-01-13 00:21 0
2016-01-13 00:20 0
2016-01-13 00:19 0
2016-01-13 00:18 0
2016-01-13 00:17 0
2016-01-13 00:16 0
2016-01-13 00:15 0
2016-01-13 00:14 0
2016-01-13 00:13 0
2016-01-13 00:12 0
2016-01-13 00:11 0
2016-01-13 00:10 0
2016-01-13 00:09 0
2016-01-13 00:08 0
2016-01-13 00:07 0
2016-01-13 00:06 0
2016-01-13 00:05 0
2016-01-13 00:04 0
2016-01-13 00:03 0
2016-01-13 00:02 0
2016-01-13 00:01 0
2016-01-13 00:00 0
2016-01-12 23:59 0
2016-01-12 23:58 0
2016-01-12 23:57 0
2016-01-12 23:56 0
2016-01-12 23:55 0
2016-01-12 23:54 0
2016-01-12 23:53 0
2016-01-12 23:52 0
2016-01-12 23:51 0
2016-01-12 23:50 0
2016-01-12 23:49 0
2016-01-12 23:48 0
2016-01-12 23:47 0
2016-01-12 23:46 0
2016-01-12 23:45 0
2016-01-12 23:44 0
2016-01-12 23:43 0
2016-01-12 23:42 0
2016-01-12 23:41 0
2016-01-12 23:40 0
2016-01-12 23:39 0
2016-01-12 23:38 0
2016-01-12 23:37 0
2016-01-12 23:36 0
2016-01-12 23:35 0
2016-01-12 23:34 0
2016-01-12 23:33 0
14

So - why THIS question, then

?

15

Hi RudiC,

The script is now working. i forgot to place "-" on the script itself.

Thanks.