# Shell script to pull certain fields **URL:** \ **Category:** Shell Programming and Scripting\ **Created:** [October 6, 2016, 5:43pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884 "2016-10-06T17:43:53Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 6, 2016, 5:43pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/1 "2016-10-06T17:43:53Z")
I/m a beginner so be easy. I have text files that live on an AIX server. The files come in and I've been charged with writing a shell script to email out that pulls the first date, and the last date of the file. I need to load these 2 dates into 2 separate variables. I can figure out the variables, but what I'm needing help on is what command to use and what syntax to pull the first and last date/time value for the 14th column in the text file. The columns are separated by a single tilde ~. This part is new to me, so I'd appreciate an experts assistance! Thanks in advance, Matt ---
**Author:** ![pilnet101](https://community.unix.com/user_avatar/community.unix.com/pilnet101/32/2965_2.png) [@pilnet101](https://community.unix.com/u/pilnet101)\ **Post date:** [October 6, 2016, 7:42pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/2 "2016-10-06T19:42:10Z")
What have you tried so far? Can you provide some sample input data/desired output data? ---
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 6, 2016, 9:49pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/3 "2016-10-06T21:49:46Z")
I haven't tried much in regards to printing this desired data. I'm hoping someone could tell me what command would be best to do this task, and possibly what parameters without being exact and doing it for me. Hope that helps. ---
**Author:** ![Chubler\_XL](https://community.unix.com/user_avatar/community.unix.com/chubler_xl/32/2077_2.png) [@Chubler\_XL](https://community.unix.com/u/Chubler_XL)\ **Post date:** [October 6, 2016, 10:28pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/4 "2016-10-06T22:28:52Z")
How are you to determine the "first date" and "last date" from these two files? eg "first date" could be: field #14 from 1st record of file 1 minimum value of field #14 from file 1 and file2 "last date" could be field #14 from last record of file 2 maximum value of field #14 from file 1 and file2 most recent modification time of file1 and file2 ---
**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\ **Post date:** [October 6, 2016, 11:31pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/5 "2016-10-06T23:31:11Z")
In addition to what Chubler\_XL has already said, if the dates to be used are based on the values in field #14 instead of being the dates on the 1st and last records in the file, you have to know the format of the date fields to determine how to perform the comparisons. And, if the values are to be extracted from the 1st and last data records in the file, we have to know if there are any header and/or trailer lines in the file. This is why it is crucial that you give us a representative sample of the file(s) you will be processing. Furthermore, you aren't at all clear about what tools are available for this project. You say the files you want to process are on a server. Will your script be run on that server while you are logged into that server? Will the files be located on one server while the script will be running on a different server? The more details you hide from us, the less help we will be able to provide. ---
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 7, 2016, 10:15am UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/6 "2016-10-07T10:15:11Z")
> [@don cragun](#): > > In addition to what Chubler\_XL has already said, if the dates to be used are based on the values in field #14 instead of being the dates on the 1st and last records in the file, you have to know the format of the date fields to determine how to perform the comparisons. And, if the values are to be extracted from the 1st and last data records in the file, we have to know if there are any header and/or trailer lines in the file. > > This is why it is crucial that you give us a representative sample of the file(s) you will be processing. > > Furthermore, you aren't at all clear about what tools are available for this project. You say the files you want to process are on a server. Will your script be run on that server while you are logged into that server? Will the files be located on one server while the script will be running on a different server? > > The more details you hide from us, the less help we will be able to provide. ---------- Post updated at 09:15 AM ---------- Previous update was at 09:14 AM ---------- Chubler and don, Thanks so much for your reply. I'll upload a sample file today. Thanks a million. ---
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 10, 2016, 4:35pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/7 "2016-10-10T16:35:03Z")
Here you go....i'm wanting to load a variable with the first and last column FirstExternalDateTime's values (last one would be 09032016011500PM) into an existing script. Please let me know if you have any ideas of how I can reference this particular file. See txt file attached. Thanks in advance! ---
**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/5_5575768a8748004e209b776fc1b2916d.png) [@RudiC](https://community.unix.com/u/RudiC)\ **Post date:** [October 10, 2016, 4:54pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/8 "2016-10-10T16:54:41Z")
If we only knew what you are talking about we might have some ideas. Are you talking of rows or of columns? What tools are available and/or preferred (has been asked before)? What OS and shell version are you using? Just some wild guessing: You want to ignore the header line, then from the second line extract field 14 (with ` ~ ` being the field separator) into variable1, and then, from the last line, extract field 14 into variable2. Were this correct, and your shell were a recent `bash` or `ksh` , would this come close to what you want: ```nohighlight read var1 var2 <<< $(awk -F~ 'NR==2 {T = $14} END {print T, $14}' /tmp/datafile.txt) echo $var1, $var2 09032016114500AM, 09032016011500PM ``` ---
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 10, 2016, 5:41pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/9 "2016-10-10T17:41:57Z")
sorry, my terminology may be bad. I think AWK or SED would be the best native tools. I think you hit on what I was after. Just one row under the FirstExternalDateTime's for the first value and the one row under FirstExternalDateTime's last value. Let me take it for a spin and get back to you! ---------- Post updated at 04:41 PM ---------- Previous update was at 03:57 PM ---------- RudiC, First and foremost, THANK YOU...was almost perfect. The provided code returned the following, including the comma: 09032016114500AM, It didn't seem to return the very last row. Any quick thoughts? ---
**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/5_5575768a8748004e209b776fc1b2916d.png) [@RudiC](https://community.unix.com/u/RudiC)\ **Post date:** [October 11, 2016, 3:09am UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/10 "2016-10-11T03:09:31Z")
Which is a bit hard to believe. The comma in my proposal above comes from the `echo` statement, and the two values are from line 2 and the LAST line of the "datafile.txt" that you attached. First thought would be the comma is in the data. Are my assumptions in post#8 correct? WHat results do get applying the proposal to the attached "datafile.txt"? Does your other/new file obey the exactly same structure? ---
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 11, 2016, 2:07pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/11 "2016-10-11T14:07:02Z")
Last question, do you know how I can date format the two variables of var1 and var2 so that they echo in a format such as `date '+%d/%m/%Y_%H:%M:%S'`? ---
**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/5_5575768a8748004e209b776fc1b2916d.png) [@RudiC](https://community.unix.com/u/RudiC)\ **Post date:** [October 11, 2016, 2:13pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/12 "2016-10-11T14:13:03Z")
You still didn't answer the questions for your OS and shell. In `bash` , there's "parameter expansion/Substring Expansion": `${parameter:offset:length}` that could fulfill your requirement. Give it a try and come back with the result, or any problem that might arise. ---
**Author:** ![mattadams1983](https://community.unix.com/letter_avatar/mattadams1983/32/5_5575768a8748004e209b776fc1b2916d.png) [@mattadams1983](https://community.unix.com/u/mattadams1983)\ **Post date:** [October 11, 2016, 3:05pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/13 "2016-10-11T15:05:41Z")
Thanks, sorry. uname -s shows: AIX 00F64BC74C00 Bash 3.00.16(1) ---------- Post updated at 02:05 PM ---------- Previous update was at 01:49 PM ---------- RudiC, Please pardon my lack of experience here. I'm not sure how to use this suggestion to apply a date format `09032016011500PM` into a readable format in a line of code such as ```nohighlight read rdfirst <<< $(awk -F~ 'NR==2 {T = $14} END{print T}' datafile.txt) echo $rdfirst ``` ---
**Author:** ![RudiC](https://community.unix.com/letter_avatar/rudic/32/5_5575768a8748004e209b776fc1b2916d.png) [@RudiC](https://community.unix.com/u/RudiC)\ **Post date:** [October 11, 2016, 3:31pm UTC](https://community.unix.com/t/shell-script-to-pull-certain-fields/362884/14 "2016-10-11T15:31:42Z")
`man bash` : ```nohighlight Parameter Expansion . . . ${parameter:offset} ${parameter:offset:length} Substring Expansion. Expands to up to length characters of the value of parameter starting at the character specified by offset. If parameter is @, an indexed array subscripted by @ or *, or an associative array name, the results differ as described below. If length is omitted, expands to the substring of the value of parameter starting at the character specified by offset and extending to the end of the value. length and offset are arithmetic expressions (see ARITHMETIC EVALUATION below). ``` So, with your above `$rdfirst` variable containing the quoted value, ```nohighlight echo ${rdfirst:2:2}/${rdfirst:0:2}/ 03/09/ ``` shows how to proceed. Give it a try and come back with your results.