Table like formatting in Linux

Shell Programming and Scripting
1

Dear experts,

I need bit help in formatting ..

I have csv file file , i will read that file by passing one column value as input parameter and display header row and corresponding row for that parameter.

I have shell script like this:

#!/bin/bash
#key_word_I_am_looking_for=$1
#awk -F: -v keyword="$1" '$1 == keyword {$1=$1; print}' Book1.csv
#my_variable=`cat Book1.csv | awk -F: -v keyword="$key_word_I_am_looking_for" '( $1 == keyword )' END{print "$@" }'`
#echo "$my_variable"

SERVICE=$1
head -n 1 /opt/scripts/devv/book7.csv > message.out
awk '{gsub(/\,/,"       ");print;}' message.out
grep -i $1 /opt/scripts/devv/book7.csv > message1.out
awk '{gsub(/\,/,"          ");print;}' message1.out

I tried to use tab for header line..used same tab as delimiter for second line but second values not aligned with first line...can some one suggest please..

Current out put as below:

SERVICE NAME       TEAM NAME       CI OWNER       SECONDARY CONTACT       PAYCHECK NAME       DEVV       ADEINT       ENV DEV       ENV STAGE       ENV PROD       ENV PROD
checkout          SARD          opalan                        Jef May              abcdef             Yes          Yes          Yes          Yes          Yes          Yes
2

If you want to properly format the output, then I suggest you use printf statement.
For example:

awk -F, '{printf "%13-s %10-s %9-s %18-s %14-s %5-s %7-s %8-s %10-s %9-s %9-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11}' /opt/scripts/devv/book7.csv

Note: the - sign should be behind the procent sign in the above example..

SERVICE NAME  TEAM NAME  CI OWNER  SECONDARY CONTACT  PAYCHECK NAME  DEVV  ADEINT  ENV DEV  ENV STAGE  ENV PROD  ENV PROD 
checkout      SARD       opalan    Jef May            abcdef         Yes   Yes     Yes      Yes        Yes       Yes

Or, combined with TAB's :

awk -F, '{printf "%12-s\t%9-s\t%8-s\t%17-s\t%13-s\t%4-s\t%6-s\t%7-s\t%9-s\t%8-s\t%8-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11}' /opt/scripts/devv/book7.csv

Note: the - sign should be behind the procent sign in the above example..

SERVICE NAME	TEAM NAME	CI OWNER	SECONDARY CONTACT	PAYCHECK NAME	DEVV	ADEINT	ENV DEV	ENV STAGE	ENV PROD	ENV PROD
checkout    	SARD     	opalan  	Jef May          	abcdef       	Yes 	Yes   	Yes    	Yes      	Yes     	Yes
3

Thanks for reply and suggestion..

but pardon me..i tried to implement it using your suggestion..but its displaying all records

changed by code as below. Is this correct ?

SERVICE=$1
head -n 1 /opt/scripts/devv/book7.csv
awk -F, '{printf "%12-s\t%9-s\t%8-s\t%17-s\t%13-s\t%4-s\t%6-s\t%7-s\t%9-s\t%8-s\t%8-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11}' /opt/scripts/devv/book7.csv
grep -i $1 /opt/scripts/devv/book7-1.csv
awk -F, '{printf "%12-s\t%9-s\t%8-s\t%17-s\t%13-s\t%4-s\t%6-s\t%7-s\t%9-s\t%8-s\t%8-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11}' /opt/scripts/devv/book7.csv

sorry if understanding is silly

4

What do you need the SERVICE=$1 for? The SERVICE variable is not used anywhere in your code...

Try to adapt Scrutinizer's code (you were close in your commented out samples):

awk -F, -v keyword="$1" 'NR == 1 || $1 == keyword {printf "%12-s\t%9-s\t%8-s\t%17-s\t%13-s\t%4-s\t%6-s\t%7-s\t%9-s\t%8-s\t%8-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11}' /opt/scripts/devv/book7.csv

Check for the correct field separator.
Please be aware that $1 in shell differs from $1 in awk .

5

Dear RudiC,

Thanks for followup reply.

SERVICE=$1 is input parameter that i want to pass in while executing my shell script
and data for this parameter is available in book.7 csv file
and I shall run program as below
$asp > sh test7.sh checkout -- in this checkout one of value for columna called SERVICE in of that csv file

by passing this service name i should get relevant row from that csv and display it on screen both header and its record..

and now problem is with the formatting.. I want to display it strict tabular format, but header and child record are not aligning together properly

Here is some sample data in that csv: The challenge is actual data is vary in length ..the code that i am trying working (in the sense aligning correctly and for some its not aligning correctly. In true sense the problem Column header should adjust according to rows (i.e data rows) not vice versa.. which i am trying to achieve

SERVICE NAME	TEAM NAME	CI OWNER 	SECONDARY CONTACT	PAYCHECK NAME	DEVV	ADEINT	ENV DEV	  ENV STAGE	ENV PROD	    ENV PROD
accommodation,POSE,MatDon Richardson,Chrivarian,Acc,Yes,Yes,Yes,Yes,Yes,Yes
activitycodemappingtask,DLLARS,Ian,Patrick,Acc,Yes,Yes,Yes,Yes,Yes,Yes
activity-entity,Rega,Chang,John,TER,Yes,Yes,Yes,Yes,Yes,Yes
api-test-runner,WRIGHT,Benson,Lehman,Lev,Yes,Yes,No,No,No,No
assembly-service,Snap,Macdonald,Nikseth,Lev,Yes,Yes,Yes,No,No,Yes
authorizationprocessing,DLLARS,Ian,Pat,Acc,Yes,Yes,Yes,Yes,Yes,Yes

for example ; the below service having slight larger length paycheck name hence format is disturbed. you can see COLUMN Heading SECONADARY CONTACT NOT ALIGNED, PAYCHECK VALUE not aligned properly, However in CSV file data does not have any extra spaces it looks perfect like above sample data.

SERVICE NAME	TEAM NAME	CI OWNER 	SECONDARY CONTACT	PAYCHECK NAME	DEVV	ADEINT	ENV DEV	  ENV STAGE	ENV PROD	    ENV PROD
     cmc                   MULTIPLE     Timothy sutherland     Wills Robinson                              TER or LEV depending on plugin  Yes     Yes     Yes     Yes             Yes             Yes
6

Try:

awk -F'\t *' '{printf "%25-s%16-s%20-s%26-s%16-s%8-s%10-s%14-s%10-s%10-s%-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11} NR==1{FS=","}' file

Note: the - sign should be behind the procent sign in the above example..

7

Hi..

Thanks for reply..

now it better..but not completely

I see two problems now:

  1. headers row not aligned
  2. Most of data rows are aligned..i see only problem in those rows where SERVICE column data is having huge length that why its pushing other columns..the max length of this column is 40....

after position 40 the rest of the columns to be printed .. how can i force to print 2nd column onwards after 40th position

8

Well then try enlarging the first column to 40:

awk -F'\t *' '{printf "%40-s%16-s%20-s%26-s%16-s%8-s%10-s%14-s%10-s%10-s%-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11} NR==1{FS=","}' file

--edit--
As RudiC pointed out, the minus sign is in the wrong place, even though some awks seem to be tolerant of the different position, it is not correct..

awk -F'\t *' '{printf "%-40s%-16s%-20s%-26s%-16s%-8s%-10s%-14s%-10s%-10s%-s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11} NR==1{FS=","}' file

--
OR:, try making it flexible by first determining each max column width in the entire file and then read and process the file again using the max column widths:

awk '{for(i=1;i<=NF;i++){if(NR==FNR){l=length($i); if(l>W) W=l} else $i=sprintf("%-*s",W+1,$i)}}1' FS='\t *|,' OFS= file file

SERVICE NAME            TEAM NAME CI OWNER          SECONDARY CONTACT PAYCHECK NAME DEVV ADEINT ENV DEV ENV STAGE ENV PROD ENV PROD 
accommodation           POSE      MatDon Richardson Chrivarian        Acc           Yes  Yes    Yes     Yes       Yes      Yes      
activitycodemappingtask DLLARS    Ian               Patrick           Acc           Yes  Yes    Yes     Yes       Yes      Yes      
activity-entity         Rega      Chang             John              TER           Yes  Yes    Yes     Yes       Yes      Yes      
api-test-runner         WRIGHT    Benson            Lehman            Lev           Yes  Yes    No      No        No       No       
assembly-service        Snap      Macdonald         Nikseth           Lev           Yes  Yes    Yes     No        No       Yes      
authorizationprocessing DLLARS    Ian               Pat               Acc           Yes  Yes    Yes     Yes       Yes      Yes

NOTE: the file is specified twice

9

Thank you for reply

Now most the rows are finely place..except few i will adjust position like you suggested.

only problem i see now is the header is row not aligned at all

it just normally printed seperated by commas

10

You're welcome. Since it appears to work with the sample header row:
How does your real life header row differ from your sample header row.

11

In post#5' sample, the header's field separators seem to be <TAB> characters, in the data line it's commas...

12

@RudiC: Yes that is what posts #6 and #8 take into account.

13

Those who can read ... I overlooked the redefinition of FS after the first line. Sorry!

Still I'm afraid the format specifiers should read %-40s (leading minus, not trailing) as correctly used in your post#8's second proposal.

1 Like
14

You are correct. It should be the other way around according to specification. However some awks (mine) seem to be tolerant to the switcheroo! Corrected in my posts.

15

Hi all,

Thanks for all your patience and help

Its looking bit great from both header and rows perspective

Ideally there should be a standadised package on linux which should do the prettify the csv to table kind of stuff :frowning: that would really help .. do we have such tools in unix ?

On screen everything looks good.. I got to post these results into slack channel..which is printing all things ugly..(however that is different story)

I can tell them out my onscreen output is too good

Thanks for all your inputs

16

You're welcome. That the slack channel prints ugly may have to do with the use of proportional vs. monospace fonts (hint: You get the monospace font by using "code tags" on this forum ;))

17

Hi.

We use code align for tasks like this. It automatically aligns, allows more spacing (gutter), etc. Here's an example:

#!/usr/bin/env bash

# @(#) s1       Demonstrate automatic field alignment, align.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
em() { pe "$*" >&2 ; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C align

FILE=${1-data2}

pl " Input data file $FILE, sample, show TABs, clipped to 78"
cat -A $FILE |
cut -c 1-78 |
head

GUTTER=3
pl " Results, spaces for (right) adjustment, gutter=$GUTTER:"
align -st -j_ -ar -g$GUTTER $FILE

exit 0

producing:

$ ./s1 

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-5-amd64, x86_64
Distribution        : Debian 8.9 (jessie) 
bash GNU bash 4.3.30
align 1.7.0

-----
 Input data file data2, sample, show TABs, clipped to 78
SERVICE NAME^ITEAM NAME^ICI OWNER^ISECONDARY CONTACT^IPAYCHECK NAME^IDEVV^IADE
checkout^ISARD^Iopalan^IJef May^Iabcdef^IYes^IYes^IYes^IYes^IYes^IYes$

-----
 Results, spaces for (right) adjustment, gutter=3:
SERVICE NAME   TEAM NAME   CI OWNER   SECONDARY CONTACT   PAYCHECK NAME   DEVV   ADEINT   ENV DEV   ENV STAGE   ENV PROD   ENV PROD
    checkout        SARD     opalan             Jef May          abcdef    Yes      Yes       Yes         Yes        Yes        Yes

Here are some details about align :

align   Align columns of text. (what)
Path    : ~/p/stm/common/scripts/align
Version : 1.7.0
Length  : 270 lines
Type    : Perl script, ASCII text executable
Shebang : #!/usr/bin/perl
Help    : probably available with --help
Home    : http://kinzler.com/me/align/ (doc)
Modules : (for perl codes)
 Getopt::Std    1.10

Best wishes ... cheers, drl