Joining files present in different directories.

Shell Programming and Scripting
1

Hi Experts,

I have files Bank1 ,Bank2 ,Bank3 with high level customer information in one folder.

/home/Cust_information/
Bank1,Bank2,Bank3,Bank4

The detail level information of all customers of each Bank are present in separate directories .
Here Bank1 is a directory which contains set of files (details,details2,details3) which contains all the information of customers.
Similarly Bank2,Bank3,Bank4 are the folders which contains information of there customers

/home/Bankdetails/Bank1/
details,details2

/home/Bankdetails/Bank2/
details,details2,details3

/home/Bankdetails/Bank3/
details,details2,details3

/home/Bankdetails/Bank4/
details,details2,details3,details4

How to join Bank1 file in /home/Cust_information/ path with all the files present in Bank1 folder
/home/Bankdetails/Bank1/* based on Cust_ID. Same should be done for all the files present in /home/Cust_information/ folder using a loop.

For example Bank1 file in /home/Cust_information/ contains following details

9005	Arun	28	yes
9060	vinay	54	no
9182	john	23	yes

complete details of all the customers are present in /home/Bankdetails/Bank1/ path

details
9000	mary	2000	3000
9005	arun	3000	6000

PFA the folder structure for reference.

details1
9111	gary	4000	5000
9182	john	3300	2500

Here i need to join /home/Cust_information/Bank1 with files in /home/Bankdetails/Bank1/details,details1 to get a consolidated output for matching IDS

output

9005	Arun	28	yes	arun	3000	6000
9182	john	23	yes	john	3300	2500
2

It's easier in SQL than shell, and there are JDBC and ODBC tools that make flat files work like tables. You can 'sort' and 'join' files in shell to get a similar effect. You can join in bash by putting one file into an associative array and then looking up the next file's key fields to merge the output.

3

Give this a shot and come back with detailed results/logs/errors...

set -vx
for BANK in /home/Cust_information/Bank*
  do awk 'NR==1         {print FILENAME}
          NR==FNR       {DATA[$1]=$0; next}
          $1 in DATA    {$0=DATA[$1]" "$0}
          1
         ' "$BANK" /home/Bankdetails/"${BANK##*/}"/details* >> consolidated.out
  done