Excel data automation using shell script

|AGENT_CODE|LOCAL_ACC_NO|SUB_ACC_NO|SHORT_CODE|STATUS|LATEST_STMT_DATE|
|---|---|---|---|---|---|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|SUN-100-T-BFMSP470GBPZZCAP-C|L03|Active|09-Nov-23|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|SUN-100-T-BFMSP470GBPZZCAR-C|L04|Active|09-Nov-23|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|SUN-100-T-BFMSP470GBPZZCAS-C|L05|Active|09-Nov-23|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|SUN-100-T-BFMSP470GBPZZCFD-C|L06|Active|09-Nov-23|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|SUN-100-T-BFMSP470GBPZZCSY-C|L07|Active|09-Nov-23|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|IRVTGB2X-IRVTBEBB-2641418260-CX1847|S01|Dead/Inactive|01-Jan-00|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|IRVTGB2X-IRVTGB2X-2641118260-CX||||
|||||||
||S02|Dead/Inactive|17-Dec-18|||
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CASHFUND-C|MMD-006-062641418260-C|S03|Active|09-Nov-23|
|AS_RSG_IIS_TA|IIS-TA-BFM-SP470GBP-CATHOLICCHARITIESGROWTH&INCOMEFUND-C|SUN-100-T-BFMSP470GBPZZCZI-C|L01|Active|09-Nov-23|

i am having above data in the .csv file. here on line number 9 i am getting unexpectedly row and above rows last 3 columns shifted below empty rows . i want to move that data to row number 18 to 8 and delete the unnecessarily row number 9 and 10 , which will be empty. please help me as i am new here

Could you please consider this on priority

Welcome!

There is no "priority" in this forum.

Please put triple-backticks (markdowns) above and below your code/samples. This time I did it for you.

I guess the original input had fields with embedded newlines, and finally caused the input that you share?
Embedded newlines can be replaced with spaces, see

Now it is a little more difficult...

1 Like

I am not clear on "move that data to row number 18 to 8". Are there further rows -- you only show 13 rows.

Every row has exactly six field columns, so I am not convinced there are embedded newlines or any carriage-return issues. However, rows 2 and 10 have missing data, and row 11 has three fields shifted left (compared to the similar values in row 8, the other "Dead" entry).

Given the consistency of the column count, I suspect the data was munged in the preceding steps before the CSV (actually Bar-separated) was constructed. Perhaps in Excel itself, or by some anomaly in the export process. Do you have intermediate files for those steps?

If this procedure is automation that is going to be repeated, it might be expedient to find the original cause and rectify it, because the next time you run it, the results might be affected differently. So any bespoke fix would not work next time.

If this is a one-off issue, it would (almost certainly) be easier to just edit the file by hand than to write (and test) an edit script to fix up four or five lines.

AGENT_CODE	LOCAL_ACC_NO	SUB_ACC_NO	SHORT_CODE	STATUS	LATEST_STMT_DATE
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	SUN-100-T-BFMSP470GBPZZCAP-C	L03	Active	09-Nov-23
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	SUN-100-T-BFMSP470GBPZZCAR-C	L04	Active	09-Nov-23
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	SUN-100-T-BFMSP470GBPZZCAS-C	L05	Active	09-Nov-23
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	SUN-100-T-BFMSP470GBPZZCFD-C	L06	Active	09-Nov-23
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	SUN-100-T-BFMSP470GBPZZCSY-C	L07	Active	09-Nov-23
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	IRVTGB2X-IRVTBEBB-2641418260-CX1847	S01	Dead/Inactive	01-Jan-00
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	IRVTGB2X-IRVTGB2X-2641118260-CX			
					
	S02	Dead/Inactive	17-Dec-18		
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CASHFUND-C	MMD-006-062641418260-C	S03	Active	09-Nov-23
AS_RSG_IIS_TA	IIS-TA-BFM-SP470GBP-CATHOLICCHARITIESGROWTH&INCOMEFUND-C	SUN-100-T-BFMSP470GBPZZCZI-C	L01	Active	09-Nov-23

need to write script to move data from B10, C10 , D10 to D8 , E8 and F8

@khoses , colleague @MadeInGermany asked you to enclose code/samples in backticks, ignoring requests will not compel the team assist when you ignore requests, please make the minimum of effort and do as requested, tks

1 Like
#Move data
# Use awk to extract data from row 10, column B
data_to_move=$(awk 'NR == 10 {print $2}' "$input_file")

# Use awk to insert the data into row 8, columns D
awk -v data="$data_to_move" 'NR == 8 { $4 = $4 " " data } 1' FS='\t' OFS='\t' "$input_file" > temp_file

# Replace the original file with the modified file
mv temp_file "$input_file"

echo "Data moved successfully from row 10, column B to row 8, columns D in $input_file."

@khoses,
you've been asked to use the required markdown code tags already.
I don't think ignoring the "ask" would get you ANY traction on this thread.
Please spend some time reading through the above link and follow the requested "asks" going forward.
I'd ask the forum members to start ignoring any further posts in this thread unless the "asks" are acted upon.
Thanks

Again I have put the triple-backticks ( ``` ) before and after your code/samples, to make it readable on the Web.

The following embedded awk script works with your sample given in your initial post.
It fills up an array. If a field is empty, it fills it from the first not-empty field of the following line(s).

#!/bin/sh
awk '
  BEGIN {
     FS="|" # Field separator
     offs=1 # Skip first and last field
     ff=1+offs
  }
  {
    # Copy the input fields to inp[] fields
    for (f=1+offs; f <= NF-offs; f++) {
      if (inp[ff] == "" && $f != "") { inp[ff++]=$f }
    }
    if (f == ff) {
      # All inp[] fields filled, print them
      out=sep=""
      for (f=1; f <= NF; f++) {
        out=(out sep inp[f]); sep=FS
      }
      print out
    # Clear inp[]
      split("", inp)
      ff=1+offs
    }
  }
' "$@"

The "$@" passes on the given arguments, so you can run it with
/bin/sh scriptname inputfile.csv
or
/bin/sh scriptname < inputfile.csv

This topic was automatically closed 300 days after the last reply. New replies are no longer allowed.