Oracle table extract: all columns are not converting into pipe delimited in flat file

Shell Programming and Scripting
1

Hi All,

I am writing a shell script to extract oracle table into a pipe dilemited flat file. Below is my code and I have attached two files that I have abled to generate so far.

  1. Table.txt ==> database extract file
  2. flat.txt ==> pipe delimited after some manipulation of the original db extract (Table.txt) file.

My challenges:

The table I am extracting has 13 columns, I want all the columns (null or not null) to be pipe delimited. Problem is that after the last column that is not null (for example say 9th column out of 13th) in the flat file is not coming out with pipe '|', rather it shows up as empty space. I am struggling with this and need your kind help to resolve the issue . This is very urgent and I need to have the code ready by next week. Thank you in advance for your help!


#!/bin/ksh

FILE="Table.txt"
FLAT_FILE="flat.txt"
sqlplus -s user/password@DB <<EOF

SET HEADING OFF
SET PAGESIZE 50000
SET COLSEP "|"
SET LINESIZE 200
SPOOL $FILE
select * from activity;
SPOOL OFF
EXIT
EOF
sed -e '/^\s*$/d' -e 's/^ *//' -e 's/ *| */|/g' $FILE > $FLAT_FILE

Attached Files
Table.txt (62.4 KB, 0 views)
flat.txt (2.3 KB, 0 views)

2 
#!/bin/ksh

FILE="Table.txt"
FLAT_FILE="flat.txt"
sqlplus -s user/password@DB <<EOF > $FILE
SET HEADING OFF
SET PAGESIZE 0
SET COLSEP "|"
SET LINESIZE 32767
select * from activity;
EXIT;
EOF

sed -e 's/ *|/|/g' -e 's/| */|/g' $FILE > $FLAT_FILE
3

There's no need for two substitute commands in the sed command. It can be done with one as in:

sed 's/ *| */|/g' "$FILE" > "$FLAT_FILE"
1 Like
4

I forgot that I am using *
Thanks for the suggestion

5

Thank you very much SriniShoo and Don for your kind help! The trick was set line size to max value 32767.

However, I am still facing two issues...

  1. The SQL result still showing up in the console, which is not the best practice I would say. Below are the SQL commands in the TABLE_NAME.sql that I am passing as a parameter in the shell script. Any idea why it is doing what? What is missing in my code?
TABLE_NAME.sql
SET ECHO OFF
SET FEEDBACK OFF
SET HEADING OFF
SET PAGESIZE 0
SET COLSEP "|"
SET LINESIZE 32767
SET TRIMSPOOL ON
SPOOL $OUTPUT
SELECT * FROM TABLE_NAME;
SPOOL OFF
EXIT;
EOF
  1. Is there a way to pass the SPOOL $OUTPUT (file name) from the shell script to the TABLE_NAME.sql file? When I am running the script as flat_file.ksh TABLE_NAME.sql it did not recognize it because TABLE_NAME.sql is not able to get anything from the flat_file.ksh script.
flat_file.ksh
#!/bin/ksh
 
db_connection="$DBUSER/$DBPWD@$ORA_SID"

SQL_FILE=$1
TABLE_NAME=`echo $SQL_FILE | awk -F"." '{print $1}'`

OUTPUT="${TABLE_NAME}.dat"
 
sqlplus -s $db_connection << EOF @TABLE_NAME.sql

---------- Post updated 04-29-14 at 07:44 PM ---------- Previous update was 04-28-14 at 08:55 PM ----------

I found a workaround for the console display issue..

 sqlplus -s $db_connection << EOF @TABLE_NAME.sql > /dev/null
6

If null fields still a problem you can bypass the problem easily setting a fixed value for them:

SET NULL ''