Hi -
I'm having a problem with hidden characters on Linux. When I produced an output from Oracle database, there is a an extra "Hidden Character" included on the output. How can I remove that character? See below:
The extra dollar sign is creating a new line on my .csv output file. I tried using sed, but no luck.
Note: The '$' is not being included on HP-UX machines upon my query.
Hi
sed -e 's/InterUnit\$/InterUnit/g' file.csv > file2.csv
Guru
Try this..
tr -cd '\11\12\40-\176' < myfile1 > myfile2
Hi guru,
Thanks for the fast response but my problem is not solved using sed.
I tried using your command but the Linux still cannot interpret the hidden '$' even if we used a escape character. I can only view the '$' using the "set list" function of vi, but I cannot grep the string using a normal vi.
Also, I tried viewing the file using 'od -c' and the output is below:
The '$' is interpreted as "next line".
---------- Post updated at 03:23 PM ---------- Previous update was at 03:19 PM ----------
Hi itkamaraj,
Can you please explain what the command is doing? I can't see the '$' in there.
UPDATE: I tried the command tr -cd command, but the '$' is still there.
The code i provided is to remove the special characters from the file
Dec value 128 to 225
Hex value 80 to FF
Is the output produced in windows platform ?
try
dos2unix < infile > outf
perl -i -pe 's/\r//g' inputfile
The files will be generated by the script on Linux box, then send the file as attachment to MS Outlook users.
I tried both of the commands, but the '$' is still there. Am I the only one experiencing this?
just try this.
This will remove all the special characters except the keyboard characters
perl -i -pe 's/[^\x20-\x7f]//g' examplefile
---------- Post updated at 01:16 PM ---------- Previous update was at 01:07 PM ----------
Is the file generated by perl script ?
did u use chomp method for each variable to remove the new line character ?
The problem with removing ALL non-keyboard characters is that the needed hidden characters will be deleted too.
The file is generated from mysql, then dumped to a file. The file will be sed to replace the comma with |. The | is the colseparator on the mysql.
I don't know much about chomp. I'll be researching on that.
---------- Post updated at 05:50 PM ---------- Previous update was at 05:33 PM ----------
This one worked. But it removed all the Hidden Characters.
I just want to remove the hidden character on 'InterUnit$' string and its result from sql query.
InterUnit column will give a Y or N result from the database.
---------- Post updated 06-24-11 at 02:40 PM ---------- Previous update was 06-23-11 at 05:50 PM ----------
Hi all, I am now facing a more complicated problem with hidden characters '$'.
I noticed that the '$' characters are printed in a particular pattern resulting for my CSV file to produce incorrect format upon the execution of the ksh on Linux.
Do you guys know how to remove unnecessary hidden characters on Linux in ksh?
There are unnecessary '$' characters on my output file, but not all '$'. As we know, '$' is like \n on hidden character when we used "set list" in vi.
If my question is unclear, please let me know how could I elaborate it better.
how you are taking dump (using which command) from mysql DB ?
what is your email command ? (which sends the file as attachment ) ?
I have a function to call the run_sql.ksh script using sqlplus.
Whenever I call the function, I dump the result on a $LOGFILE and also spool it on ${file}.csv
Now, the CSV file will be "sed" using the following commands:
#Remove commas seen on all results
sed 's/,/ /g' ${file}.csv > $CSV_RESULT_PIPE
#Change pipes delimiter to comma
sed 's/|/,/g' $CSV_RESULT_PIPE > $CSV_RESULT_UNZIPPED
The $CSV_RESULT_UNZIPPED file will be compressed to ${CSV_RESULT}
Then the following command:
uuencode ${CSV_RESULT} ${CSV_FILE_ZIPPED} > $EMAIL_ATTACHMENT
Then send the email using:
cat $EMAIL_HEADER | /usr/sbin/sendmail -t $RECIPIENTS
Note that the $EMAIL_HEADER includes the $EMAIL_ATTACHMENT, and I can receive the email successfully but the data in it are misplaced.
ok, in the inital csv file itself the $ value appears ?
${file}.csv # is the file having $value ?
#Remove commas seen on all results
sed 's/,/ /g' ${file}.csv > $CSV_RESULT_PIPE
Yes, it has. The value is the real file name to be generated. Please consider file=CSV_FILE:
export CSV_FILE=$PATH/MI_RESULT_OF_QUERY_DATETIME
The file is successfully generated and received but the content is not aligned properly.
Please describe what you see, mentioning what program you are using to view the file. If it is the "stairs" effect it will almost certainly be due to the line terminator.
This happens when you send a unix format text file to a MSDOS platform.
In unix the line terminator is just a linefeed character. This is conventionally represented as "$" or "\n" in unix.
In MSDOS the line terminator is two characters: carriage-return then linefeed.
You will probably need to convert the file to MSDOS format before using "uuencode".
We need to know what Operating System and version you have.
Where supplied, the program is variously called "ux2dos" or "unix2dos". Otherwise the conversion can be achieved with "awk". In your case you could presumably handle the whole conversion (including the CSV bit) in SQL.
deleted
wrong data
I found out that it's because of a '$' being generated automatically; which I do not know how or why.
The code that I've used is shown above. Post #10
Our OS is Linux with x86_64 GNU/Linux version. (Is this correct? I generated it using uname -a command)
I saw the unnecessary $ upon using vi, set list command. The script is creating an extra '$' character after 1100 characters.
---------- Post updated at 05:11 PM ---------- Previous update was at 09:40 AM ----------
Hi, I got a preview of what's the problem by viewing the entire output/file using UltraEdit.
The problem is that the csv is inserted with '$' hidden charactr every after 1100 characters. Say for example: 1 character = 1 column, column 1101 is inserted with '$' character whereas I still have data to print after that column. I can't remove all the '$', I need it for proper printing of rows.
Does your Oracle SQL*Plus program contain a "set linesize" command?
Do you have default values set for SQL*Plus which might include a "set linesize" command? Look for login.sql in your home directory or glogin.sql in the Oracle tree.
$ORACLE_HOME/sqlplus/admin/glogin.sql
Oh, that's it! The linesize on sql.
Thank you so much!