Hello. I'm ingesting files from one system (db hosted on Solaris 10) to another (db hosted on Solaris 9). Files come in zipped, and contain various txt files, which I'll use SQL*Loader to load.
The unzipping, loading etc. is all handled in a ksh shell script. Sadly, we use Putty for all our unix work (and that's part of the problem I think). The files normally arrive at the target host in pairs. When I try to unzip the files, a load of gibberish is sent to the screen. Like this:
mv: cannot access C/$tf_peo_deals_1to1_20100121080000.txt�Z[o�8~`�{2H�][m8��v:͡�L��-��~�d���N�O:<�(��|<Wi:n&�δ7�z]9�e9��f_O��d�G3���h8
(that's only a fragment. The whole thing is like Lord of the Rings.)
This is a problem as we have associated trace files which our ops guys need to look at, and they're filling up to monstrous sizes with these strange characters. At a guess, I'd say this is due to differing character sets on the source and target machines, but I haven't a clue about how to resolve this.
Has anyone seen this before? Any feedback would be gratefully received.
Strangely, I don't see this problem when there is just one file to process, which is sad, as that's never the case.
Thanks,
Ray
Can you go into the directory where the files reside and do a ls -l on the zip files?
PuTTY's a pretty good terminal actually. I'd suspect your shell script first.
Are you using putty to connect to a serial terminal by any chance? Or are you making network connections?
@incredible: yes, the ls -l looks fine:
-rw-r--r-- 1 fed_cntl appl 6326 Feb 18 08:56 tf_mna_deals_ingest_20100104080000.zip
-rw-r--r-- 1 fed_cntl appl 9235 Feb 18 08:56 tf_peo_deals_ingest_20100121080000.zip
-rw-r--r-- 1 fed_cntl appl 46 Feb 17 14:43 xyz.zip
The issue only seems to happen when I unzip more than one file, then all kinds of stuff gets sent to the screen (and my output files).
---------- Post updated at 09:11 AM ---------- Previous update was at 09:04 AM ----------
@Corona688: we're using Putty to make network connections. Does that narrow down the potential cause of the issue?
Can you capture this output now once all the below steps have been executed?Thx
Login as the user
chmod 744 .zip
unzip \.zip > /dev/null
Yes, please see below. Please note that xyz.zip is not a real zip file, but a file I put there for test purposes. The unzip command seems to have worked ok.
dtcd-cnrnda1:fed_cntl $ chmod 744 *.zip
dtcd-cnrnda1:fed_cntl $ unzip \*.zip > /dev/null
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
note: xyz.zip may be a plain executable, not an archive
2 archives were successfully processed.
1 file had no zipfile directory.
dtcd-cnrnda1:fed_cntl $ ls
tf_mna_deals_1to1_20100104080000.txt tf_peo_deals_1to1_20100121080000.txt
tf_mna_deals_acqtech_20100104080000.txt tf_peo_deals_audit_20100121080000.txt
tf_mna_deals_afinadv_20100104080000.txt tf_peo_deals_delete_20100121080000.txt
tf_mna_deals_alegal_20100104080000.txt tf_peo_deals_eventhist_20100121080000.txt
tf_mna_deals_consid_20100104080000.txt tf_peo_deals_ingest_20100121080000.zip
tf_mna_deals_delete_20100104080000.txt tf_peo_deals_law_20100121080000.txt
tf_mna_deals_ingest_20100104080000.zip tf_peo_deals_mgr_20100121080000.txt
tf_mna_deals_source_20100104080000.txt tf_peo_deals_price_20100121080000.txt
tf_mna_deals_targetstock_20100104080000.txt tf_peo_deals_source_20100121080000.txt
tf_mna_deals_tfinadv_20100104080000.txt xyz.zip
tf_mna_deals_tlegal_20100104080000.txt
---------- Post updated at 09:56 AM ---------- Previous update was at 09:44 AM ----------
In fact, looping through the directory contents seems to cause the error too, consider this simple script:
#!/usr/bin/ksh
export FEEDS=/opt/appl/feeds
export inputDir=${FEEDS}/nda_tf_deals
for filename in $(ls ${inputDir}/arrival/ | grep -v "tf_"*.zip);
do
echo "Filename ${filename} is not valid - moving to unproc directory"
done
for filename in $(ls ${inputDir}/arrival/tf_*.zip);
do
echo "Filename ${filename} is valid - so extract txt files from zip file"
done
which, when given the directory contents:
test.ksh
tf_mna_deals_ingest_20100104080000.zip
tf_peo_deals_ingest_20100121080000.zip
xyz.zip
should output messages saying the tf_* files are valid, and the others aren't. Instead, I get this (fragment):
Filename Po6<X2��� is not valid - moving to unproc directory
Filename C/$tf_peo_deals_1to1_20100121080000.txt�Z[o�8~`�{2H�][m8��v:͡�L��-��~�d���N�O:<�(��|<Wi:n&�δ7�z]9�e�"&r_O��d�G3���h8{�9��f�&���z"��ɴ7x7�s=�Mof�,���z�g��4�Q���Yo2�6} >Hg:7JT]4�q'�v.�G�Q{\ is not valid - moving to unproc directory
Filename ���{��S{�Xr��ݿq4�7�XQod���}�c��n>} is not valid - moving to unproc directory
Filename X:���f;sB6 is not valid - moving to unproc directory
Filename +�m7��@v is not valid - moving to unproc directory
But in your script, I don't see anywhere that you're trying to "unzip" ??
In your do loop, can you try omitting the "ls" command?
I checked this again - looks like the culprit is actually the grep command, probably nothing to do with unzip.
Doing this:
grep -v tf*zip
is what causes the funny output. ls is fine.
I can use egrep -sv to suppress output, but I do need to see on screen which files were invalid.
Can you think of why grep does this?
The
grep -v "tf_"*.zip
statement is bogus, i.e. doesn't do what it was supposed to. That should be:
grep -v "tf_.*.zip"
Oh dear Lord, you're right. Tried it and the problem has disappeared.
Thanks everyone.