Unix, a file is asciii or not

UNIX for Advanced & Expert Users
1

Hi,
This query might sound familiar, but I have an issue in finding out whether the file is ascii or not.

The requirement:

  1. Check if the file has any only ascii ,line feed , tab. If it contains any non-ASCII it has to fail.
  2. The file size will go upto 1.5GB in size.
  3. The length of every line is fixed . The number of lines will be upto 1 - 2 million.

What I tried:
I tried the file utility:
-------------------------------------------------------
file "$1" | grep -Fq executable &&
printf "%s is executable.\n" "$1"

and some script suggested in this forum like

ftype=2
file "$1" | /usr/xpg4/bin/grep -Fq ascii && ftype=0
file "$1" |/usr/xpg4/bin/grep -Fq executable && ftype=1
echo $ftype
-------------------------------------------------------
The manual says it will only check for the first 512 bytes. So the above samples aren't working in my case.

Could you please let me know how this can be achieved. Do I need to read every character and check for the ascii value of that character? If yes could you please provide me some sample scripts.

Thanks in advance
Hemanth.

2

man cat
See the "-v" option which highlights non-printable characters with a "^" .
You can then count (wc -l) the number of lines containing non-printable characters.

#!/bin/ksh
FILENAME="my_file_name"
BAD=$(cat -v ${FILENAME}|grep "\^"|wc -l)
if [ ${BAD} -ne 0 ]
then
       echo "Bad file"
       exit
fi
3

If the OP meant true ascii - ie., 7 bits 0 -127 allowed, then cat -v will flag valid characters with a ^. But it flags chars > 127 with M-^

#!/bin/ksh
FILENAME="my_file_name"
cat -v ${FILENAME}|grep -q 'M-\^'  # quit early when we find just one
if [ $? -eq 0 ]
then
       echo "Bad file"
       exit
fi