Read a large file

Shell Programming and Scripting
1

Dear Experts,

Would like to know if there's a way if we can read a large file(like 1-2GB) using a shell command or a script and that can show the output as well of the contents of the file in the CLI?
Will appreciate your response. Thank you

2

There are hundreds of ways to read a text file in a shell. The definition of a text file varies from operating system to operating system. The ways to read a text file vary from shell to shell.

What operating system are you using?

What shell are you using?

Is the file you want to read a text file? If it contains binary data or one or more lines containing more than LINE_MAX bytes, your options for reading the file and displaying its contents in a readable form will be much more limited.

Of course, you could write your own CLI in C or C++ and make it read any type of file and print it in whatever format you desire.

3

The file is text file containing texts separated by new lines but only thing is file size is around more than a GIG, the OS is Red hat linux and shell is Bash, please do let us know how to approach on this?

4

If you want to display it on your screen with a new line from the file being displayed every second, try something like:

while read -r line
do	printf '%s\n' "$line"
	sleep 1
done < filename

If you're a very fast reader with nimble fingers and you want to print the file to your terminal as quickly as possible stopping once-in-a-while when you see something interesting going by, use:

cat filename

and while it is printing use ctl-s (i.e. hit the control key and the s key at the same time) to temporarily stop the output and ctl-q to restart the output.

If you want to display the output a page at a time, use:

less filename

or:

more filename

If you want to skip around in the file searching for certain patterns and you're used to vi editing commands, use:

view filename

(note that view is a link to vi that opens its file operands read-only instead of read-write).

If you're looking for certain patterns in your file and only want to see lines that contain that pattern, use grep .

If you're trying to do something else, use or write a utility, that does what you want... ... ...

1 Like
5

Appreciate and thanks very much Don, let me try those suggestions.

6

Could you please explicate what would be meant by