Store Large data in Variable in csh

Shell Programming and Scripting
1

Hi All,

This is my first post!! :slight_smile:

How to store huge data in a single variable in csh.

Right now i can store upto ~700kb of data. I still want to store more data and i dont want to use perl.

Is there any method to store like that...Please let me know..

Thanks

2

You shouldn't be doing that in any shell. The maximum is system-dependent and sometimes surprisingly small.

There's almost certainly better ways to do what you want than storing huge things in variables. What are you actually trying to do?

3

Thanks for reply..

I am just storing data from a file to a variable and process each record of that file. But if the file size is huge then it will fail to process the records....So i need to have that variable to store huge data.

If any better ways, let me know..

4

We need more detail to help. Show a sample of the input you have and the output you want. Also, tell us what system you're using so we know what tools you have available. We'll need to see what you're wanting to do, not just how you want to do it, to help.

If you really do need to keep large things in memory at once, csh -- and shells in general -- are simply the wrong tool for the job. A shell is meant to run and join together processes and streams, not process megabytes of raw data by themselves; they're not meant for that and will handle it badly. You might only get 4 kilobytes in some systems and shells, let alone 700! A different language like perl or awk may serve better.

But I'm not convinced this isn't overkill yet. You may be able to handle the records as a stream instead of keeping them all in one blob.