Remove first 2 characters and last two characters of each line

here's what im trying to do.

i have a file containing lines similar to this:

data.txt:

1hsRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTnRU
1lKUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG5k
195Um5kaFZ6VnFURzFPZG1KVFFsQlZhVUp0WTFkU2RVOXVRbTVqClIyZzBZek5TYWxsWVFYZE5S
1wxWTBkb05FeHRaR2hqUjJ4MVdYazFhbUl5TUdkVU1VbG5DbHB1Um10aWFuQjNXak5DYjJWSVRq
22oKTWtaM1RVUkJla3h1UW05bFF6VnVXVmhDY0dKdFRYVlpNamwwU1VVNVUwbEhXbmhhUnpRMlkw
ZGtkMkZJYUhwa1IwNW9ZMFJCZDA1RApOWGNLWVVobmRWb3lSbmRoVnpWcVRHMU9kbUpUUWxCVmFV
66qRZMWRTZFU5dVFtNWpSMmcwWXpOU2FsbFlRWGROUkZWMVkwZG9ORXh0ClpHaGpSMngxV1hrMWFt

I need to remove the first two characters and the last two characters of each line. but i also need to be able to put those removed characters back to where they were before they were removed.

so, here's what i'm doing so far:

sed -e 's ..  ' -e  's ..$  ' data.txt > data.txt.pruned

This only removes the first two characters of each line and the last two.

where im getting stuck is, how do i get back the removed characters?

a thought that crossed my mind is to first get the first two characters of each line line. save it to a file and do the same for the last two characters.

some_magical_command data.txt > firstwo.characters
some_magical_command data.txt > lastwo.characters

then find a way to merge firstwo.characters and lastwo.characters with data.txt.pruned so they look like the original text in data.txt?

i suppose the commands join may be used here. but im looking for something that can work on all Unix systems (portable), especially those systems that are considered "embedded". if awk can be used here, that'll be great.

Hello SkySmart,

Could you please try following and let me know how it goes then, though I am not sure about your complete requirement.

cat script.ksh 
FIRST_TWO=$(awk '{Q=Q?Q "@@" substr($0,1,2):substr($0,1,2)} END{print Q}' file1)
LAST_TWO_CHAR=$(awk '{Q=Q?Q "@@" substr($0,length($0)-1,2):substr($0,length($0)-1,2)} END{print Q}' file1)
REST_CHAR=$(awk '{Q=Q?Q "@@" substr($0,3,length($0)-4):substr($0,3,length($0)-4)} END{print Q}' file1)

echo "Separate ones:"
echo "First 2 chars of each line: " $FIRST_TWO
echo "Last 2 chars of each line: " $LAST_TWO_CHAR
echo "Lines without first 2 chars and last 2 chars:" "$REST_CHAR"


echo "combining the lines again here..."
awk -vfirst_two="$FIRST_TWO" -vlast_two_char="$LAST_TWO_CHAR" -vrest_char="$REST_CHAR" 'BEGIN{num=split(first_two, A,"@@");split(last_two_char, B,"@@");split(rest_char, C,"@@");for(i=1;i<=num;i++){print A C B}}'

So when we are running the script then following will be the output.

./script.ksh 
Separate ones:
First 2 chars of each line:  1h@@1l@@19@@1w@@22@@ZG@@66
Last 2 chars of each line:  RU@@5k@@5S@@Rq@@kw@@FV@@Ft
Lines without first 2 chars and last 2 chars: sRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTn@@KUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG@@5Um5kaFZ6VnFURzFPZG1KVFFsQlZhVUp0WTFkU2RVOXVRbTVqClIyZzBZek5TYWxsWVFYZE@@xWTBkb05FeHRaR2hqUjJ4MVdYazFhbUl5TUdkVU1VbG5DbHB1Um10aWFuQjNXak5DYjJWSV@@oKTWtaM1RVUkJla3h1UW05bFF6VnVXVmhDY0dKdFRYVlpNamwwU1VVNVUwbEhXbmhhUnpRMl@@tkMkZJYUhwa1IwNW9ZMFJCZDA1RApOWGNLWVVobmRWb3lSbmRoVnpWcVRHMU9kbUpUUWxCVm@@qRZMWRTZFU5dVFtNWpSMmcwWXpOU2FsbFlRWGROUkZWMVkwZG9ORXh0ClpHaGpSMngxV1hrMW
combining the lines again here...
1hsRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTnRU
1lKUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG5k
195Um5kaFZ6VnFURzFPZG1KVFFsQlZhVUp0WTFkU2RVOXVRbTVqClIyZzBZek5TYWxsWVFYZE5S
1wxWTBkb05FeHRaR2hqUjJ4MVdYazFhbUl5TUdkVU1VbG5DbHB1Um10aWFuQjNXak5DYjJWSVRq
22oKTWtaM1RVUkJla3h1UW05bFF6VnVXVmhDY0dKdFRYVlpNamwwU1VVNVUwbEhXbmhhUnpRMlkw
ZGtkMkZJYUhwa1IwNW9ZMFJCZDA1RApOWGNLWVVobmRWb3lSbmRoVnpWcVRHMU9kbUpUUWxCVmFV
66qRZMWRTZFU5dVFtNWpSMmcwWXpOU2FsbFlRWGROUkZWMVkwZG9ORXh0ClpHaGpSMngxV1hrMWFt

This is just an example where you could use the first 2 characters, last 2 characters and rest of the line without them in a script itself rather than saving them into files. If you have more requirements then kindly do let us know on same.

Thanks,
R. Singh

What do you mean putting those characters back? You could put the characters back by simply using the original file.

That said, to split the file into there different files, try:

awk -v n=2 '
{
  len=length($0)
  print substr($0,1,n)>left
  print substr($0,n+1,(len-2*n))>middle
  print substr($0,len-n+1)>right
}
' left=file1 middle=file2 right=file3 file 

If you want to combine the files you can use the paste command:

paste -d'\0' file1 file2 file3  

Just for the fun of it, using standard text tools:

tee >(cut -c-2 > left) <file | cut -c3- | rev | tee >(cut -c-2 | rev > right) | cut -c3- | rev > middle

cf left middle right 
left:
1h
1l
19
1w
22
ZG
66
middle:
sRmRsbHRiSFZNTTA1dlEyMWFkbU5wUW5CSlIyeDFTVU5SYjJOSFRuWmpia0ZuWXpKV2FHTn
KUnpWMldrZFZaMG95V25oYQpSelEyWTBka2QyRklhSHBrUjA1b1kwUkJkd3BOVXpWM1lVaG
5Um5kaFZ6VnFURzFPZG1KVFFsQlZhVUp0WTFkU2RVOXVRbTVqClIyZzBZek5TYWxsWVFYZE
xWTBkb05FeHRaR2hqUjJ4MVdYazFhbUl5TUdkVU1VbG5DbHB1Um10aWFuQjNXak5DYjJWSV
oKTWtaM1RVUkJla3h1UW05bFF6VnVXVmhDY0dKdFRYVlpNamwwU1VVNVUwbEhXbmhhUnpRMl
tkMkZJYUhwa1IwNW9ZMFJCZDA1RApOWGNLWVVobmRWb3lSbmRoVnpWcVRHMU9kbUpUUWxCVm
qRZMWRTZFU5dVFtNWpSMmcwWXpOU2FsbFlRWGROUkZWMVkwZG9ORXh0ClpHaGpSMngxV1hrMW
right:
RU
5k
5S
Rq
kw
FV
Ft

Use paste as Scrutinizer did for reconstruction.

perl -nle 'print /^(..).+(..)$/' data.txt > data.txt.pruned

perl -pe '' data.txt.pruned

1hRU
1l5k
195S
1wRq
22kw
ZGFV
66Ft

My suspicion is that this is a private security key and you want to be able to email it safely. My concern is that you will also transmit the way that the key is reconstituted and thereby expose how to reconstitute your private key to anyone snooping your traffic.

There are various ways that this can be done as shown already, and more besides, but I just wanted to check that we are not encouraging you to follow an insecure design.

Please feel free to tell me I'm wrong - I'd be relieved.

Kind regards,
Robin