# Script Shell Extract substring

**URL:** <https://community.unix.com/t/script-shell-extract-substring/353632>\
**Category:** Shell Programming and Scripting\
**Created:** [June 27, 2015, 10:03am UTC](https://community.unix.com/t/script-shell-extract-substring/353632 "2015-06-27T10:03:13Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![chercheur111](https://community.unix.com/letter_avatar/chercheur111/32/6_449c533f586939b6e5f246632a46421f.png) [@chercheur111](https://community.unix.com/u/chercheur111)\
**Post date:** [June 27, 2015, 10:03am UTC](https://community.unix.com/t/script-shell-extract-substring/353632/1 "2015-06-27T10:03:13Z")

</div>

Hi all,

Please, i'd like to extract string just before '.fr'.  
Here is some lines of my file:

```nohighlight
g-82.text.text1.fr.worker1
g-xx.yyyyyy.zzzz.fr.worker2

```

i'd like to extract this text:

```nohighlight
g-82.text.text1
g-xx.yyyyyy.zzzz

```

Please, which command i have to use in my script shell ?

Thank you much.  
Best Regards.

---

<div class="post-metadata">

**Author:** ![sea](https://community.unix.com/user_avatar/community.unix.com/sea/32/2946_2.png) [@sea](https://community.unix.com/u/sea)\
**Post date:** [June 27, 2015, 11:44am UTC](https://community.unix.com/t/script-shell-extract-substring/353632/2 "2015-06-27T11:44:34Z")

</div>

Hi, if its bash, this should work fine:

```nohighlight
while read input
do echo "${input/.fr.*}"
done<file

```

Prints:

```nohighlight
g-82.text.text1
g-xx.yyyyyy.zzzz

```

hth

EDIT:  
And to illustrate the substitution:

```nohighlight
PATTERN=".fr."
VARIABLE="g-xx.yyyyyy.zzzz.fr.worker2"
echo "${VARIABLE/$PATTERN}"

```

Using a single ` /` in `${VARIABLE}` , removes the following pattern, including `*?.` expansion, unless quoted or escaped  
Using two `//` will replace the the 1nd pattern string with the 2nd pattern string.  
Example:

```nohighlight
echo "${VARIABLE/$SEARCH/$REPLACE}"

```

However, this will only apply to the first found occourence!

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 27, 2015, 3:44pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/3 "2015-06-27T15:44:47Z")

</div>

And, with any POSIX-conforming shell (e.g., `ash` , `bash` , `dash` , `ksh` , etc.) you could use:

```nohighlight
while read -r input
do	echo "${input%.fr.*}"
done<file

```

or with `sed` :

```nohighlight
sed 's/[.]fr[.].*//' file

```

or with `awk` :

```nohighlight
awk '{sub(/[.]fr[.].*/, "")}1' file

```

or ...

---

<div class="post-metadata">

**Author:** ![Mannu2525](https://community.unix.com/letter_avatar/mannu2525/32/6_449c533f586939b6e5f246632a46421f.png) [@Mannu2525](https://community.unix.com/u/Mannu2525)\
**Post date:** [June 27, 2015, 4:32pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/4 "2015-06-27T16:32:05Z")

</div>

You could use :

```nohighlight
sed 's/\(.fr.*\)//g' file

```

HTH

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 27, 2015, 5:05pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/5 "2015-06-27T17:05:17Z")

</div>

> [@mannu2525](#):
>
> You could use :
> 
> ```plaintext
> sed 's/\(.fr.*\)//g' file
> 
> ```
> 
> HTH

No.

With an input line like:

```plaintext
g-xx.Africa.zzzz.fr.worker3

```

the above `sed` command would produce the output:

```plaintext
g-xx.

```

instead of the requested:

```plaintext
g-xx.Africa.zzzz

```

In a filename matching pattern, `.` matches a period. But in a basic regular expression (which is what is used in `sed` ), an unadorned `.` matches any single character.

---

<div class="post-metadata">

**Author:** ![Aia](https://community.unix.com/user_avatar/community.unix.com/aia/32/917_2.png) [@Aia](https://community.unix.com/u/Aia)\
**Post date:** [June 27, 2015, 5:26pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/6 "2015-06-27T17:26:23Z")

</div>

> [@mannu2525](#):
>
> You could use :
> 
> ```plaintext
> sed 's/\(.fr.*\)//g' file
> 
> ```
> 
> HTH

A few things you might want think about:  
`\(\)` : you are telling to capture the matched part. Since you are not using it, it is a waste of effort.  
`g` : matching as many times as possible in a line, is not the intention in this case.

---

<div class="post-metadata">

**Author:** ![senhia83](https://community.unix.com/letter_avatar/senhia83/32/6_449c533f586939b6e5f246632a46421f.png) [@senhia83](https://community.unix.com/u/senhia83)\
**Post date:** [June 27, 2015, 6:15pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/7 "2015-06-27T18:15:05Z")

</div>

How about

```nohighlight
$ cat > tmp
g-82.text.text1.fr.worker1
g-xx.yyyyyy.zzzz.fr.worker2

$ awk -F".fr" '{print $1}' tmp
g-82.text.text1
g-xx.yyyyyy.zzzz

```

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 27, 2015, 6:30pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/8 "2015-06-27T18:30:08Z")

</div>

> [@senhia83](#):
>
> How about
> 
> ```plaintext
> $ cat > tmp
> g-82.text.text1.fr.worker1
> g-xx.yyyyyy.zzzz.fr.worker2
> 
> $ awk -F".fr" '{print $1}' tmp
> g-82.text.text1
> g-xx.yyyyyy.zzzz
> 
> ```

This suffers from exactly the same problem noted in post #5 in this thread!

---

<div class="post-metadata">

**Author:** ![senhia83](https://community.unix.com/letter_avatar/senhia83/32/6_449c533f586939b6e5f246632a46421f.png) [@senhia83](https://community.unix.com/u/senhia83)\
**Post date:** [June 27, 2015, 6:44pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/9 "2015-06-27T18:44:41Z")

</div>

This might work?

```nohighlight
$ cat > tmp
g-82.text.text1.fr.worker1
g-xx.yyyyyy.zzzz.fr.worker2
g-xx.Africa.zzzz.fr.worker3

$ awk -F"[.]fr" '{print $1}' tmp
g-82.text.text1
g-xx.yyyyyy.zzzz
g-xx.Africa.zzzz

```

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 27, 2015, 7:02pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/10 "2015-06-27T19:02:09Z")

</div>

> [@senhia83](#):
>
> This might work?
> 
> ```plaintext
> $ cat > tmp
> g-82.text.text1.fr.worker1
> g-xx.yyyyyy.zzzz.fr.worker2
> g-xx.Africa.zzzz.fr.worker3
> 
> $ awk -F"[.]fr" '{print $1}' tmp
> g-82.text.text1
> g-xx.yyyyyy.zzzz
> g-xx.Africa.zzzz
> 
> ```

No. Try adding an input line:

```plaintext
g-xx.freezer.zzzz.fr.worker4

```

You have to match both periods; not just the first one. But, it would work if you used:

```plaintext
-F'[.]fr[.]'
   or 
-F'\\.fr\\.'

```

instead of:

```plaintext
-F"[.]fr"

```

---

<div class="post-metadata">

**Author:** ![Peasant](https://community.unix.com/user_avatar/community.unix.com/peasant/32/2251_2.png) [@Peasant](https://community.unix.com/u/Peasant)\
**Post date:** [June 28, 2015, 1:26am UTC](https://community.unix.com/t/script-shell-extract-substring/353632/11 "2015-06-28T01:26:19Z")

</div>

Perhaps a sed like :

```nohighlight
user@host:~/unix$ ls g*
g-82.text.text1.fr.worker1 g-xx.yyyyyy.zzzz.fr.worker2
g-xx.Africa.zzzz.fr.worker3
user@host:~/unix$ ls g* | sed "s/\(.*\)\(.fr.*\)/\1/g"
g-82.text.text1
g-xx.Africa.zzzz
g-xx.yyyyyy.zzzz

```

So `\1` is what you want and `\2` (which is not used) is the extension being removed from the ls output.

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 28, 2015, 2:53am UTC](https://community.unix.com/t/script-shell-extract-substring/353632/12 "2015-06-28T02:53:14Z")

</div>

> [@peasant](#):
>
> Perhaps a sed like :
> 
> ```plaintext
> user@host:~/unix$ ls g*
> g-82.text.text1.fr.worker1 g-xx.yyyyyy.zzzz.fr.worker2
> g-xx.Africa.zzzz.fr.worker3
> user@host:~/unix$ ls g* | sed "s/\(.*\)\(.fr.*\)/\1/g"
> g-82.text.text1
> g-xx.Africa.zzzz
> g-xx.yyyyyy.zzzz
> 
> ```
> 
> So `\1` is what you want and `\2` (which is not used) is the extension being removed from the ls output.

In post #3 in this thread, I suggested simple, clear shell, `sed` and `awk` scripts that did exactly what was requested. Since then there have been several proposals that, although they work with the sample provided, would not work in the general case. Peasant's proposal above is more complicated than the `sed` suggestion I made earlier:

```plaintext
sed 's/[.]fr[.].*//' file

```

and, it works fine as long as the final string after the `.fr.` in the filenames being processed is literally `worker` followed by a string of digits. But, it fails if the two characters `fr` can ever appear anywhere in the filenames after the `.fr.` such as:

```plaintext
g-82.text.text1.fr.Alfred_Jones

```

With the sparse specification of the allowed filename format, we don't know if this is a problem in this case or not.

---

<div class="post-metadata">

**Author:** ![Mannu2525](https://community.unix.com/letter_avatar/mannu2525/32/6_449c533f586939b6e5f246632a46421f.png) [@Mannu2525](https://community.unix.com/u/Mannu2525)\
**Post date:** [June 30, 2015, 1:44am UTC](https://community.unix.com/t/script-shell-extract-substring/353632/13 "2015-06-30T01:44:03Z")

</div>

Thanks [Don Cragun](http://www.unix.com/members/302122727.html) and [Aia](http://www.unix.com/members/302041415.html) for clarification.

I have a doubt - If the text file is like -

```plaintext
cat file.txt
g-xx.fr.zzzz.fr.worker3
g-82.text.text1.fr.worker1
g-xx.yyyyyy.zzzz.fr.worker2

```

then -

```plaintext
sed 's/[.]fr[.].*//' file.txt

```

will result -

```plaintext
g-xx
g-82.text.text1
g-xx.yyyyyy.zzzz

```

I hope below one is suitable if file structure remains the same.

```plaintext
awk -F"." '{print $1"."$2"."$3}' file.txt

```

or

```plaintext
awk -F'.' 'BEGIN{OFS="." ;} {print $1,$2,$3}' file.txt

```

this will result -

```plaintext
g-xx.fr.zzzz
g-82.text.text1
g-xx.yyyyyy.zzzz

```

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 30, 2015, 3:22am UTC](https://community.unix.com/t/script-shell-extract-substring/353632/14 "2015-06-30T03:22:01Z")

</div>

> [@mannu2525](#):
>
> Thanks [Don Cragun](http://www.unix.com/members/302122727.html) and [Aia](http://www.unix.com/members/302041415.html) for clarification.
> 
> I have a doubt - If the text file is like -
> 
> ```plaintext
> cat file.txt
> g-xx.fr.zzzz.fr.worker3
> g-82.text.text1.fr.worker1
> g-xx.yyyyyy.zzzz.fr.worker2
> 
> ```
> 
> then -
> 
> ```plaintext
> sed 's/[.]fr[.].*//' file.txt
> 
> ```
> 
> will result -
> 
> ```plaintext
> g-xx
> g-82.text.text1
> g-xx.yyyyyy.zzzz
> 
> ```
> 
> I hope below one is suitable if file structure remains the same.
> 
> ```plaintext
> awk -F"." '{print $1"."$2"."$3}' file.txt
> 
> ```
> 
> or
> 
> ```plaintext
> awk -F'.' 'BEGIN{OFS="." ;} {print $1,$2,$3}' file.txt
> 
> ```
> 
> this will result -
> 
> ```plaintext
> g-xx.fr.zzzz
> g-82.text.text1
> g-xx.yyyyyy.zzzz
> 
> ```

We assume that you realize that the requirements you have presented here are completely different from the requirements you presented in post #1 in this thread (where the request was to remove `.fr.` and everything following it). But, yes, with these new requirements, the above `awk` script does what you want.

---

<div class="post-metadata">

**Author:** ![Aia](https://community.unix.com/user_avatar/community.unix.com/aia/32/917_2.png) [@Aia](https://community.unix.com/u/Aia)\
**Post date:** [June 30, 2015, 12:34pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/15 "2015-06-30T12:34:09Z")

</div>

Don,  
If you do not mind to explain why you chose to use a character class to represent the single period instead of escape it.

```nohighlight
sed 's/[.]fr[.].*//' file.txt

```

---

<div class="post-metadata">

**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\
**Post date:** [June 30, 2015, 2:14pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/16 "2015-06-30T14:14:27Z")

</div>

> [@aia](#):
>
> Don,  
> If you do not mind to explain why you chose to use a character class to represent the single period instead of escape it.
> 
> ```plaintext
> sed 's/[.]fr[.].*//' file.txt
> 
> ```

It's easier for me to use a single construct in a BRE or ERE that never changes depending on context.

All of the following commands remove the 1st two adjacent periods from each input line processed:

```plaintext
sed 's/\.\.//'
awk '{sub("\\.\\.","")}1'
awk -v RE='\\\\.\\\\.' '{sub(RE,"")}1'
sed 's/[.][.]//'
awk '{sub("[.][.]","")}1'
awk -v RE='[.][.]' '{sub(RE,"")}1'

```

Note the varying number of backslashes required when using escapes (one, two , or four for each escaped period), depending on context. But when using the character class expression, the same string works in all three cases.

---

<div class="post-metadata">

**Author:** ![Aia](https://community.unix.com/user_avatar/community.unix.com/aia/32/917_2.png) [@Aia](https://community.unix.com/u/Aia)\
**Post date:** [June 30, 2015, 2:31pm UTC](https://community.unix.com/t/script-shell-extract-substring/353632/17 "2015-06-30T14:31:06Z")

</div>

Don,  
Thank you for explaining your thought process, related to it.
