I am trying to work out how I can ammend a log file to remove variable strings in order to remove confidential information which I cant pass on.
As an example I have used phone numbers. A large log file contains multiple lines containing something like the below
ses:3317] start call ms <xxxxxxxxxxxx> id < 13102 > id2 <0a23> index 858
In the code above i have placed x's where the variable numbers will be that i need to change, these varible numbers i need to be able to anonymize/replace. Is there a way of doing this ? Note that the numbers will always start with a set prefix as the first 2/3 digits also.
I'm not sure I understand what you're trying to do. Do you have a file that contains a string a numbers on each input line that you want to change each digit in that string to an "x"? (Could there by hyphens, plus-signs, and/or parentheses in the string? If so, should they be changed too?)
Or do you have a file that contains a string of "x" characters that you want to change to a phone number?
Are all lines in the same format, or are there other lines in different formats that are not supposed to be modified?
Please show us sample input (source file and whatever defines the "set prefix") and the corresponding output you want to produce from those inputs.
Hi Don - the file contains numbers which need to be changed to x's.
There are different formatted lines however say if the line contained a 10 digit number and started with 078 this should be modified to x's this will then allow me to amend the prefix accordingly.
The code in my post is the output I want to obtain, the only difference between the source and output is where the x's are there are numbers.
For the sample output you showed us (with the output being 10 "x"s surrounded by angle brackets), the following should work with any version of sed that conforms to the standards:
If we are dealing only with 10 digit numbers lead by "078", that will work. If not, we need to know more to identify the digits to be replaced, like "the first angle bracketed number in the resp. line", or "the number following the 'start call' string"
Isolate the number pattern and place it inside the capturing group in red.
$ cat test.file
ses:3317] start call ms <0781234567> id < 13102 > id2 <0a23> index 858
ses:3318] start call ms <0781234568> id < 13102 > id2 <0a23> index 858
ses:3319] start call ms <0801234568> id < 13102 > id2 <0a23> index 858
ses:3320] start call ms <0781234569> id < 13102 > id2 <0a23> index 858
$ perl -pe 's/(?<=<)(078\d+)(?=>)/x x length($1)/e' test.file
ses:3317] start call ms <xxxxxxxxxx> id < 13102 > id2 <0a23> index 858
ses:3318] start call ms <xxxxxxxxxx> id < 13102 > id2 <0a23> index 858
ses:3319] start call ms <0801234568> id < 13102 > id2 <0a23> index 858
ses:3320] start call ms <xxxxxxxxxx> id < 13102 > id2 <0a23> index 858
The red pattern can be just the current example if it would satisfy the phone number without matching any other unwanted parts.
It could be more specific as: 078(?:\d){7}
If you want to match, strictly, 078 followed by seven more digits.
Or, just \d{10} for any 10 digits.
Either way, you have flexibility since x x length($1) will convert whatever is matched into a length of x, accordingly.
Based on your example, it appears that there's only one occurrence of the phone number per line, therefore no attempt was done to match more than once per line.