I need to write a sed that replaces the value of all environment variables that have PASS in the name with ***** . I have a log file who prints the value of all environment variables ,including those who hold passwords. In the name of these variables I found always the PASS string, ex: OP_ORA_PASS, OP_DSU_PASS,etc.
What I did until now is the next sed :
sed 's/'"$OP_ORA_PASS"'/******/g;s/'"$DSU_ORA_PASS"'/******/g'
I need an improvment so that the sed will check and replace all the variables with PASS in the name,kind of $*_PASS.
Please provide an example of your current logfile; otherwise, we're likely to miss a corner case.
Also, you say you're replacing the values of those variables, but it looks like you're just changing the name to "*****". So an example of ideal output from your logfile snippet would be appreciated, too.
If I understand you right, you are going to add some filter into the print-env-var-script?!
If so, (assuming the environment printed out with variable names) you coud 'sed' for any line with substring 'PASS=' and replase following with '*****':
> env | sed 's/^\(.*PASS=\).*/\1*****/'
so, it's remembering as \1 a beggining of a string with 'PASS=' and replace that string with the remembered \1 part and '*****'
The same pipe you can add in command of executing the script; so it will filtering the script's output.
If you do not have the variable's names (seems unbelievable), you can get all *PASS variables into a file or local variables and filtering printout in the way you already have, using retrieved values.
. (By the way, having a variables in sed-command part it is better to use:
... sed "commands" file
. ( - IMHO) )
For local variable: (I've used '*NAME' variables)
> for ln in $(env|grep -i name); do
>> sed_cmd=$sed_cmd"s/${ln#*=}/*****/g; ";
>>done
>echo $sed_cmd
s/dstnsun0/*****/g;
s/geo_usa/*****/g;
s/dca0701/*****/g;
> env|sed "$sed_cmd" | grep -i name
MAPMARKER_RPC_HOSTNAME=*****
MAPMARKER_DBNAME=*****
LOGNAME=*****
For file:
>rm sed_cmd.t
> for ln in $(env|grep -i name); do ec "s/${ln#*=}/*****/g;">>sed_cmd.t; done
> env|sed -f sed_cmd.t | grep -i name
Thank you Alex for the examples, its exactly what I needed.
I used sed 's/^\(.*PASS=\).*/\1*****/' and it did the job.
Can you explain me what this sed contains? Or maybe give a link to read? I need to make an adaptation of it and I need to know more.
I have the following line in the log :
ORA_CONNECT_STRING=username value/password
The username name is taken from the $USERNAME variable.
How do I write the sed to change the password to ?
sed 's/^\(.*CONNECT_STRING='"$USERNAME"'/\).*/\1/'
Is that correct ?
env10 is the value of $ORA_USER. Unfortunely this variable is an application-related one, and does not exist anymore when I execute the sed(after the log is writen). Because of this I get the following result :
+ sqlplus -s /****@DBL9
Thats because $ORA_USER is empty....
What I need is a sed that looks up the string :
+ sqlplus -S ,skips the rest until the / and then changes the string between the / and @ with *****
1 - commands are between single quotes (could be between double quots.)
2 - s is for substitute; the form is s/<search part>/<replace part./[<option>]
3 - in the 'search': the ^ - represent the 'beginning of the string'
----- the \( and \) are the define the part that need to be remembered. To reference the remembered part a consecutive number is used: here it is the first pair; so, it is refered by the \1
----- the back slashes in \(, \), \1 are used to escape the simbol to be not treted by the executing shell as a simbol to be 'expanded' (how it is named in manuals) So, to be not replaced by the shell before 'sed' has received the command
----- the dot after \( is represents an any character. The * after the dot means 'to have a character 0 or more times' (Therefore I do not understand the BMDancorrection with addition the [^=] - the ^ inside of the means 'not'; not sure if '=' get some meaning, but without any special meaning the [^=]* means: 'does not have '=' 0 or more times' that make no sence for me. Also, Ive checked the 'PASS=' and it has been selected (as expected) by the ^.PASS= ... - so, I am not clear what the note was about)
----- After than in the command is the litteral string "PASS=", followed by '.', which is the same - any character 0 or more times (and the end of the 'remember' part)
4 - In the 'replace' part there is the remembered part and the litteral 5 *. So, even if line with no password string will be found, it will put the '*****' after '=', as it was something.
5 - there is no needs for any option after third /
6 - any command could be finished by ';' and another command could be given. That how I've process later with 'for .. - loop'.
I hope it explains the construction of the sed command.
You can find more than enough links with sed-explanations (including the man-pages,) by google, for example, but it is not easy to understand, IMHO.
As you have later said the $USERNAME is not available in time of processing. So, it is better to reffer it as an 'something', than as the exact word, how you have it done here. And it is not needed the .* in beginning, because you exactly know what it should be. So:
's/^\(ORA_CONNECT_STRING=.*\/\).*/\1*****/'
see the '/' is used in search string, but to prevent the sed from getting it as the search part end, it has to be escaped by the back-slash: \/
(I did not checked it, but it should work.)
Again, do not use the $ORA_USER, but specify where exactly it should be found: between + sqlplus -S and / and password is end by the @
So:
s/^\(+ sqlplus -S .*\/\).*\(@.*\)/\1*******\2/
and it is pretty the same as it is advised by Franklin52, but more specific
As I see Frankling52 is using the '!' instead of the '/' to be able to use the '/' inside of the search string. I preffer to do it with escape: \/ - little bet ugly, but the standards command form.
and you'll understand my change. "Does not have '=' 0 or more times" is a poor way to describe it; "0 or more characters that aren't a '='", or more simply, "grab characters until you hit a '='" would be more accurate.
Agree!
I has been aplying description automaticaly,
but more precize reading of the [^=]* is:
--- 'not '=' character(s) 0 or more times',
concluding, as you have said:
--- anything, untill '='
I see.
So, if the specified string exist more than once, reg-ex selecting last one!
Hm, not obviouse logic!
But, the solution 'until not =' is not the best. How to do the same if defined search part will be, say, 'BOOBOO'?
I am not a guru in reg-ex.
I know in 'patern removal expantion' that situation could be prevented by syntaxis: ${varnm%boo*} - removes first match from end, ${varnm%%boo*} - removes all, as they said - longest match.
Maybe something like that is available for sed regular exprecions, but I do not know.
Each RegEx, as read left-to-right, consumes as many characters as it can. Given that context, perhaps it now makes more sense?
In Perl, one can append the "?" character to specify a non-greedy match--that is, match only as much as is needed to satisfy the condition. In this case, however, there's a simple answer: give it what it wants:
env |sed 's/PASS=.*/PASS=*****/' |grep -i pass
Since sed will match the earliest occurrence of "PASS=" it can find in each line, we've now hit paydirt. There are some corner cases here (like "UNRELATEDVARIABLE=None may PASS= this point."), but all can be defeated by including a test:
angry-chipmunk:~$ env | grep PASS
SECRETPASS=n0b0dyc4nevergue55myPASS=word
UNRELATEDVARIABLE=None may PASS= this point
MYPASS=foobar
MYPASSWORD=foobar
angry-chipmunk:~$ env|sed '/^[^=]*PASS=/ {s/PASS=.*/PASS=*****/}' |grep -i pass
SECRETPASS=*****
UNRELATEDVARIABLE=None may PASS= this point
MYPASS=*****
MYPASSWORD=foobar
Thank you BMDan for explanation.
I remember you have mention about some other potentional problem with that statement. I do not see anythink. Did you changed your mind about that or just have no time and interest to show that?