I need to grab a value from a config file (config.toml) and store it to a system variable.
I prefer to use a single command without pipes in the most compatible way possible (across linux systems)
Should return https://abridge.netlify.app with the config file set any of these ways:
base_url="https://abridge.netlify.app"
base_url = "https://abridge.netlify.app"
base_url = "https://abridge.netlify.app"
The solution I have so far based on some research is the following:
baseurl="$(sed -n -E 's/^base_url.*=\s+?\"//p' config.toml)"
echo $baseurl
My solution still has the trailing quotation mark, I have not yet figured out how to deal with it.
The other problem is I am not certain that this is the most universal solution. I prefer to use something that will work universally on most linux systems.
Appreciate all feedback, Thank You!
Different implementations of sed understand different types of regex. Only BRE is portable:
sed -n 's/^base_url[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' config.toml
Edit, based on #jhnc feedback. I now use this:
baseurl="$(sed -n -E 's/^base_url.*=.*"(.*)"/\1/p' config.toml)"
echo $baseurl
I am still not sure if some Linux systems might run into issues with this or not.
If anyone sees an issue with this solution or has a more universally compatible solution please let me know.
Related
I'm writing a CLI using node and I've arrived at the part where I take user input and append it to a string that is the command for the child_process.exec function.
const CURL_CHILD = exec('npm view --json ' + process.argv[2] + ...
I am trying to figure out what I need to do to process.argv[2] before I pass it to the exec function. I've surfed around for a while and haven't found any questions or answers that address this specific case.
What is the best way to sanitize this user input for this particular use case? What is actually needed here?
Update
I'm still surfing around trying to learn and answer my own question and found this link which suggests I use js-string-escape (a node package). I'd really like to use something native/vanilla to do this. Does node have any tools for this?
Update 2
I finally stumbled upon the buzzwords "command injection" and found a slew of articles recommending the use of child_process.execFile or child_process.spawn. I'm still curious if there is a native way to sanitize the input, while still securing the full shell process created by child_process.exec. I am leaving this open in hopes someone can answer it.
Your user input arguments may contain variable chars that the shell is going to interpret them in its own way. So for instance, in Linux, $ has a special meaning.
If you want to use such argument as-is avoiding the shell interpretation, you have to escape them. The same we do with some special chars in HTML (eg. < and > has special meaning so we use to escape them in HTML like < and respectively >). The same applies here.
So the answer to your question is first to find out the special chars in your shell/environment and escape them.
A good rule of thumb is to escape chars like double-quote ", single-quote ', space , the dollar-sign $ (because it's an Illuminati symbol, right? ;-), the grave-accent ` and obviously the backslash \.
So let's suppose that your command is the one below. To escape it just use some simple regex like this:
cmd = "npm view --json " + process.argv[2];
escapedCmd = cmd.replace(/(["\s'$`\\])/g,'\\$1');
I hope it helps :-)
Try the npm package escape-it. Should work with *nix OSes, but has some support for Windows, too.
I have spent all day today trying to find a proper solution, but I am not able to. My problem:
I have an XML file with tags containing multiple of the same.
Example:
<TASK INSTANCE />
<WORKFLOWLINK CONDITION=""/>
<WORKFLOWLINK CONDITION=""/>
I want to add the contents of an other XML file before the first <WORKFLOWLINK. The issue I've ran into is that this file is full of double quotes and slashes. I've tried replacing them and escaping them, but to no avail.
My tries mainly culminated on something like:
sed -e "0,/<WORKFLOWLINK/ /<WORKFLOWLINK/{ r ${filename}" -e "}" ${sourcefile}
If this isn't clear enough I'll get the exact data so you can see.
For the fun of sed:
sed -e "0,/<WORKFLOWLINK/{/<WORKFLOWLINK/{r ${sourcefile}" -e"}}"
The trick is to start a new "pattern/command" pair after your first address range condition 0,/<WORKFLOWLINK/.
Two nested patterns/addresses are not understood, there must be a command after the first pattern. Using an additional pair of curlies {} does that for you.
Apart from the brain exercise to do it in sed, #EdMorton is right in recommending to use an XML-processor. Also his request for an MCVE is appropriate. I had to do some guessing to see what you want and I hope I guessed right.
The mcve should at least have included
the error message or problem description defining your problem
the initialisation of your environment variables
some sample input; not the original data
You surely would have had an answer earlier and (in case mine does not satisfy you) probably a better one by now.
So, before your next question, please take the https://stackoverflow.com/tour
GNU sed version 4.2.1
GNU bash, version 3.1.17(1)-release (i686-pc-msys)
Everyone,
Thank you for thinking with me, even if I apparently broke some rules.
I have figured out a solution, granted it is not as pretty as can be, but for a one time action it is good enough.
I have moved from a single command to a combination of first detecting the location I want to put my data:
sed -e "0,/<WORKFLOWLINK/ s/<WORKFLOWLINK/##MARKER##\n\t<WORKFLOWLINK'" which will put the marker string in the desired location.
After this I replace the marker with the contents of the file I have. I managed to make the individual statements working when I was trying to do it all in a single statement before, so I just execute them separately.
sed -e "/##MARKER##/{r ${sourcefile}" -e 'd}'
Until recently, I was under the impression that by convention, all Linux command options were required to be prefixed by a hyphen (-). So for example, the instruction ls –l executes the ls command with the l option (here we can see that the l option is prefixed by a hyphen).
Life was good until I got to the chapter of my Linux for beginners book that explained the ps command. There I learned that I could write something like ps u U xyz where as far as I can tell, theu and U are options that are not required to be prefixed by a hyphen. Normally, I would have expected to have to write that same command as something like ps –uU xyz to force the usage of a hyphen.
I realize that this is probably a stupid question but I was wondering if there is a particular reason as to why the ps command does not follow what I thought was the standard way of specifying command options (prefixing them with hyphens). Why the variation? Is there a particular meaning to specifying hyphen-less options like that?
There are a handful of old programs on Unix that were written when the conventions were not as widely adopted, and ps is one of them. Another example is tar, although it has since been updated to allow options both with and without the - prefix.
IMO the best practice concerning hyphenation is to use them as the default go-to. More times than not, they have accepted hyphen prefixes to most or all flags/options available for commands. Happy to be corrected if I am wrong in this instance. I am still new to this myself! :)
I'm stuck in this very easy problem (I hope it is for you).
I need to substitute several strings with special characters in a huge file.
I'm trying using sed and bash because I'm a linux user but I've only used sed for "standard" string so far.
These are the kind of strings that I'm trying to manipulate
(alpha[1],alpha[2]) and diff(A45(i,j),alpha[1])
and the substituting strings would be
(i,j) and dzA45(i,j)
I tried sed -i 's/(alpha[1],alpha[2])/(i,j)/g' $filetowork and
sed -i 's/\(alpha\[1\],alpha\[2\]\)/i,j/g' $filetowork without any success
The second option seems to work for the first kind of string but it doesn't for the second one, why?
could you please help me? I took a look around stackoverflow old questions without any help, unfortunately :(
I just tried on the command line, but
echo "(alpha[1],alpha[2])" | sed 's/(alpha\[1\],alpha\[2\])/(i,j)/
worked for the first case. Please note that you should not escape ( or ), because that is how you activate groups.
For the second one
echo "diff(A45(i,j),alpha[1])" | sed 's/diff(A45(i,j),alpha\[1\])/dzA45(i,j)/'
worked for me. The same case, don't escape brackets!
I have a short text file with the following syntax:
FileName: some name
Version: 3
Length: 45
hello, this is an irrelevant, unimportant text.
So is this line.
Now, I'm trying to write a script that replace the version number with a given new number.
Anyone knows how to? I really don't mind it to be ugly
thanks,
Udi
Why not just use sed?
sed -i 's/^Version: .*$/Version: 99/' foo.txt
I don't know how to do it in csh. However, at the risk of coming across as one of those annoying people who tells you to use their favourite thing at every opportunity, there are better ways than using csh. The traditional unix command sed is good at this stuff, or a language like Perl is also useful.
perl -p -i -e 's/Version: 3/Version: 4/g;' myfile
should do it.