Within our pipeline's we would like to set a variable based on some user defined capabilities. For example, agent-1 may store all python versions under "C:/Python" whereas agent-2 may store all python versions under "C:/Documents/Python" and a script may need to know of all the contents stemming from this folder. So, to fix this, we set some user capabilities of where it's stored.
Agent 1: PYTHON_DIR = C:/Python
Agent 2: PYTHON_DIR = C:/Documents/Python
We would like to extract these from in our azure-pipelines.yml for use in future script steps.
We initially tried using the syntax:
variables:
PYTHON_EXE: $(PYTHON_DIR)\Python38\...\python.exe
but this simply echos out as
$(PYTHON_DIR)\Python38\...\python.exe even after an agent reboot.
I am using the Config crate in Rust, and would like to use environment variables to set keys inside a section of the config. The end goal is to override application settings from a docker compose file, or docker command line, using the environment.
If my config was the following, could I use a specifically crafted environment variable to set database.echo ?
(code blurb below is taken from this example)
debug = true
[database]
echo = true
The example code to configure this using the environment variables illustrates only to set keys at the top level. Wondering how to extend this. The .set() takes a hierarchical key, so I'm hopeful that there's a way to encode the path in the env variable name.
Answering my own question.
I just noticed that the Environment code accepts a custom separator which will get replaced with . (dot).
So one can set the separator to something like _XX_ and that would get mapped to a ".". Setting DATABASE_XX_ECHO=true, for instance would then change the database.echo key.
I need to use a variable between node modules folder and src folder.Is there a way to use a variable to be used within the whole project?
Thanks.
Typically this is done with process environment variables. Any project can look for a common environment variable and handle that case accordingly, as well as the node modules of said project. As long as they agree on the name, they all share the same environment. Have a good default, don't force people to set this environment variable.
The environment can be set using an environment file (do not check this into source control!), on your container or cloud configuration, or even right on the command line itself.
TLOG=info npm test
That is an example that I use frequently. My project runs logging for the test cases only at alert level - there are a lot of tests so it makes the output less verbose. However, sometimes while developing I want to see all the logs, so my project is looking for an environment variable TLOG (short for "test logging") and I can set it just for that run! Also no code change is needed, which is nicer than JavaScript variables that need to be set back to original values, forget to be turned off, etc.
Is it possible to reference environment variables in logstash configuration?
In my case, i want to make my elasticsearch address configurable that i have set in the environment.
With logstash 2.3, you can set environment variable references into Logstash plugins configuration using ${var} or $var.
https://www.elastic.co/guide/en/logstash/current/environment-variables.html
Before logstash 2.3, you can use the "environment" filter plugin which is community maintained.
Documentation at : https://www.elastic.co/guide/en/logstash/current/plugins-filters-environment.html#plugins-filters-environment-add_field_from_env
How to install this plugin:
$LOGSTASH_HOME/bin/plugin install logstash-filter-environment
Source code at : https://github.com/logstash-plugins/logstash-filter-environment
The main part is:
# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"
# Set fields from environment variables
class LogStash::Filters::Environment < LogStash::Filters::Base
config_name "environment"
# Specify a hash of fields to the environment variable
# A hash of matches of `field => environment` variable
config :add_field_from_env, :validate => :hash, :default => {}
public
def register
# Nothing
end # def register
public
def filter(event)
return unless filter?(event)
#add_field_from_env.each do |field, env|
event[field] = ENV[env]
end
filter_matched(event)
end # def filter
end # class LogStash::Filters::Environment
I can hardly believe, that these are the only solutions left: Hacking logstash or using some kind of templating system to re-write the config.
Actually, I do not want to touch or tweak the config for different deployment-scenarios: All I want is to pass in some parameters to connect logstash to the outside world (e.g. where elasticsearch is located, usernames/credentials to connect to other systems). I googled for an hour now an all I could find were these awkwardly complicated solutions for this very simple and common problem.
I sincerely hope, that someone comes up with a better idea like
%{ENV[ELASTICSEARCH_HOST]}}
That's not directly supported, no.
However, if you're running a version later than 1.4.0, it would be pretty trivial to edit elasticsearch.rb to add this feature. Around line 183:
client_settings["network.host"] = #bind_host if #bind_host
You could tweak it to read an environment variable:
if ENV["ESHOST"].nil? then
client_settings["network.host"] = ENV["ESHOST"]
else
client_settings["network.host"] = #bind_host if #bind_host
end
If you prefer, you can run Logstash with the -e command-line option to pass config via STDIN. You could cat in some file with special tokens that you've replaced with your environment variable(s).
The logstash configuration as of this writing is just a configuration file, but it's not a programing language. Thus, it has a few reasonable "limitations", for example, it cannot reference environment variables, cannot pass parameters it, hard to reuse other configuration file. Those limitations would make the logstash configuration file hard to maintain when the configuration file grows or you want to adjust its behavior on the fly.
My approach is to use template engine to generate the logstash configuration file. I used Jinja2 in Python.
For example, the elastic search output could be templated as
output {
elasticsearch {
index => {{ es_index_name }}
host => {{ es_hostname }}
}
}
Then I wrote a simple python code using Jinja2 to generate the logstash configuration file, and the value of es_index_name and es_hostname could be passed via the Python script argument. See here for Jiaja2 tutorial: http://kagerato.net/articles/software/libraries/jinja-quickstart.html
In this way, a big logstash configuration could be splitted into reusable pieces and its behavior can be adjusted on the fly.
As explained in logstash-issues
Connections are established at plugin registration time (during initialization, as they almost certainly should be), but field interpolation (like %{escluster}) is an event-processing-time operation. As such, host isn't really eligible for this behavior.
So unless input or output plugin will natively supports %{foo} syntax, doing any environment variable evaluation at the stage of event filtering is too late for the input and output plugin to take advantage of it.
.conf file support environment variables.
you just have to export the environment variable:
export EXAMPLE_VAR=123
and use it in the configuration file this way:
${EXAMPLE_VAR}
What is the typical use-case for adding --prefix support to a scons build?
Currently I use Variables to persist user options from the command-line, but that requires I use the name PREFIX to be recognized as a variable. The docs show a way to specify the preifx using AddOption but then this option must be specified everytime scons is run: it isn't persisted like the variables.
What is the typical way to handle this?
You will need to use Add/GetOption to create an option the user can provide on the command line.
After, yous users will need to set the environment variable SCONSFLAGS with options they want to provide to SCons everytime. SCons will read this variable everytime.
http://www.scons.org/doc/2.0.1/HTML/scons-user/c2076.html
It's not a good idea to have many environment variable on the user machine because you lost the documentation and which variable you really need to compile.