We have multiple environments for our apps, and I am trying to use generic consul template and an environment variable to create haproxy configurations for different environments.
This is what I am trying to do:
{{$environment := "yellow" }}
....
balance leastconn {{range service "myservice-{{$env}}"}}
....
I cannot seem to find a way to use this variable within this range. Is there a way to get this working?
Thanks!
You just need to define the variable and then use it in the range statement.
{{ $service_name := env "CONSUL_SERVICE_NAME" }}
{{range service $service_name }}
Related
I am trying to make the sending of emails depend on the execution environment so that other business users don’t get flooded with mails coming from a lower environment i.e test or dev
The email should be only send to the business from the production.
I am using the following code as dynamic content for the if condition however this is very specific to only one environment:
#equals(pipeline().DataFactory,'AdPrd117')
However, if I can make the above dynamic content more robust for all the 3 environments then is there a way: where I can use the if condition as below :
if data factory name.
Is there a possibility that we can make the email alerts getting generated(alerts which is with the help of a notebook) for each environment(dev, test, prod) and for a specific email id that can be added to the if condition?
We set up our DataFactory with a suffix that indicates the environment (Dev, Tst, Prod).
I have the following expression that checks which environment a pipeline is running in to figure out which directory of an SFTP Server to use (Development, Test, or Production):
#if(
startswith(
pipeline().DataFactory
, 'dev'
)
, 'Development'
, if(
startswith(
pipeline().DataFactory
, 'tst'
)
, 'Test'
, 'Production'
)
)
Likewise, we use similar code to process our cubes, and to figure out which LogicApp to use to send messages (we have 1 logic app per environment).
Edit:
Based on the comments, maybe this would help you. This assumes your environments have a consistent naming convention, and your dev/test ones do not contain prd.
Create a variable called Recipient of type string in your pipeline. Then add a Set Variable component to your pipeline. Give it the name "Set recipient" on the General tab, and then on the Variables Tab, set the name to the "Recipient" variable you just created.
In the Value field, use the following:
#if(
equals(
substring(pipeline().DataFactory, 2, 3)
, 'Prd'
)
, 'MyBusinessUsers#contoso.com'
, 'MyDevelopers#contoso.com'
)
This means you now have a variable which will contain the group you want to send mails to, depending on the name of the ADF name.
I have a DevOps variable group with a variable like that: VARIABLE=['a', 'b', 'c'].
Then in Azure pipeline, there is a parameter like that:
parameters:
- name: parameter_test
displayName: 'Test paramter'
type: string
default: a
values:
- a
- b
- c
I want to use the variable instead of the hardcoded list, since it's present in multiple pipelines. Tried this, but docs say I can't use expressions in parameters section:
parameters:
- name: parameter_test
displayName: 'Test paramter'
type: string
default: a
values:
- ${{ each group in variables.VARIABLE }}:
- ${{ group }}
Have you ever tried things like that or have any idea how to parametrize it?
Thanks for any help!
According to this document Variable groups for Azure Pipelines - Azure Pipelines | Microsoft Docs, to reference a variable group, use macro syntax or a runtime expression, therefore the parameter cannot be defined with the value of variable from a variable group.
Instead of defining the parameter with the value of the variable in a variable group, you may consider using a core YAML to transfer the parameter/variable value into a YAML Template. Kindly refer to the below sample YAML pipeline.
# Core YAML
trigger:
- none
pool:
vmImage: ubuntu-latest
variables:
- name: Variable_core
value: b
parameters:
- name: Parameter_core
default: c
values:
- a
- b
- c
steps:
- template: Parameters.yml
parameters:
parameter_test_Azure_Repos_1: ${{ variables.Variable_core }}
parameter_test_Azure_Repos_2: ${{ parameters.Parameter_core }}
# Parameters.yml from Azure Repos
parameters:
- name: parameter_test_Azure_Repos_1
displayName: 'Test Parameter 1 from Azure Repos'
type: string
default: a
- name: parameter_test_Azure_Repos_2
displayName: 'Test Parameter 2 from Azure Repos'
type: string
default: a
steps:
- script: |
echo ${{ parameters.parameter_test_Azure_Repos_1 }}
echo ${{ parameters.parameter_test_Azure_Repos_2 }}
All non yaml files is not recommended as this is not as code, very difficult to check & audit & versionning, so as to variable group, release pipeline etc.
Azure pipeline has indeed some limitations, we can reuse the variables but not the parameters.
If I was you, even multiple pipelines use the same parameter, I will still "hard code" this directly in the pipelines just like what you wrote:
parameters:
- name: parameter_test
displayName: 'Test paramter'
type: string
default: a
values:
- a
- b
- c
I am recently doing an CI/CD setup using Azure. The goal is to have the developer select the type of build to be created i.e Staging / Prod.
Thanks to How to write if else condition in Azure DevOps Pipeline, I have added following code -
parameters:
- name: selectConfiguration
displayName: Select build configuration
type: string
default: Debug
values:
- Debug
- Release
variables:
- name: config
${{ if eq(variables['parameters.selectConfiguration'], 'Debug') }}:
value: Debug
${{ else }}:
value: Release
This gives me the following result -
But no matter what I select in this radio group, it always run the else block. i.e. the if-else always fails. Any help to understand what I am doing wrong here?
Try the below, it should work. I am using the same logic to switch between different agent pools.
variables:
${{ if eq(parameters.selectConfiguration, 'Debug') }}:
config: Debug
${{ else }}:
config: Release
In YAML pipeline, you can not use the if...else expression to assign different values to a variable in different conditions.
You can only use the if expression to determine a variable which has one specified value can be available in the specified condition. See "Conditionally assign a variable".
The if...else expression can be used to:
assign different values to an input of the task in different conditions. See "Conditionally set a task input".
run different steps in a job in different conditions. See "Conditionally run a step".
Although I am not familiar with DevOps best practices, I am trying to come up with a reliable and efficient method for managing multiple variables in production. The following represents my current approach:
/
|ENV_VAR.sh
|--/api1
|--/staging.api1
|--/api2
|--/staging.api2
Where:
ENV_VAR.sh
### API 1 variables ###
export API1_VAR_1=foo
export API1_VAR_2=foo2
export API1_STAG_VAR_1=foo_stag
export API1_VAR_2=foo2_stag2
### API 2 variables ###
export API2_VAR_1=foo
export API2_VAR_2=foo2
export API2_STAG_VAR_1=foo_stag
export API2_VAR_2=foo2_stag2
The API 1 and 2 are two nodejs-based apps running in the same server using a reverse-proxy configuration.
If nothing goes bad with the server (e.g. unexpected shutdown), I just have to (re)set the variables once in a while via SOURCE ENV_VAR.SH in order to make sure that new variables are defined.
Before proceeding with this approach, I would like to know whether it is correct at all, or if it has a big flaw.
If this approach is alright, how to automatically (re)source the environment variables from the package.json whenever a new version of any App is deployed? (just to guarantee that the variables are still defined)
Thanks in advance.
I like using Loren West's config package for these configuration parameters. I happen to like to extend it with the properties package: that way I don't have to put parameters in valid, comment-free, JSON format. JSON5 also helps solve the readability problem, but I haven't tried it.
Why do I like this?
It gives a structured way of dealing with development / test / staging / production environments. It keys off the ENV environment variable, which of course has values like development and production.
All properties files go into a single directory, typically ./config. Your production krewe can tell what they're looking at. default.properties, development.properties and production.properties are the names of typical files.
Most configuration parameters don't have to be secret, and therefore they can be committed to your repository.
Secrets (passwords, connection strings, API keys, etc) can be stored in local.properties files placed into ./config by your deployment system. (Mention local.properties in your .gitignore file.)
Secrets can also be loaded from environment variables, named in a file called ./config/custom_environment_variables.json.
It works nicely with pm2.
This is really easy to configure.
Your files:
default.properties (used when not overridden by another file)
[API1]
VAR_1 = foo
VAR_2 = foo2
[API2]
VAR_1 = foo
VAR_2 = foo_for_api2
staging.properties
[API1]
VAR_1 = foo_stag
VAR_2 = foo2_stag2
[API2]
VAR_1=foo_stag
VAR_2=foo2_stag2
custom_environment_variables.json
{
"API1" : {
"password": "API1_PASS"
},
"API2" : {
"password": "API2_PASS"
}
}
Your nodejs program:
const config = require( 'config' )
require( 'properties' )
const appConfig = config.get( 'API1' )
const var1 = appConfig.VAR_1
const password = appConfig.password
Then you run your program with API1_PASS=yaddablah nodejs program.js and you get all your configs.
While deploying a build to multiple environments like UAT, Production. I want to replace one file config.uat.json or config.prod.json with config.json. Is there any option available? just like we have XML Transformation.
I am aware of Json Variable substitution but that doesn't serve my purpose as the variable list is long (almost 50 entries)
Thanks in Advance!
JSON variable substitution should be a good option, if you don't want to add variables one by one on Variables tab, you can update them with VSTS REST API, or add a powershell script to set the variables.
Otherwise, you may need to delete the config.uat.json or config.prod.json, and copy the config.json to the target machine.
Inside your Program.cs file, you can get an environment variable which will represent your current environment:
var builder = WebHost.CreateDefaultBuilder(args);
var currentEnv = builder.GetSetting("environnement");
using this currentEnv value, you will be able to load the file config.{currentEnv}.json
builder.AddJsonFile($"config.{currentEnv}.json", optional: false, reloadOnChange: true);
EDIT:
If you want to do this in powershell, you can make a transofmration of your configuration file with a default: appsettings.json containing keys, and appsettings.env.json containing overriding.
To transform your configuration, you can do something like this:
Param(
[Parameter(Mandatory=$true)][string]$SpecificConfig
)
$defaultConfig = "AppSettings.json";
$settingsContent = ConvertFrom-Json $defaultConfig;
$specificContent = ConvertFrom-Json $SpecificConfig;
# Do this on each <property> to override
if (![string]::IsNullOrEmpty($specificContent.<property>))
{
$settingsContent.<property> = $specificContent.<property>;
}
Write-Host $settingsContent > $defaultConfig;