In my projects Docker file I have some environment variables, like this:
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=Password
ENV MSSQL_PID=Developer
ENV MSSQL_TCP_PORT=1433
And I would like to pass the password here as an environment variable set in my pipeline.
In Azure DevOps I have two pipelines. One for building the solution and one for building and pushing docker images to DockerHub. There are options to set variables in both these pipelines:
I have set the password in both pipelines and edited my password in the Dockerfile to look like this:
ENV SA_PASSWORD=$(SA_PASSWORD)
But that does not seem to be working. What is the correct way of passing environment variables from Azure DevOps into a Docker image?
Also, is this a safe way of passing secrets? Is there any way someone could read secrets from a Docker image?
Thanks!
You can set an ARG var_name and reference ENV to the ARG variables. Then you can replace those variables when docker build the image $ docker build --build-arg var_name=$(VARIABLE_NAME)
For example the add ARG in dockerfile, and have the ENV variable refer to it:
ARG SECRET
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=$SECRET
ENV MSSQL_PID=Developer
ENV MSSQL_TCP_PORT=1433
You can use dock build task and dock push task separately, as buildandpush command cannot accept arguments. And set a variable SECRET in your pipeline.
The set the Build Arguments SECRET= $(SECRET) to replace the ARG SECRET
You can also refer to a similar thread.
I am using the Replace Tokens extension for exactly tasks like this: https://marketplace.visualstudio.com/items?itemName=qetza.replacetokens
However, putting secrets into your Dockerfile might not be the best idea. Usually you would provide secrets or generally runtime configuration as environment variables when you actually execute the container.
I suggest to set the environment variables at runtime. If you are deploying to an Azure App Service, app settings are injected into the process as environment variables automatically.
You can then use the same image for multiple environments. With the Deploy Azure App Service task in a release pipeline, you can change the app settings for each environment.
https://learn.microsoft.com/en-us/azure/app-service/configure-custom-container?pivots=container-linux#configure-environment-variables
Also, is this a safe way of passing secrets? Is there any way someone
could read secrets from a Docker image?
This questions really depends on the appproach and the importance of your image here. Usually there are 2 ways that somebody would look at to achieve this. Build arguments and Environment variables.
Build Arguments are usually declarted in the dockerfile, and are supplied using --build-arg parameter to the docker builder (docker build). Nothe that the docker build will complain if you declare an argument and not pass it during build if you also didnt supply a default value during declaration. These are available only when the image is being built. The subsequent containers from this image will not have access to the ARG variables, as long as you dont set them again using ENV.
ENV are environment varriables which can be declared and set from the dockerfile or at the os-level. The ENV variables are then available for use in subsequent instructions. These are available during the image build and all the subsequent containers from this specific image will have access to these variables. So if you ssh/exec into the container and take a look at which environment variables are set you will find them.
But that does not seem to be working. What is the correct way of
passing environment variables from Azure DevOps into a Docker image?
buildAndPush command will ignore ARG by default in the task-inputs.. So either use a bash step to build the image or separate build and push tasks as described earlier.
In release, choose deploy azure app service task. Provide required properties at App settings section under Application and Configuration Settings option.
Related
When I build my react app locally, all the envoriment variables get read from my local .env file, which is inside the projects root folder. We use gulp for the building process.
Now i want the same variables available in my azure pipeline, which also builds the app via gulp and deploys it to my Azure Static Web App.
I already tried pushing my .env file to the repo and also tried to set these variables in the pipelines YAML file via
env:
HOSTNAME: 'google.com'
And I also tried putting the values in my pipelines variables and accessing it in YAML like
env:
HOSTNAME: $(HOSTNAME)
Lastly I tried uploading my .env to Devops Pipelines Secure files, then adding Tasks to my pipeline to access this file and copiying it to my repos root folder.
All ways ended up with just random String like "8a8878aa1317" inside this variables, once the app is deployed and running. This random strings changes each time I run the pipeline. Does anyone know how to get the right values to the variables?
HOSTNAME is a tricky name for a variable, because it might be used by underlying OS or DevOps Agent (a random string value suggests that).
Try changing it to something like MY_HOSTNAME (in .env file, in the pipeline and in your app).
My requirement is to set few environment variables on the docker image that is built by azure devops pipeline.
This pipeline is used for dev/stage/production environments, each stage is tied to its own set of group variables, these variables I want to be available in nodejs app.
I can read the variables within the pipeline using $VARNAME but cant read same when the code runs using process.env.VARNAME.
I understand that it is not the best approach as the image would have environment variables which may potentially have secrets hence I am open to different ideas as well.
What I have tried so far:
Added ARG in dockerfile
ARG VARNAME=somevalue
on the docker build task added
- task: Docker#2
displayName: Build docker image
inputs:
command: build
repository: $(imageName)
tags: $(Build.BuildId)
buildContext: '$(Pipeline.Workspace)/PublishedWebApp'
arguments: --build-arg SOMEVAR=anewvalue
I try to access this as process.env.SOMEVAR in nodejs code.
I can actually see the --build-arg on the docker build executed in the pipeline but in the code it never appears.
What I am after is pretty standard requirement, we have multiple environments, each environment will have different keys (different group variables tied to different stage), how do I pass different keys to the deployment?
There are a few components from what I can see that need to align:
You are passing the key via --build-arg. This is correct.
You have an ARG in the dockerfile to receive what is passed in via --build-arg. This is correct.
You are missing setting the environment variable equal to the passed-in arg.
FROM python:3.7-slim
ARG SOME_PASSED_IN_ARG
ENV SOMEVARNAME $SOME_PASSED_IN_ARG
...
You would access SOMEVARNAME like process.env.SOMEVARNAME from the node application.
I give credit to #levi-lu-msft in their post here: https://stackoverflow.com/a/59784488/1490277
As for best practice, defining logic default environment variables in the container at BUILD is perfectly acceptable (e.g. ENV=PRODUCTION). At RUN, you are able to to overwrite any environment variable using passed in arguments:
docker run ... --e SOME_PASSED_IN_ARG=someOtherValue
A few "better" practices:
Containers are meant to be repeatable. Don't embed at BUILD time environment variable that will expire/change over time. In other words, if you decide to deploy a version from 3yrs ago and a BUILD time environment variable has an expired value (e.g. coupon code), your environment could be fragile and unpredictable.
Never embed secrets in the container at BUILD. #1 above because they may change overtime and also for security. Secret values (e.g., one-time keys, tokens, etc.) are available to anyone (bad actor or mishap actor) to extract simply by listing the environment variables of the image when running.
My goal is to be able to develop/add features locally then create a local docker build and create a container using the Bitbucket Pipeline Repo Variables. I don't want to hard code any secrets on the host machine or inside the code. I'm trying to access some api keys hosted in the Bitbucket pipeline repo variables.
Anyone know how to do this? I am thinking some script inside the Dockerfile that will create environment variables inside the container.
You can pass these variables to your container as environment variables when you run the container with the -e flag (see: this question), you could use the bitbucket variables at this point. When you do this the variables are available in your docker container, but of course you will then still have to be able to use them in your python script I suppose?
You can easily do that like this:
variable = os.environ['ENV_VARIABLE_NAME']
If you do not want to pass the variables in plain text to the commands like this you could also set up a MySQL container linked to your python container which provides your application with the variables. This way everything is secured, dynamic and not visible from anywhere except to users with acces to your database and can still be modified easily. It takes a bit more time to set up, but is less of a hassle than an .env file.
I hope this helps you
I've set up a Jenkins CI/CD on Azure using Azure VM agents to build my android application.
For the build agent I use a template that is an "Advanced Image Configuration" using the following Image Reference:
Canonical, UbuntuServer, 16.04-LTS, latest
In my Initialization Script I installed all required components to build my application (e.g. the android-sdk). It is run as Root, using sudo command for every operation.
The first time I launched my build it failed, because ANDROID_HOME was not defined. So I decided to add the Environment Injector Plugin to solve this.
My Questions are:
Is it possible to define the ENV within the Initialization script too?
Do I have to configure my agent in a different way?
Will I have to create and configure a VM image and use that instead?
Edit / Solution:
sudo cat >> /etc/environment <<EOL
ANDROID_HOME=/opt/android-sdk
PATH=${PATH}:/opt/android-sdk/tools:/opt/android-sdk/tools/bin:/opt/android-sdk/platform-tools
EOL
This was successful thanks for all the help :)
yeah, why not? just set an environment variable as part of your script.
not sure what you ask here, what do you want to achieve?
i dont like images, i prefer an automated way to create a working vm with scripts. but you can certainly do that
Essentially, I need a way to inject environment variables for my app since I don't want to check a .env file into my repo. I was hoping to run something like API_URL=api.example.com npm run build, but it looks like you can't prepend things before the task command.
Is there a way to do this or is there a better way to create environment variables for a node app hosted in Azure?
You can add some build variables to the build definition and then reference those in your build steps somewhere. For example, for your API_URL add a build variable with the same name and value. If you need the variable to be secret for any reason (passwords, etc.) just click the lock icon next to the value field.
Then add a new cmd task to your build and move it to the top to set your environment variables before you start your build. The way you reference the build variables is like this...
set API_URL=$(Build.API_URL)
In the UI it will look like this:
I added two cmd tasks to a test build just to show that it is working. The first one is used to set the environment variable and then I used the second to dump all the environment variables so I could see them. You can see in the build logs that it worked.
If you want to reference the build variables using something other than the command line you can find examples of the different ways to reference build variables here. There are also examples of how to use secret variables on that page.
EDIT:
Since you have some concerns about it not being available in your node app I tested it out in the console real quick to show that it will definitely work.
Using yaml config you can switch to script task and then do something like this:
- script: 'npm run'
workingDirectory: 'src'
displayName: 'npm run'
env:
{ API_URL: 'api.example.com' }
tehbeardedone's answer is great for the build machine, and the answer is similar for when running your app in an Azure app service, which I'm thinking is what you really need. This is for if you need your access to these .env files just previous to or after your app has started (post build).
To the Azure portal we go.
Navigate to your app service.
Click "Configuration" under the "Settings" heading.
For each of the variables you'll need during a production run of your app you'll need to:
Click "New application setting"
Paste the name of the variable (exactly as it appears in your .env) into the "Name" field.
Similarly, paste the value of the variable into the next field.
Check the "deployment slot setting" checkbox. As nebulous as this field is, it's necessary. When I didn't do this, the app didn't have access to this variable after running the start command.
Restart your app. If you deployment when well, then your app should have access to the variables add in the manner specified above.