I'm using gitlab-ci & my .gitlab-ci.yml looks like
variables:
DASHBOARD_RESET_ROUTE: '/#/reset-password'
DEFAULT_LOCATION_TYPE: 'P'
This was deployed successfully but when I changed the value of DEFAULT_LOCATION_TYPE to some other text.
The console logs are still showing the old values
Then, even after removing the whole variable from both .yml file & gitlab.com secret variables. The console.logs still print the old values.
Output at gitlab.com UI is:
Running with gitlab-runner 10.1.0 (c1ebf97h)
on Test - ip-172-31-12-234 (6d0vb950)
Using Shell executor...
Running on ip-XXX-XX-XX-XXX...
Fetching changes...
Removing node_modules/
HEAD is now at fff6491 wip
Checking out fff64913 as feat/ci-cd...
Skipping Git submodules setup
Checking cache for default...
Successfully extracted cache
$ npm install
Thanks In Advance
Okay, So reason was using shell as an executor, That was the reason the environment variables are not reloaded again.
Then I reconfigured with docker as an executor & it's working fine.
Related
I have been developing Terraform v0.14.7 to run via GitLab Pipeline. Everything's been going well until I tried upgrading Terraform to v0.15.5 (Upgrading above that is beyond my control at the moment). I followed the advice here and a how-to guide here.
When I run terraform version on my Terraform server, it shows the proper upgraded version, and I can successfully run terraform init each time just fine. However, when I run my pipeline, the version seems to flip between the pre, validate, and plan stages with no consistent pattern. I have it print terraform version at the start of each stage, and sometimes pre outputs 0.14.7, validate outputs 0.15.5, and then plan flips back to 0.14.7. Sometimes the order is different as well, such as pre outputting 0.15.5 and the other two stages 0.14.7. I can't figure out why it's doing this since I can't recreate the problem manually.
It's been very confusing trying to solve - I tried clearing the runner cache, removing the .terraform cache and .terraform.lock.hcl files, and switching up how I run the init (adding backend=false or lockfile=readonly flags). Clearing the cache and files allows for one single clean pipeline run, but the next run is always back to being mixed versions. Does anyone know what I'm doing wrong?
Edit: The closest documented issue I can find is this issue here, but a solution was never posted. I have also tried terraform refresh, init -migrate-state, and init -reconfigure with no luck.
Edit 2: I am running Terraform offline with no internet access, so upgrades need to be done manually in case that changes any answer.
Edit 3: I'm unable to paste the exact code but here's as much as I can give, not that the output at the beginning of each stage is Terraform has been successfully initialized!:
.gitlab-ci.yml:
before_script:
- |
export TF_VAR_pipeline=$CI_PIPELINE_URL
terraform -chdir="$WORKING_DIR" init -backend=false
pre:
stage: pre
script:
- which terraform
- terraform version
validate:
stage: validate
script:
- which terraform
- terraform version
- terraform -chdir="$WORKING_DIR" validate
plan:
stage: plan
script:
- which terraform
- terraform version
- terraform -chdir="$WORKING_DIR" plan -out=plan_file.txt
artifacts:
paths:
- $WORKING_DIR/plan_file.txt
when: on success
example pre output:
$ which terraform
/usr/bin/terraform
$ terraform version
Terraform v0.15.5
on linux_amd64
Saving cache
Creating cache default-3...
...
No URL provided, cache will be not up-loaded to shared cache server. Cache will be stored only locally.
Created cache
Cleaning up file based variables
Job succeeded
example validate output:
$ which terraform
/usr/bin/terraform
$ terraform version
Terraform v0.15.5
on linux_amd64
$ terraform -chdir="$WORKING_DIR" validate
Success! The configuration is valid.
Saving cache
Creating cache default-3...
...
No URL provided, cache will be not up-loaded to shared cache server. Cache will be stored only locally.
Created cache
Cleaning up file based variables
Job succeeded
example plan output:
$ which terraform
/usr/bin/terraform
$ terraform version
Terraform v0.14.7
$ terraform -chdir="$WORKING_DIR" plan -out=plan_file
An execution plan has been generated and is shown below.
...
Unfortunately this issue was never solved. I believe it had something to do with the Gitlab cache but I couldn't figure it out. I ended up spinning up a new Terraform server instance and installing a fresh gitlab-runner, all seems to be working well now.
I have a gitlab job that does not seem to update the repository before being run. Sometimes it leaves some files in their old states and run the script... Any idea ?
For instance when I have a
packagePython:
stage: package
script:
- .\scripts\PackagePython.ps1
tags:
- myServer
cache:
paths:
- .\python\cache\
only:
changes:
- python/**/*
I finally managed to understand what was happening :
I realised that the gitlab-runner did not use exactly the same path for each run on my server, and my script assumed that it did... So I ended up pointing on a build made on the wrong path.
I guess if you think that it is not updating the repository (like I did) make sure you are not referencing hardcoded path/package in your scripts that could refer to previous versions !
For all the other stackoverflow questions, it seems like people are asking either about a private npm git repository or about a different technology stack. I'm pretty sure I can use a private npm registry with GAE Flexible, but I was wondering if it was possible with the Standard version?
From the GAE standard docs, doesn't seem like it is possible. Anyone else figure out otherwise?
Google marked this feature request as "won't fix, intended behavior" but there is a workaround.
Presumably you have access to the environment variables during the build stage of your CI/CD pipeline. Begin that stage by having your build script overwrite the .npmrc file using the value of the environment variable (note: the value, not the variable name). The .npmrc file (and the token in it) will then be available to the rest of the CI/CD pipeline.
For example:
- name: Install and build
env:
NPM_AUTH_TOKEN: ${{ secrets.PRIVATE_REPO_PACKAGE_READ_TOKEN }}
run: |
# Remove these 'echo' statements after we migrate off of Google App Engine.
# See replies 14 and 18 here: https://issuetracker.google.com/issues/143810864?pli=1
echo "//npm.pkg.github.com/:_authToken=${NPM_AUTH_TOKEN}" > .npmrc
echo "#organizationname:registry=https://npm.pkg.github.com" >> .npmrc
echo "always-auth=true" >> .npmrc
npm install
npm run compile
npm run secrets:get ${{ secrets.YOUR_GCP_PROJECT_ID }}
Hat tip to the anonymous heroes who wrote replies 14 and 18 in the Issure Tracker thread - https://issuetracker.google.com/issues/143810864?pli=1
If you have a .npmrc file checked in with your project's code, you would be wise to put a comment at the top, explaining that it will be overwritten during the CI/CD pipeline. Otherwise, Murphy's Law dictates that you (or a teammate) will check in a change to that .npmrc file and then waste an unbounded amount of time trying to figure out why that change has no effect during deployment.
It's been 2 days since one of my project' build starts failing on Gitlab CI. The main error was E_MISSING_APP_KEY and when I check another variable just by echoing $HOST and $PORT from my .gitlab-ci.yml config, like this
tests:
script:
- echo "${HOST} ${PORT}"
- node -e "console.log(process.env.HOST, process.env.PORT)"
- node_modules/.bin/nyc node ace test -t 0
I got nothing.
The build was failed because it can't read my environment variable that I set on its CI Settings.
Anyone experiencing same issue? & how to solve this?
Update:
I'm trying to create new project with only containing .gitlab-ci.yml file here and it's seems working just fine
But why the world it's still failing on my main project?
For anyone else having a similar problem:
check your variable, if it is protected your branch has to be protected as well or remove the protected option on your variable
The issue is solved by delete all of my variables I've had & set them back from the CI Setting. And the build pipeline is running without any errors. (except the actual testing is still failed, lol)
Honestly, I'm still wondering why this could happened? and hopefully no one will experiencing same kind of issue like me here..
I have GitLab running on computer A, development environment (Visual studio Pro) on computer B and Windows Server on computer C.
I set up GitLab-Runner on computer C (Windows server). I also set up .gitlab-ci.yml file to perform build and run tests for ASP.NET Core application on every commit.
I don't know how can I get code on computer C (Windows server) so I can build it (dotnet msbuild /p:Configuration=Release "%SOLUTION%"). It bothers me that not a single example .gitlab-ci.yml I found on net, doesn't pull code form GitLab, before building application. Why?
Is this correct way to set-up CI/CD:
User create pull request (a new branch is created)
User writes code
User commit code to branch from computer B.
GitLab runner is started on computer C.
It needs to pull code from current branch (CI_COMMIT_REF_NAME)
Build, test, deploy ...
Should I use common git command to get the code, or is this something GitLab runner already do? Where is the code?
Why no-one pull code from GitLab in .gitlab-ci.yml?
Edited:
I get error
'"git"' is not recognized as an internal or external command
. Solution in my case was restart GitLab-Runner. Source.
#MilanVidakovic explain that source is automatically downloaded (which I didn't know).
I just have one remaining problem of how to get correct path to my .sln file.
Here is my complete .gitlab-ci.yml file:
variables:
SOLUTION: missing_path_to_solution #TODO
before_script:
- dotnet restore
stages:
- build
build:
stage: build
script:
- echo "Building %CI_COMMIT_REF_NAME% branch."
- dotnet msbuild /p:Configuration=Release "%SOLUTION%"
except:
- tags
I need to set correct variable for SOLUTION. My dir (where GitLab-Runner is located) currently holds this folder/files:
- config.toml
- gitlab-runner.exe
- builds/
- 7cab42e4/
- 0/
- web/ # I think this is project group in GitLab
- test/ # I think this is project name in GitLab
- .sln
- AND ALL OTHER PROJECT FILES #Based on first look
- testm.tmp
So, what are 7cab42e4, 0. Or better how to get correct path to my project structure? Is there any predefined variable?
Edited2:
Answer is CI_PROJECT_DIR.
I'm not sure I follow completely.
On every commit, Gitlab runner is fetching your repository to C:\gitlab-runner\builds.. on the local machine (Computer C), and builds/deploys or does whatever you've provided as an action for the stage.
Also, I don't see the need for building the source code again. If you're using Computer C for both runner and tests/acceptance, just let the runner do the building and add Artifacts item in your .gitlab-ci.yaml. Path defined in artifacts will retain your executables on Computer C, which you are then able to use for whatever purposes.
Hope it helps.
Edit after comment:
When you push to repository, Gitlab CI/CD automatically checks your root folder for .gitlab-ci.yaml file. If its there, the runner takes over, parses the file and starts executing jobs/stages.
As soon as the file itself is valid and contains proper jobs and stages, runner fetches the latest commit (automatically) and does whatever script item tells it to do.
To verify that everything works correctly, go to your Gitlab -> CI / CD -> Pipelines, and check out whats going on. You should see something like this:
Maybe it would be best if you posted your .yaml file, there could be a number of reasons your runner is not picking up the code. For instance, maybe your .yaml tags are not matching what runner is created to pick up etc.