Cloud Variables not being set using Nerdbank.GitVersioning with Gitlab CI - gitlab

I'm using the Nuget package Nerdbank.GitVersioning to version my C# .Net DLL project and it is all working for local builds (tried within VS2017).
However when I try with the Gitlab runner to obtain the version number for use during my Nuget packaging steps none of the environment variables have been set regardless of what I have in my version.json file and/or using the nbgv cloud tool
My version.json file looks like this and is in the root of the repo;
{
"$schema": "https://raw.githubusercontent.com/AArnott/Nerdbank.GitVersioning/master/src/NerdBank.GitVersioning/version.schema.json",
"version": "2.0",
"cloudBuild": {
"setVersionVariables": true,
"setAllVariables": true,
"buildNumber": {
"enabled": false,
"includeCommitId": {
"when": "nonPublicReleaseOnly",
"where": "buildMetadata"
}
}
}
}
My .gitlab-ci.yml file looks like this;
#Windows Configuration
variables:
GIT_SUBMODULE_STRATEGY: "normal"
URL: "https://localhost/repository/nuget-hosted"
before_script:
- echo "Restoring nuget packages"
- 'nuget restore MyProject.sln'
#- echo "Installing NB.GV - GitVersioning Tool"
- dotnet tool install --tool-path . nbgv
vs-15-2017-release-develop-build:
stage: deploy
script:
- echo "Building in Release mode"
- 'msbuild my-project.sln /p:Configuration=Release /p:PublicRelease=true'
- echo "Testing in Release mode"
- 'VSTest.Console.exe build/Release/MyProject.Test/MyProject.Test.dll'
- echo "deploying build to internal nuget"
- .\nbgv cloud
- set #output all env variables for debug
- nuget pack build/Release/MyProject/MyProject.nuspec -Version %BuildVersion% -Verbosity detailed
- nuget push MyProject.%VERSION%.nupkg -source %URL% -apikey [removed]
tags:
- vs-15-2017
I've tried various different combinations of setting the cloudBuild parameters but nothing ever outputs the variables to the environment.
A secondary issue (and may be related) is that setting buildNumber.enabled = true does not update the Gitlab build/job number too.
Thanks in advance for any help.
Cheers
Dave

Nerdbank.GitVersioning doesn't have an implementation for GitLab CI that can set the cloud build number. It can recognize when running in GitLab CI, but has no code in the SetCloudBuildNumber method. Care to share a PR if you know how to make this work?

Related

Can we use Trigger in Gitlab community edition

Can we use trigger in Gitlab community edition? I receive an error while trying to configure multi project pipeline.
Here is my code:
stages:
- test
testing:
stage: test
script:
- echo "Parent pipeline is working"
trigger:
- project: testgroup/childproject
- branch: master
Error: jobs:testing config contains unknown keys: trigger
Could you please share version of runner that you are running this code ? Community Edition of Gitlab allows only from 12.8.
Additionally:
Remember that jobs with trigger are compatible with script section
Do not use " - " before project and branch elements
Reference: https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html#trigger-job-configuration-keywords

Setup Gitlab CI/CD environment variables via UI

I am new at gitlab CI/CD Settings. I don't want to put sensitive credentials (like API-Keys, passwords...) into my branch. For this, GitLab (and other CI/CD-Services) are able to set environment variables.
What I have done so far:
Via UI (Project ⇒ Settings ⇒ CI/CD ⇒ Variables)
First go to Project ⇒ Settings ⇒ CI/CD ⇒ Variables and add them like this:
enter image description here
Now here trying to get the File with all your config-values(e.g. with dotenv).
require("dotenv");
module.exports = process.env.NODE_ENV.trim() === "production" ? _config.production : _config.development;
Current .gitlab-ci.yaml file is:
image: node:8.9.0
cache:
paths:
- node_modules/
stages:
- ver
- init
- test
- build
- deploy
ver:
stage: ver
script:
- node -v
init:
stage: init-dev
script:
- npm install
tags:
- dev_server
only:
- dev
variables:
ENV_PRODUCTION: "/builds/AkdiD/8/abcde/projectName/ENV_PRODUCTION"
test:
stage: test
script:
- npm test
build:
stage: build
script:
- echo "BUILD_VERSION=production" >> build.env
artifacts:
reports:
dotenv: build.env
deploy:
stage: deploy-dev
script:
- npm run killcurrent
- echo $ENV_PRODUCTION
- echo $BUILD_VERSION
- npm run staging
tags:
- dev_server
only:
- dev
Question: where do I need to keep this ENV_PRODUCTION file name (yaml file or other place) so that server take that value ??
Edited variable like this- still server it not fetching these variables. Should I change/put something in .gitlab-ci.yml file?
Settings up a Custom environment variables of type File (GItLab 11.11+) does not seem the way to reference/set a list of variables, including ones with sensitive information.
A variable of type file is generally there to represent, for instance, a certificate.
You should define variables, possibly Group-level environment variables
You can define per-project or per-group variables that are set in the pipeline environment.
Group-level variables are stored out of the repository (not in .gitlab-ci.yml) and are securely passed to GitLab Runner, which makes them available during a pipeline run.
For Premium users who do not use an external key store or who use GitLab’s integration with HashiCorp Vault, we recommend using group environment variables to store secrets like passwords, SSH keys, and credentials.

Azure pipeline task DotNetCoreCLI#2 - output folder of artifact

In our .NET Web API project, we tried to build API project in Azure DevOps and publish the artifact to a folder with the pipeline task below:
- task: DotNetCoreCLI#2
displayName: Publish web API artifact
inputs:
command: publish
publishWebProjects: false
arguments: '$(Build.SourcesDirectory)\XYZ.Research.API\XYZ.Research.API.csproj --configuration $(BuildConfiguration) --output testpath'
zipAfterPublish: true
modifyOutputPath: true
But I am not sure which folder the artifact is kept. Below is the log from this step:
2020-07-31T12:04:23.6282186Z ##[section]Starting: Publish web API artifact
2020-07-31T12:04:23.6590490Z ==============================================================================
2020-07-31T12:04:23.6591051Z Task : .NET Core
2020-07-31T12:04:23.6591393Z Description : Build, test, package, or publish a dotnet application, or run a custom dotnet command
2020-07-31T12:04:23.6591740Z Version : 2.172.2
2020-07-31T12:04:23.6591974Z Author : Microsoft Corporation
2020-07-31T12:04:23.6592357Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
2020-07-31T12:04:23.6592942Z ==============================================================================
2020-07-31T12:04:25.5581194Z [command]C:\windows\system32\chcp.com 65001
2020-07-31T12:04:25.5581889Z Active code page: 65001
2020-07-31T12:04:25.5583746Z Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
2020-07-31T12:04:25.5588792Z [command]C:\hostedtoolcache\windows\dotnet\dotnet.exe publish d:\a\1\s\XYZ.Research.API\XYZ.Research.API.csproj --configuration Release --output testpath
.....
some warning message ignored
.....
2020-07-31T12:04:38.0843543Z XYZ.Research.API -> d:\a\1\s\XYZ.Research.API\bin\Release\netcoreapp3.0\XYZ.Research.API.dll
2020-07-31T12:04:38.9127845Z XYZ.Research.API -> d:\a\1\s\testpath\
2020-07-31T12:04:46.0295716Z Info: Azure Pipelines hosted agents have been updated to contain .Net Core 3.x (3.1) SDK/Runtime along with 2.1. Unless you have locked down a SDK version for your project(s), 3.x SDK might be picked up which might have breaking behavior as compared to previous versions.
2020-07-31T12:04:46.0296632Z Some commonly encountered changes are:
2020-07-31T12:04:46.0297619Z If you're using `Publish` command with -o or --Output argument, you will see that the output folder is now being created at root directory rather than Project File's directory. To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
2020-07-31T12:04:46.0442329Z ##[section]Finishing: Publish web API artifact
Because we will need the file location in next step (deployment), I tried
d:\a\1\s\testpath\XYZ.Reserch.API.zip
d:\a\1\s\testpath\XYZ.Reserch.API\XYZ.Reserch.API.zip
but none of these location has the artifact file.
Did anyone see this issue before? Any help would be appreciated.
------------------- update -------------------------------
As #Source Code suggested, I used task "PowerShell#2" and find that the artifact file are actually in "D:\a\1\s\testpath\testpath.zip". That means the 'testpath' sub-folder are created in $(Build.SourceDirectory) and the artifact file are also renamed to 'test.zip'.
I would recommend that you add a PowerShell/Bash/Cmd task after your DotNetCoreCLI#2 task and run a inline script with the 'ls' command that should list all the items to the results for you. This will allow you to see what is actually there after the task.
If on a Windows agent:
- task: PowerShell#2
displayName: List Files Post Publish
inputs:
targetType: inline
script: Get-ChildItem
If on Linux or Mac
- task: Bash#3
displayName: List Files Post Publish
inputs:
targetType: inline
script: ls
Additionally I noticed you're providing your csproj file via the arguments parameter. There is a parameter named projects which should be used for this. Also you may consider using the the artifacts staging directory as your output directory. The task would look like this:
- task: DotNetCoreCLI#2
displayName: Publish web API artifact
inputs:
command: publish
projects: '$(Build.SourcesDirectory)\XYZ.Research.API\XYZ.Research.API.csproj'
publishWebProjects: false
arguments: '--configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: true
modifyOutputPath: true
One important thing to note is if you do change the output directory ensure that you change the working directory for the PowerShell or Bash tasks so you output the contents of the correct directory. It defaults to the $(Build.SourcesDirectory) so ensure that you change this if needed.

Unable to connect to container in Gitlab CI in my free account

I have a free account of gitlab.
I also have a company account (not sure which plan).
I have the exact same project, a wrapper on EventStore.
In the CI pipeline I want to spin up a container with event store so that I can run some integration tests against it.
This is my .gitlab-ci.yml that restores, compiles, runs tests and publishes nuget packages
#Stages
stages:
- ci
- pack
#Global variables
variables:
GITLAB_RUNNER_DOTNET_CORE: mcr.microsoft.com/dotnet/core/sdk:2.2
EVENT_STORE: eventstore/eventstore:release-5.0.2
NUGET_REPOSITORY: $NEXUS_NUGET_REPOSITORY
NUGET_API_KEY: $NEXUS_API_KEY
NUGET_FOLDER_NAME: nupkgs
#Docker image
image: $GITLAB_RUNNER_DOTNET_CORE
#Jobs
ci:
stage: ci
services:
- $EVENT_STORE
variables:
# event store service params testing with standard ports
EVENTSTORE_INT_TCP_PORT: "1113"
EVENTSTORE_EXT_TCP_PORT: "1113"
EVENTSTORE_INT_HTTP_PORT: "2113"
EVENTSTORE_EXT_HTTP_PORT: "2113"
EVENTSTORE_EXT_HTTP_PREFIXES: "http://*:2113/"
script:
- dotnet restore --no-cache --force
- dotnet build --configuration Release
- dotnet vstest test/*Tests/bin/Release/**/*Tests.dll
pack-beta-nuget:
stage: pack
script:
- export VERSION_SUFFIX=beta$CI_PIPELINE_ID
- dotnet pack *.sln --configuration Release --output $NUGET_FOLDER_NAME --version-suffix $VERSION_SUFFIX --include-source --include-symbols -p:SymbolPackageFormat=snupkg
- dotnet nuget push **/*.nupkg --api-key $NUGET_API_KEY --source $NUGET_REPOSITORY
except:
- master
pack-nuget:
stage: pack
script:
- dotnet restore
- dotnet pack *.sln --configuration Release --output $NUGET_FOLDER_NAME
- dotnet nuget push **/*.nupkg --api-key $NUGET_API_KEY --source $NUGET_REPOSITORY
only:
- master
As you can see, I spin up the event store container.
From my integration tests I try to connect to the container within the CI using the following connection string:
"ConnectTo=tcp://admin:changeit#127.0.0.1:1113; HeartBeatTimeout=500;";
With my work account it works fine, there is a container listening on 127.0.0.1 on port 1113 and I can connect to it using the above connection string.
With my free personal account it is unable to connect.
Why?
I suspect it has something to do with the way docker is available on both gitlab CI runners, but why is different?
And more important, how can I configure event store on my personal CI pipeline on my free account so that I can connect to it if the localhost is not a valid host Uri for whatever reason?
Well, you have not provided any details but it seems you're using the Docker executor. In that case, services are not available on localhost but only accessible as service aliases.
This is an extract from the working CI file:
test:
stage: test
script:
- dotnet test
variables:
ASPNETCORE_ENVIRONMENT: Testing
EVENTSTORE_EXT_HTTP_PORT: 2113
EVENTSTORE_EXT_TCP_PORT: 1113
EVENTSTORE_RUN_PROJECTIONS: all
EVENTSTORE_START_STANDARD_PROJECTIONS: "true"
EventStore__ConnectionString: ConnectTo=tcp://admin:changeit#eventstore:1113
services:
- name: eventstore/eventstore:latest
alias: eventstore
only:
refs:
- branches
- tags
For this to work, your appsettings.Testing.json file needs to point to ConnectTo=tcp://admin:changeit#eventstore:1113.
If you want to keep using the appsettings file with the configuration that points to localhost, you can override the setting using env variable in the CI file. Just remember to add environment variables as the configuration source. The code snippet above has such an override, matching our settings structure:
{
"EventStore": {
"ConnectionString": "ConnectTo=whatever"
}
}
If you ever decide using the Kubernetes executor, you will need to revert to using localhost, because Kubernetes executor creates one pod per build with multiple containers, including all service containers. There's an open issue to support service aliases with Kubernetes runners, I think it will be like 12.9 or 13, pretty soon. That being said, using service aliases is a safe, future proof way of making it all work.
P.S. Just noticed that your setup works with one account and doesn't work with another. My guess would be that you either use different executors (Docker doesn't work and Kubernetes works) or different GitLab versions (if the service alias issue has already been fixed).

how to deploy a custom dll along with azure function using appveyor

Deploying azure function using appveyor is straightforward as described by this article - https://alastairchristian.com/deploying-azure-functions-from-appveyor-75fe03771d0c#.x7stvprna
Further what I am not able to figure out is how to deploy a custom dll (library part of the same repo) in the /bin folder to be able to use #r and use the reference.
Manual process is just to copy the dll in the /bin folder using Kudu and start using it.
Also refer to this discussion on appveyor - http://help.appveyor.com/discussions/questions/2842-deployment-to-azure-function-app
Here is my appveyor.yml -
-
branches:
only:
- master
version: 0.0.{build}
os: Visual Studio 2015
configuration: Release
init:
- nuget sources update -Name nuget.org -Source https://api.nuget.org/v3/index.json
environment:
SolutionDir: $(APPVEYOR_BUILD_FOLDER)\
cache:
- packages -> **\packages.config
assembly_info:
patch: true
file: '**\AssemblyInfo.*'
assembly_version: '{version}'
assembly_file_version: '{version}'
assembly_informational_version: '{version}'
before_build:
- nuget restore -verbosity detailed
build:
verbosity: normal
project: MyLibrary\MyLibrary.csproj
publish_wap: true
publish_azure: true
parallel: true
artifacts:
- path: AzureFuncCIPOC
name: AzureFuncCIPOC
deploy:
- provider: WebDeploy
server: https://functioncipoc.scm.azurewebsites.net:443/msdeploy.axd?site=FunctionCIPOC
website: FunctionCIPOC
username: $FunctionCIPOC
password:
secure: <secure>
artifact: AzureFuncCIPOC
If your build process is building that library and moving the output (your custom DLL) so that is is packaged in the artifact created by AppVeyor, it will be deployed with the scripts.
I was able to copy the required dlls as project post-build event and then use the same .yml as posted in question to achieve what I was after. Really hope that VS 2017 will have better support for this when Azure Functions tools for VS2017 are launched.
below is the post-build event command
if not exist "$(SolutionDir)AzureFuncCIPOC\\ManualTriggerCSharp\bin\" mkdir "$(SolutionDir)AzureFuncCIPOC\\ManualTriggerCSharp\bin\"
copy /y "$(TargetDir)" "$(SolutionDir)AzureFuncCIPOC\\ManualTriggerCSharp\bin\"

Resources