I have a project (PROJECT_A) that is triggered through a webhook, and expects the variable $PRODUCT to be set. Its value is used to trigger a certain path in the build. The job in the .gitlab-ci.yml file looks like this:
deploy:
stage: publish
script:
- ./generate_doc.sh $PRODUCT
A webhook call looks like this:
http://<GITLAB_URL>/api/v4/projects/710/ref/master/trigger/pipeline?token=<TOKEN>&variables[PRODUCT]=<PRODUCT>
I call this trigger through a webhook from other projects, including PROJECT_B.
So I manually filled in the desired value in the respective webhooks, e.g. for PROJECT_B:
http://<GITLAB_URL>/api/v4/projects/710/ref/master/trigger/pipeline?token=<TOKEN>&variables[PRODUCT]=PROJECT_B
When the pipeline in PROJECT_A is triggered, $PRODUCT has the value PROJECT_B, as expected.
I would like to parameterize the pipeline further and take, among others, the commit message into account. All the information I need is apparently provided in the webhook payload.
Is there a built-in way to read this payload in a pipeline? Or alternatively, put contents of the payload into a variable in the webhook like this:
http://<GITLAB_URL>/api/v4/projects/710/ref/master/trigger/pipeline?token=<TOKEN>&variables[COMMIT_REF]=???
I have found discussions about doing parameterized Jenkins builds using the webhook payload, including this related question. There is also a similar question in the Gitlab forum, without any answer.
Is there a way to do access that payload in a Gitlab CI pipeline? I could probably extract the provided values with a jq call, but how can I get the Json in the first place?
If you run compgen -v to show the environment variables when triggering the pipeline in the UI (without JSON payload) you get 3 fewer lines in your job log than when POSTing a JSON payload.
The additional variables are:
CI_BUILD_TRIGGERED
CI_PIPELINE_TRIGGERED
TRIGGER_PAYLOAD
If you print their values out and re-run the pipeline:
echo CI_BUILD_TRIGGERED=$CI_BUILD_TRIGGERED
echo CI_PIPELINE_TRIGGERED=$CI_PIPELINE_TRIGGERED
echo TRIGGER_PAYLOAD=$TRIGGER_PAYLOAD
You get (for username YOUR_USER_NAME and repo name YOUR_REPO_NAME)
CI_BUILD_TRIGGERED=true
CI_PIPELINE_TRIGGERED=true
TRIGGER_PAYLOAD=/builds/YOUR_USER_NAME/YOUR_REPO_NAME.tmp/TRIGGER_PAYLOAD
So as you can see the payload is stored as TRIGGER_PAYLOAD in a temporary directory suffixed .tmp, which re-running the pipeline and printing it out (cat) shows it contains the payload, in my case that’s JSON.
Related
When I start a job and run a command
npm run e2e_test,
it successfully executed. I set a variable let say REPORT_URL using process.env.REPORT_URL . Now in Jenkins post action I would like to access this variable REPORT_URL.
This url is construed at runtime so it cannot be set before the job. Once the job completed all the stages, it reaches in post section and here if I mention like
echo "${REPORT_URL}" or echo "${env.REPORT_URL}"
I get an exception MissingProperty-: NO such property: REPORT_URL for class WorkflowScript.
I tried to create a variable in .env file with same name but didn't get success.
So I can access a variable in post build in Jenkins pipeline which set in code using process.env.XXXX?
Answer is based on: Share variables between Jenkins pipeline stages
Create a global Jenkins variable and initialize it with a dummy value like null.
Pass any value to the variable
Access the global variable in your post stage.
I'm triggering an Azure Logic App from an https webhook for a docker image in Azure Container Registry.
The workflow is roughly:
When a HTTP request is received
Queue a new build
Delay until
FinishTime of Queue a new build
See: Workflow image
The Delay until action doesn't work in that the queueried FinishTime is 0001-01-01T00:00:00.
It complains about the wrong format, so I manually added a Z after the FinishTime keyword.
Now the time stamp is in the right format, however, the timestamp 0001-01-01T00:00:00Z obviously doesn't make sense and subsequent steps are executed without delay.
Anything that I am missing?
edit: Queue a new build queues an Azure pipeline build. I.e. the FinishTime property comes from the pipeline.
You need to set a timestamp in future, the timestamp 0001-01-01T00:00:00Z you set to the "Delay until" action is not a future time. If you set a timestamp as 2020-04-02T07:30:00Z, the "Delay until" action will take effect.
Update:
I don't think the "Delay until" can do what you expect, but maybe you can refer to the operations below. Just add a "Condition" action to judge if the FinishTime is greater than current time.
The expression in the "Condition" is:
sub(ticks(variables('FinishTime')), ticks(utcNow()))
In a word, if the FinishTime is greater than current time --> do the "Delay until" aciton. If the FinishTime is less than current time --> do anything else which you want.(By the way you need to pay attention to the time zone of your timestamp, maybe you need to convert all of the time zone to UTC)
I've been in touch with an Azure support engineer, who has confirmed that the Delay until action should work as I intended to use it, however, that the FinishTime property will not hold a value that I can use.
In the meantime, I have found a workaround, where I'm using some logic and quite a few additional steps. Inconvenient but at least it does what I want.
Here are the most important steps that are executed after the workflow gets triggered from a webhook (docker base image update in Azure Container Registry).
Essentially, I'm initializing the following variables and queing a new build:
buildStatusCompleted: String value containing the target value completed
jarsBuildStatus: String value containing the initial value notStarted
jarsBuildResult: String value containing the default value failed
Then, I'm using an Until action to monitor when the jarsBuildStatus's value is switching to completed.
In the Until action, I'm repeating the following steps until jarsBuildStatus changes its value to buildStatusCompleted:
Delay for 15 seconds
HTTP request to Azure DevOps build, authenticating with personal access token
Parse JSON body of previous raw HTTP output for status and result keywords
Set jarsBuildStatus = status
After breaking out of the Until action (loop), the jarsBuildResult is set to the parsed result.
All these steps are part of a larger build orchestration workflow, where I'm repeating the given steps multiple times for several different Azure DevOps build pipelines.
The final action in the workflow is sending all the status, result and other relevant data as a build summary to Azure DevOps.
To me, this is only a workaround and I'll leave this question open to see if others have suggestions as well or in case the Azure support engineers can give more insight into the Delay until action.
Here's an image of the final workflow (at least, the part where I implemented the Delay until action):
edit: Turns out, I can simplify the workflow because there's a dedicated Azure DevOps action in the Logic App called Send an HTTP request to Azure DevOps, which omits the need for manual authentication (Azure support engineer pointed this out).
The workflow now looks like this:
That is, I can query the build status directly and set the jarsBuildStatus as
#{body('Send_an_HTTP_request_to_Azure_DevOps:_jar''s')['status']}
The code snippet above is automagically converted to a value for the Set variable action. Thus, no need to use an additional Parse JSON action.
In our pipeline we programmatically create a pull request (PR). The branch being merged into has a policy on it that launches a build. This build takes a variable amount of time. I need to query the build status until it is complete (or long timeout) so that I can complete the PR, and clean up the temp branch.
I am trying to figure out how to get the build that was kicked off by the PR so that I can inspect the status by using Microsoft.TeamFoundation.Build.WebApi, but all overloads of BuildHttpClientBase.GetBuildAsync require a build Id which I don't have. I would like to avoid using the Azure Build REST API. Does anyone know how I might get the Build kicked off by the PR without the build ID using BuildHttpClientBase?
Unfortunately the documentation doesn't offer a lot of detail about functionality.
Answering the question you asked:
Finding a call that provides the single deterministic build id for a pull request doesn't seem to be very readily available.
As mentioned, you can use BuldHttpClient.GetBuildsAsync() to filter builds based on branch, repository, requesting user and reason.
Adding the BuildReason.PullRequest value in the request is probably redundant according to the branch you will need to pass.
var pr = new GitPullRequest(); // the PR you've received after creation
var requestedFor = pr.CreatedBy.DisplayName;
var repo = pr.Repository.Id.ToString();
var branch = $"refs/pull/{pr.PullRequestId}/merge";
var reason = BuildReason.PullRequest;
var buildClient = c.GetClient<BuildHttpClient>();
var blds = await buildClient.GetBuildsAsync("myProject",
branchName: branch,
repositoryId: repo,
requestedFor: requestedFor,
reasonFilter: reason,
repositoryType: "TfsGit");
In your question you mentioned wanting the build (singular) for the pull request, which implies that you only have one build definition acting as the policy gate. This method can return multiple Builds based on the policy configurations on your target branch. However, if that were your setup, it would seem logical that your question would then be asking for all those related builds for which you would wait to complete the PR.
I was looking into Policy Evaluations to see if there was a more straight forward way to get the id of the build being run via policy, but I haven't been able to format the request properly as per:
Evaluations are retrieved using an artifact ID which uniquely identifies the pull request. To generate an artifact ID for a pull request, use this template:
vstfs:///CodeReview/CodeReviewId/{projectId}/{pullRequestId}
Even using the value that is returned in the artifactId field on the PR using the GetById method results in a Doesn't exist or Don't have access response, so if someone else knows how to use this method and if it gives exact build ids being evaluated for the policy configurations, I'd be glad to hear it.
An alternative to get what you actually desire
It sounds like the only use you have for the branch policy is to run a "gate build" before completing the merge.
Why not create the PR with autocomplete.
Name - autoCompleteSetBy
Type - IdentityRef
Description - If set, auto-complete is enabled for this pull request and this is the identity that enabled it.
var me = new IdentityRef(); // you obviously need to populate this with real values
var prClient = connection.GetClient<GitHttpClient>();
await prClient.CreatePullRequestAsync(new GitPullRequest()
{
CreatedBy = me,
AutoCompleteSetBy = me,
Commits = new GitCommitRef[0],
SourceRefName = "feature/myFeature",
TargetRefName = "master",
Title = "Some good title for my PR"
},
"myBestRepository",
true);
Is there a way to reference the output of an executed pipeline in the activity "Execute pipeline"?
I.e.: master pipeline executes 2 pipelines in sequence. The first pipeline generates an own created run_id that needs to be forwarded as a parameter to the second pipeline.
I've read the documentation and checked that the master pipeline log the output of the first pipeline, but it looks like that this is not directly possible?
We've used until now only 2 pipelines without a master pipeline, but we want to re-use the logic more. Currently we have 1 pipeline that calls the next pipeline and forwards the run_id.
ExecutePipline currently cannot pass anything from its insides to its output. You can only get the runID or name.
For some weird reason, the output of ExecutePipeline is returned not as a JSON object but as a string. So if you try to select a property of output like this #activity('ExecutePipelineActivityName').output.something then you get this error:
Property selection is not supported on values of type 'String'
I found that I had to use the following to get the run ID:
#json(activity('ExecutePipelineActivityName').output).pipelineRunId
The execute pipeline activity is just another activity with outputs that can be captured by other activities. https://learn.microsoft.com/en-us/azure/data-factory/control-flow-execute-pipeline-activity#type-properties
If you want to use the runId of the pipeline executed previosly, it would look like this:
#activity('ExecutePipelineActivityName').output.pipeline.runId
Hope this helped!
Before starting a new issue, I always create a new branch for it (directly from Gitlab). When I finish the job on that issue (and tests are Ok), I create a merge request (from Gitlab).
After the merge is done, I have an "auto-generated" message linked to that merge (this message is very generic and identical to all merges I done).
The some thing happens also when I merge develop into master:
Is there a way to customize the merge request message to have a message like this:
Merge {shortIssueName}: {issueDescription} into {develop|master}
Note:
I'm using GitLab Community Edition 8.15.3.
Globally, automatically - I don't think so. As I see, it's hard-coded:
message = [
"Merge branch '#{source_branch}' into '#{target_branch}'",
title
]
if !include_description && closes_issues_references.present?
message << "Closes #{closes_issues_references.to_sentence}"
end
message << "#{description}" if include_description && description.present?
message << "See merge request #{to_reference}"
message.join("\n\n")
You can override message for any merge request manually:
It's also possible if you create merge request with API. It requires your time but you can build some mechanism that fetches all data with API and set it as a description (but you must ensure all is available with API, issueDescription and so on).
Even tho #piotr-dawidiuk makes a good point, I believe is outdated.
According to gitlab docs, you can create your .md files, changing all templates. Check it here. As it states,
Similarly to issue templates, create a new Markdown (.md) file inside the .gitlab/merge_request_templates/ directory in your repository. Commit and push to your default branch.