Listing GitLab Repositories name using GitLab api - gitlab

I want to list only GitLab repositories name using GitLab API.
i tried command curl "https://gitlab.com/api/v4/projects?private_token=*************"
it is listing all merge requests, issues and also repository names.
How can i list only the repository names?

As Igor mentioned the Gitlab API does not let you to limit the response to a single field.
I would recommend combining the original API call with the jq command.
This will enable parsing of the returned json
curl "https://gitlab.com/api/v4/projects?private_token=*************" | jq '.[].name'

There's no way to limit the response to a single field only via API, but there's simple option to return minimal set of fields:
https://docs.gitlab.com/ee/api/projects.html#list-all-projects
curl "https://gitlab.com/api/v4/projects?simple=true&private_token=*************" would return something like:
[
{
"id": 4,
"description": null,
"default_branch": "master",
"ssh_url_to_repo": "git#example.com:diaspora/diaspora-client.git",
"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",
"web_url": "http://example.com/diaspora/diaspora-client",
"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",
"tag_list": [ //deprecated, use `topics` instead
"example",
"disapora client"
],
"topics": [
"example",
"disapora client"
],
"name": "Diaspora Client",
"name_with_namespace": "Diaspora / Diaspora Client",
"path": "diaspora-client",
"path_with_namespace": "diaspora/diaspora-client",
"created_at": "2013-09-30T13:46:02Z",
"last_activity_at": "2013-09-30T13:46:02Z",
"forks_count": 0,
"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",
"star_count": 0
},
{
"id": 6,
"description": null,
"default_branch": "master",
...

Related

How to get a list of all pipelines run for gitlab scheduled pipeline

I have a scheduled pipeline setup in gitlab, but when I query it, it only shows the last run pipeline. I would like to be able to get a list of all of the pipelines it has run so I can perform some analysis on them. I tried looking into both the rest and graphql apis and I didn't seem to find anything that would solve this. Is there a way to do this?
You can actually list all pipelines triggered by schedule using GitLab REST API
GET /projects/:id/pipeline_schedules/:pipeline_schedule_id/pipelines
Your request attributes must contain the project id and the pipeline_schedule_id as both are required. You could replace the project id with the URL-encoded path of your owned projects.
Example Request:
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13/pipelines"
Example Response:
[
{
"id": 47,
"iid": 12,
"project_id": 29,
"status": "pending",
"source": "scheduled",
"ref": "new-pipeline",
"sha": "a91957a858320c0e17f3a0eca7cfacbff50ea29a",
"web_url": "https://example.com/foo/bar/pipelines/47",
"created_at": "2016-08-11T11:28:34.085Z",
"updated_at": "2016-08-11T11:32:35.169Z"
},
{
"id": 48,
"iid": 13,
"project_id": 29,
"status": "pending",
"source": "scheduled",
"ref": "new-pipeline",
"sha": "eb94b618fb5865b26e80fdd8ae531b7a63ad851a",
"web_url": "https://example.com/foo/bar/pipelines/48",
"created_at": "2016-08-12T10:06:04.561Z",
"updated_at": "2016-08-12T10:09:56.223Z"
}
]
Reference link https://docs.gitlab.com/ee/api/pipeline_schedules.html#get-all-pipelines-triggered-by-a-pipeline-schedule

How to get date of tag from Azure Devops Cli

I'm using the Azure devops cli to get a list of the tags for a particular repo. Documented here: https://learn.microsoft.com/en-us/cli/azure/repos?view=azure-cli-latest. For example
az repos ref list -p <project>-r <name of repo> --filter tags
The result looks something like this
[
{
"creator": {
"descriptor": "aad.ODAzODM5N2YtNWE3Yi03MzBkLThlNTktMzhmMTJlOTU0MTJi",
"directoryAlias": null,
"displayName": "<name>",
"id": "8038397f-5a7b-630d-8e59-38f12e95412b",
"imageUrl": "https:<redacted>",
"inactive": null,
"isAadIdentity": null,
"isContainer": null,
"isDeletedInOrigin": null,
"uniqueName": "<email>",
"url": "https://<redacted>"
},
"isLocked": null,
"isLockedBy": null,
"name": "refs/tags/1.0.0-tag-name",
"objectId": "6ed8645a6e1c6686aa9eead0e60365665d76dfb5",
"peeledObjectId": null,
"statuses": null,
"url": "https://<redacted>"
}
]
There does not seem to be a date available. I get the object id and if this was local, I could do git show <objectid>, but there doesn't seem to be anything in the api to get the date or any other specific information, based on an object id
Just to clarify, in my case, the date of tag creation and the date of commit is typically the same so either would be fine
I am afraid that Azure DevOps CLI doesn't support to get the specific information based on Object Id of the Repo tag.
I suggest that you can use Rest API to achieve the requirement: Annotated Tags - Get
Rest API:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/annotatedtags/{objectId}?api-version=6.0-preview.1
Result sample:
{
"name": "refs/tags/v0.1-beta2",
"objectId": "69080710948ac8ba63e44eca2daf0b30f38c428d",
"taggedObject": {
"objectId": "c60be62ebf0e86b5aa01dbb98657b4b7e5905234",
"objectType": "commit"
},
"taggedBy": {
"name": "Norman Paulk",
"email": "Fabrikamfiber16#hotmail.com",
"date": "2017-06-22T04:28:23"
},
"message": "First beta release",
"url": "https://dev.azure.com/fabrikam/c34d5807-1734-4541-ad1c-d16e9ac1faca/_apis/git/repositories/ca93c3a5-87bb-4b5b-a62f-1f971d677c79/annotatedTags/69080710948ac8ba63e44eca2daf0b30f38c428d"
}
You can also use Rest API to list tags of the repo: Refs - List

Add steps to a build definition in AzureDevOps 2019

I'm trying to create ADOS build definitions programmatically. I found a similar question with an answer here: How to create Build Definitions through VSTS REST API
In the answer example, the steps property is empty. I included some steps (taken from a JSON gotten from another build definition using the same API). The result is that the created build definitions has no steps.
I dug into the .NET API browser, and found that there is a BuildProcess classs with a Process property which should take a DesignerProcess for TFVC pipelines (since YAML is only suported for Git repos), DesignerProcess has a Phase property which is readonly, that maybe the reason why it's not creating my steps
However I still need to find out a way to create a builds steps programmatically
However I still need to find out a way to create a builds steps programmatically.
If you don't know what to add to the step property, you can grab request body in developer console window when saving a Classic UI Pipeline.
Here are the detailed steps:
Create a Classic UI with steps you want in ADOS. (Don't save it in this step)
If you are using edge, press F12 to open developer console window. Then choose 'NetWork'.
Click Save and you will find a record called 'definitions'.
Click it and the request body is at the bottom of the page. You will find steps-related information in Process and processParameters properties.
If you are using a different browser, there might be some slight differences in step 2, 3 and 4.
Then you can edit and add the script in your REST API request body.
Here is a simple example of request body that includes a Command Line task.
"process": {
"phases": [
{
"condition": "succeeded()",
"dependencies": [],
"jobAuthorizationScope": 1,
"jobCancelTimeoutInMinutes": 0,
"jobTimeoutInMinutes": 0,
"name": "Agent job 1",
"refName": "Job_1",
"steps": [
{
"displayName": "Command Line Script",
"refName": null,
"enabled": true,
"continueOnError": false,
"timeoutInMinutes": 0,
"alwaysRun": false,
"condition": "succeeded()",
"inputs": {
"script": "echo Hello world\n",
"workingDirectory": "",
"failOnStderr": "false"
},
"overrideInputs": {},
"environment": {},
"task": {
"id": "d9bafed4-0b18-4f58-968d-86655b4d2ce9",
"definitionType": "task",
"versionSpec": "2.*"
}
}
],
"target": {
"type": 1,
"demands": [],
"executionOptions": {
"type": 0
}
},
"variables": {}
}
],
"type": 1,
"target": {
"agentSpecification": {
"metadataDocument": "https://mmsprodea1.vstsmms.visualstudio.com/_apis/mms/images/VS2017/metadata",
"identifier": "vs2017-win2016",
"url": "https://mmsprodea1.vstsmms.visualstudio.com/_apis/mms/images/VS2017"
}
},
"resources": {}
}
What's more, creating YAML pipelines by REST API is not supported currently. Click this question for detailed information.

How to retrieve Work Item linked to specific commit - Azure Devops REST API

I need to be able to retrieve the linked work item of any given specific commit. I'm currently using the following api call
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits/{commitId}?api-version=5.0
with the following response
{
"parents": [],
"treeId": "7fa1a3523ffef51c525ea476bffff7d648b8cb3d",
"push": {
"pushedBy": {
"id": "8c8c7d32-6b1b-47f4-b2e9-30b477b5ab3d",
"displayName": "Chuck Reinhart",
"uniqueName": "fabrikamfiber3#hotmail.com",
"url": "https://vssps.dev.azure.com/fabrikam/_apis/Identities/8c8c7d32-6b1b-47f4-b2e9-30b477b5ab3d",
"imageUrl": "https://dev.azure.com/fabrikam/_api/_common/identityImage?id=8c8c7d32-6b1b-47f4-b2e9-30b477b5ab3d"
},
"pushId": 1,
"date": "2014-01-29T23:33:15.2434002Z"
},
"commitId": "be67f8871a4d2c75f13a51c1d3c30ac0d74d4ef4",
"author": {
"name": "Chuck Reinhart",
"email": "fabrikamfiber3#hotmail.com",
"date": "2014-01-29T23:32:09Z"
},
"committer": {
"name": "Chuck Reinhart",
"email": "fabrikamfiber3#hotmail.com",
"date": "2014-01-29T23:32:09Z"
},
"comment": "First cut\n",
"url": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/commits/be67f8871a4d2c75f13a51c1d3c30ac0d74d4ef4",
"remoteUrl": "https://dev.azure.com/fabrikam/_git/Fabrikam-Fiber-Git/commit/be67f8871a4d2c75f13a51c1d3c30ac0d74d4ef4",
"_links": {
"self": {
"href": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/commits/be67f8871a4d2c75f13a51c1d3c30ac0d74d4ef4"
},
"repository": {
"href": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249"
},
"changes": {
"href": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/commits/be67f8871a4d2c75f13a51c1d3c30ac0d74d4ef4/changes"
},
"web": {
"href": "https://dev.azure.com/fabrikam/_git/Fabrikam-Fiber-Git/commit/be67f8871a4d2c75f13a51c1d3c30ac0d74d4ef4"
},
"tree": {
"href": "https://dev.azure.com/fabrikam/_apis/git/repositories/278d5cd2-584d-4b63-824a-2ba458937249/trees/7fa1a3523ffef51c525ea476bffff7d648b8cb3d"
}
}
}
from https://learn.microsoft.com/en-us/rest/api/azure/devops/git/commits/get?view=azure-devops-rest-5.0 and am missing a way to see what work item its linked to or if it is linked at all. Does anyone know of a way to get this information? Thanks
You could use the Get Commits API, docs here. The base request looks like:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits?api-version=5.0
You could then add the following parameters:
fromCommitId - string - If provided, a lower bound for filtering commits alphabetically
toCommitId - string - If provided, an upper bound for filtering commits alphabetically
includeWorkItems - boolean - Whether to include linked work items
So that your final query would look something like, with your toCommitId and fromCommitId parameters being your commit id that you are after (the documentation doesn't specificy whether these are inclusive or exclusive so your might have to tweak this slightly):
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits?includeWorkItems=true&.toCommitId={searchCriteria.toCommitId}&fromCommitId={searchCriteria.fromCommitId}&api-version=5.0
The result should contain a workItems property inside each commit object of the response as per this documentation.
Note:
Parameters that use the searchCriteria prefix in their name can be specified without it as query parameters, e.g. searchCriteria.$top -> $top
There is also:
ids - array - If provided, specifies the exact commit ids of the commits to fetch. May not be combined with other parameters.
Which could allow you to forgo passing in the to and from commit ids but the docs state that it May not be combined with other parameters - even though the example request does combine it with other parameters. I haven't tried this myself so please do comment when you find out whether you go with from-to id or just ids.
OPs action
The OP ended up using the following request as they didn't mind all commits being returned:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits?includeWorkItems=true&api-version=5.0

Elasticsearch: update existing document by inserting elements to its array fields

Consider the following document
{
"title": "My first blog entry",
"text": "Starting to get the hang of this...",
"tags": [ "testing" ],
"views": 0
}
I need to run kind of an upsert operation. If I encounter data like
{
"id": 1,
"tags": [ "new tag" ]
}
I want to update the existing document with same id. So result should be :
{
"id": 1,
"title": "My first blog entry",
"text": "Starting to get the hang of this...",
"tags": [ "testing", "new tag" ],
"views": 0
}
If the document with same id does not exist, I want to create a new one.
Now in databases like mongoDB, I could use update with $addToSet or $push operation. I could not find similar operation in Elasticsearch.
I read that it can be done by writing scripts in groovy. However, this needs to be done on a file containing 200 million records. I am not sure if I can use groovy in combination with bulk API. Is it possible ?
You dont need to use bulk API for this. You can use an upsert request. Upsert request can ALSO be embedded in the bulk request.
curl -XPOST 'localhost:9200/test/type1/1/_update' -d '{
"script": "if (ctx._source.tags.contains(\"tags\")) {ctx._source.tags += tag;} else {ctx._source.tags = [tag]}",
"params": {
"tag": "newTag"
},
"upsert": {
"title": "My first blog entry",
"text": "Starting to get the hang of this...",
"tags": [
"newTag"
],
"views": 0
}
}'

Resources