I would like all my projects in a GitLab group to have shared configuration for a webhook:
<MY_JENKINS_INSTANCE>/git/notifyCommit?url=$CHANGED_REPOSITORY
GitLab webhook documentation suggests it should be possible:
If you have a big set of projects in the one group then it will be convenient for you to configure web hooks globally for the whole group. You can add the group level web hooks on the group settings page.
That sound exactly like what I am after though I see no such thing on group settings page in my gitlab 7.0.0. I was not able to find out if this feature is not newer than that in the changelog.
Does the feature exist? How do I use it?
That's possible in the enterprise version only:
In GitLab Enterprise Edition you can configure web hooks globally for the whole group. You can add the group level web hooks on the group settings page Settings > Web Hooks.
Following up on #VertigoRay's comments, here's a procedure to do it using GitLab CE API:
Have, or create an user in GitLab and a personal access token with api scope:
User (top right avatar) > Settings (menu) > Access tokens (sidebar)
Check api scope (checkbox)
Click on create personal access token (button)
<my_personal_token> is the value in Your New Personal Access Token (text field)
Perform an HTTP request to get all projects:
GET https://gitlab.example.com/api/v4/projects
Private-Token: <my_personal_token>
Accept: application/json
For each project in the response:
id which is the <project_ID> to be used in the next request URL
Convert the value of ssh_url_to_repo so that it becomes URL encoded <encoded_ssh_url>
Example: ssh://git#example.com:1234/group/alpha.git becomes ssh%3A%2F%2Fgit%40example.com%3A1234%2Fgroup%2Falpha.git
For each project, perform an HTTP request to create a hook:
POST https://gitlab.example.com/api/v4/projects/<project_ID>/hooks
Private-Token: <my_personal_token>
Content-Type: application/json
{
"url": "https://jenkins.example.com/git/notifyCommit?url=<encoded_ssh_url>",
"enable_ssl_verification": true
}
This should be scripted in the langage of your choice.
Not suitable as a persistent solution, but this might be useful for someone looking for a one-time change (from the raketasks documentation):
Add a webhook for projects in a given NAMESPACE
# omnibus-gitlab
sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme
# source installations
bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
Related
We use the Microsoft.SharePoint.Client library to automate SharePoint work from our workflow engine but yesterday, one of our client informed us they wanted to disable the Legacy Authentication (LegacyAuthProtocolsEnabled to false).
Once I tried it on our end, I ended up getting an Unauthorised exception.
All in good wanting to disable the Legacy Authentication for obvious security reason, but the problem with the Modern Authentication is that it requires user interaction which is clearly not a solution since we are running tasks in the background.
I've been googling this for quite some time but I haven't found a solution as of yet on how to handle automatic authentication for background work.
Is there a way to "authenticate" to SharePoint without any user interaction while LegacyAuthProtocolsEnabled is set to false?
I found an article that suggested using the App Authentication but after reading more about it, I believe this is considered an old method to authenticate and is likely to be deprecated as well over time, but I thought I'd still give it a go just in case but it did not work. When I got to
https://tenant.sharepoint.com/_layouts/15/appregnew.aspx
Where tenant is our company domain name, and I click on the "Create" button after filling in all the relevant fields, I get the following error, which is completely useless:
Sorry, something went wrong
An unexpected error has occurred.
TECHNICAL DETAILS
According to this article HOW TO HARDEN YOUR SHAREPOINT ONLINE ENVIRONMENT BY DISABLING LEGACY AUTHENTICATION, Legacy Authentication was no longer be an option as of the 13/10/2020, yet here we are, and the option is still available in SharePoint 365 and while the article is interesting explain why Legacy Authentication should be switched off, etc... it does not get into any details as to how automated solutions should be handled.
Also found an old thread "LegacyAuthProtocolsEnabled" and Scripted Logons to SharePoint Online? where #DeanWang suggests leaving it turned on as:
All custom CSOM, PowerShell code will stop working
This may also prevent third-party apps from accessing SharePoint
Online resources.
I'm going to stop here as I could keep going and the question is already too long for my liking and bottom line is, does anyone know if there is a way, and what is the best way, to authenticate to SharePoint while running automated "scripts/code" from a background task without requiring any user interaction while the Legacy Authentication is switch off?
Thanks
Update-1
After reading articles after articles, I've yet to connect to SharePoint 365.
I also spend more time on the PnP Framework as recommended by numerous articles. I created a dummy app with the following sample code which is used again in various articles, including this one:
Secure Authentication of SharePoint with PnP Framework with C#(Code)
My code is identical as you can see:
var clientContext = new AuthenticationManager().GetACSAppOnlyContext(
"https://mycompany.sharepoint.com/sites",
"MyClientid",
"MySecretId");
using (clientContext)
{
//Get Lists
var web = clientContext.Web;
var lists = web.Lists;
clientContext.Load(lists);
clientContext.ExecuteQuery();
foreach (var list in lists)
{
}
}
And even though I've granted full control in Azure for the specific test app that's using the specific ClientId and SecretId
I'm still getting the following error (401 - unauthorized):
System.Exception
HResult=0x80131500
Message=Token request failed.
Source=PnP.Framework
StackTrace:
at SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2S2SClient.Issue(String securityTokenServiceUrl, OAuth2AccessTokenRequest oauth2Request) in /_/src/lib/PnP.Framework/Utilities/OAuth/OAuth2S2SClient.cs:line 18
at PnP.Framework.Utilities.TokenHelper.GetAppOnlyAccessToken(String targetPrincipalName, String targetHost, String targetRealm) in /_/src/lib/PnP.Framework/Utilities/TokenHelper.cs:line 116
at PnP.Framework.Utilities.ACSTokenGenerator.GetToken(Uri siteUrl) in /_/src/lib/PnP.Framework/Utilities/ACSTokenGenerator.cs:line 37
at PnP.Framework.AuthenticationManager.<GetContextAsync>b__59_0(String site) in /_/src/lib/PnP.Framework/AuthenticationManager.cs:line 971
at PnP.Framework.AuthenticationManager.<>c__DisplayClass75_0.<GetAccessTokenContext>b__0(Object sender, WebRequestEventArgs args) in /_/src/lib/PnP.Framework/AuthenticationManager.cs:line 1336
at Microsoft.SharePoint.Client.ClientRuntimeContext.OnExecutingWebRequest(WebRequestEventArgs args)
at Microsoft.SharePoint.Client.ClientContext.FireExecutingWebRequestEventInternal(WebRequestEventArgs args)
at Microsoft.SharePoint.Client.ClientContext.GetWebRequestExecutor()
at Microsoft.SharePoint.Client.ClientContext.GetFormDigestInfoPrivate()
at Microsoft.SharePoint.Client.ClientContext.EnsureFormDigest()
at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()
at ConsoleApp5.Program.Main(String[] args) in C:\Users\myuser\source\repos\ConsoleApp5\ConsoleApp5\Program.cs:line 23
This exception was originally thrown at this call stack:
[External Code]
SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2WebRequest.GetResponse() in OAuth2WebRequest.cs
SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2S2SClient.Issue(string, SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2AccessTokenRequest) in OAuth2S2SClient.cs
Inner Exception 1:
WebException: The remote server returned an error: (401) Unauthorized.
Is there another section I should be looking at (and change) in the App Registration in Azure
Since it's the SharePoint Online that we are talking about, one easy way to connect to different SharePoint Sites is by using the Azure AD App-Only approach and since you are talking about a Deamon Service you can easily use Application Permissions when registering the App Registration.
You can, and you should, read more about it from the linked Microsoft Docs article.
You can also loggin via certificate or app registration secret as it is discribed in the Log in to Microsoft 365 in order to create automated CI CD SPFx pipelines, for example.
Hope the above helps, if not feel free to ask :)
Update: Please read below in order to have a better understanding.
Firstly, in your code segment you are using a wrong method from the PnP.Framework package.
AuthenticationManager().GetACSAppOnlyContext()
The above method refers to a completely different method of obtaining an authentication token, more specifically the Sharepoint App-Only model, which... well.... more or less is not being used nowadays quite so ofte. I think I read somewhere that MS is thinking of retiring this kind of Authentication and going onwards on the path of Azure Active Directory authentication, but, unfotunately, I cannot seem to find the link.
Furthermore, I have collected three projects and uploaded them to github for you to see. You can simply clone the repo and run the projects as-is from HERE.
As you will be able to see for yourself, there are three projects in the solution, which you can run each one individually from VSCode or Vs.
More in detail:
ConsoleApp1
(sorry for the name but forgot to switch it :) )
This is a Deamon Console Project that references the PnP.Framework namespace and tries to utilize all of the goodies that the good folks form the PnP Community have contributed.
The procedure is straight forward and is the same for all three projects ->
Read the AppConfiguration
Request the Access Token with appropriate scopes (Depending the service that i am referencing)
Declare the Token to be used by our Client Context.
In the PnP.Framework-related project the above cycle can be seen as below
AuthenticationConfiguration config = AuthenticationConfiguration.ReadFromJsonFile("appsettings.json");
var authManager = new PnP.Framework.AuthenticationManager(config.ClientId, config.Certificate.CertificateDiskPath, config.Certificate.CertificatePassword, config.Tenant);
using (var cc = authManager.GetAccessTokenContext("https://<REPLACE:name of tenant>.sharepoint.com/sites/testsite2", (string siteURL) => authManager.GetAccessToken(siteURL)))
ConsoleAppMSGraph
As the name suggests this Deamon Console App utilizes GraphServiceClient graphClient in order to get all the information that you request through the graph endpoint.
Subsequntly, you will notice that for this porject the scope name changes to
string[] scopes = new string[] { "https://graph.microsoft.com/.default" };
In addition, we request a collection of all the lists that currently reside in our SharePoint Root Site with the below segment:
var lists = await graphClient.Sites["root"].Lists
.Request()
.GetAsync();
ConsoleAppSPClient
This app is the default and most simple way of accessing data on Sharepoint.
The projects utilizes MSAL.Net and Microsoft.Sharepoint.Client namespaces in order to fetch an access token and, subsequently, embed that token in all our next requests.
In order to keep the answer a bit short, please refer to here in order to see how we initiate a Confidential App Client, request for a token and, later on, embedd it in our ClientContext object.
Notes
I have listed in the Readme.md of the repo, which permissions you should give to your app registration. You can view them Here.
I am using the Sites.FullControl.All but you can narrow down the list of sites that the app registration will have access by using the Sites.Selected.
All of the above projects, reference a common class library that serves as a strongly typed configuration object.
IMPORTANT you should always use a certificate to authenticate the client app as it is mentioned here. The previous link also describes the way you can create a certificate and upload it to the store of the app registration.
Amazing! Thank you very much #Jimas13. For the last 2 weeks I was struggling to find solution to my problem!! You saved me!! If you ever been in Greece let me buy you a drink!
I have added a slack service template to my gitlab-ce via the administrators interface. Everything worked. I activated "active by default".
As a result, all projects now push notifications into the main channel. And those are a lot.
Changing the service template configuration is not inherit by the projects. Thus effectively rendering me unable to revert the setting via the admin UI.
So, how can I disable the slack service integrations for all projects before it drives all of us crazy because the general-channel is just flooded by gitlab?
That is followed by issue 40921:
Allow to apply service template to all projects
Sometimes users want to apply the same integration like JIRA across all GitLab projects, currently templates are the only way to do that through the UI, but project integration templates only works for projects that have been created after it.
Only workaround:
I had this issue too. One workaround is to patch the database like this:
sudo gitlab-rails dbconsole
UPDATE services SET properties = replace(properties, 'http://someoldurl.com', 'https://somenewurl.com');
(to be adapted to your slack setting: this is just an example)
Following #VonC's advice to dive into the depths of psql and hack my way through, I finally ran following command to disable the active flag for the relevant services (slack and mattermost in our case):
sudo gitlab-rails dbconsole
UPDATE "services" SET active = FALSE WHERE type LIKE 'SlackService' AND active = TRUE;
UPDATE "services" SET active = FALSE WHERE type LIKE 'SlackSlashCommandsService' AND active = TRUE;
UPDATE "services" SET active = FALSE WHERE type LIKE 'MattermostService' AND active = TRUE;
UPDATE "services" SET active = FALSE WHERE type LIKE 'MattermostSlashCommandsService' AND active = TRUE;
I have an HTTP triggered, NodeJs Azure Function, and I'm looking to remove the "X-Powered-By" header from my response, but have found no way to do so.
I've tried adding both this and this azure site extensions, but neither has worked for me,
Setting the response header manually, i.e. res.headers = { ['x-powered-by']: null } is ineffective.
Based on the comments made on this github issue: https://github.com/Azure/Azure-Functions/issues/290 it would seem that using either extension should have removed the headers you wanted.
Modifying the response headers will likely won't work as they are probably added further down the pipeline by the function host and not overridable, see:
Access Azure Function runtime settings
Azure functions recently removed the x-aspnet-version header, further removal of other headers is tracked as part of the azure-webjobs-script-sdk here
You should leave a comment on the github issue and you can further discuss with the team working on this.
There is an extension called Remove Custom Headers that works for Web Apps but not for functions that have their own resource group. So, what you can do is:
1. Create a regular Web App
2. Create a function and make sure you use the same Hosting Plan as the Web App (do not use Consumption).
3. Once the function is created, install the extension named: "Remove Custom Headers"
4. Restart the function and the headers (Server and X-Powered-By) should disappear.
Here's what I'm doing.
Install latest Azure SDK (by the date)
Open Visual Studio 2013
Create a new Azure Mobile App project. The simple service is created with TodoItem DataObject and 2 simple controllers - TodoItemController and ValuesController
Do not change anything
Start the project
The service is started and hosted in local IISExpress on url http://localhost:50993/ (the port may vary).
The "This mobile app is up and running" web page is opened in the browser. But http 400 error is returned when I try to invoke some GET-actions: for example http://localhost:50993/api/values or http://localhost:50993/tables/TodoItem.
Any ideas? Is something wrong in my environment or is that me doing something wrong?
Thanks.
I guess you can opt out of version checking by setting a value of true for the app setting MS_SkipVersionCheck. Specify this either in your web.config or in the Application Settings section of the Azure Portal.
This generally happens when you don't add a ZUMO-API-VERSION header to the request. This is required when making requests from a REST client, but the mobile client SDKs add the header automatically.
To fix, add the header ZUMO-API-VERSION with value of 2.0.0.
To learn more, see https://azure.microsoft.com/en-us/documentation/articles/app-service-mobile-client-and-server-versioning/.
Adding
http://localhost:50993/tables/Location?ZUMO-API-VERSION=2.0.0
at the end will do the trick.
I have connected Visual Studio Online to my Azure website. This is not a .NET ASP.NET MVC project, just several static HTML files.
Now I want to get my files uploaded to Azure and available 'online' after my commits/pushes to the TFS.
When a build definition (based on GitContinuousDeploymentTemplate.12.xaml) is executed it fails with an obvious message:
Exception Message: The process parameter ProjectsToBuild is required but no value was set.
My question: how do I setup a build definition so that it automatically copies my static files to Azure on commits?
Or do I need to use a different tooling for this task (like WebMatrix).
update
I ended up with creating an empty website and deploying it manually from Visual Studio using webdeploy. Other possible options to consider to create local Git at Azure.
Alright, let me try to give you an answer:
I was having quite a similar issue. I had a static HTML, JS and CSS site which I needed to have in TFS due to the project and wanted to make my life easier using the continuous deployment. So what I did was following:
When you have a Git in TFS, you get an URL for the repository - something like:
https://yoursite.visualstudio.com/COLLECTION/PROJECT/_git/REPOSITORY
, however in order to access the repository itself, you need to authenticate, which is not currently possible, if you try to put the URL with authentication into Azure:
https://username:password#TFS_URL
It will not accept it. So what you do, in order to bind the deployment is that you just put the URL for repository there (the deployment will fail, however it will prepare the environment for us to proceed).
However, when you link it there, you can get DEPLOYMENT TRIGGER URL on the Configure tab of the Website. What it is for is that when you push a change to your repository (say to GitHub) what happens is that GitHub makes a HTTP POST request to that link and it tells Azure to deploy new code onto the site.
Now I went to Kudu which is the underlaying system of Azure Websites which handles the deployments. I figured that if you send correct contents in the HTTP POST (JSON format) to the DEPLOYMENT TRIGGER URL, you can have it deploy code from any repository and it even authenticates!
So the thing left to do is to generate the alternative authentication credentials on the TFS site and put the whole request together. I wrapped this entire process into the following PowerShell script:
# Windows Azure Website Configuration
#
# WAWS_username: The user account which has access to the website, can be obtained from https://manage.windowsazure.com portal on the Configure tab under DEPLOYMENT TRIGGER URL
# WAWS_password: The password for the account specified above
# WAWS: The Azure site name
$WAWS_username = ''
$WAWS_password = ''
$WAWS = ''
# Visual Studio Online Repository Configuration
#
# VSO_username: The user account used for basic authentication in VSO (has to be manually enabled)
# VSO_password: The password for the account specified above
# VSO_URL: The URL to the Git repository (branch is specified on the https://manage.windowsazure.com Configuration tab BRANCH TO DEPLOY
$VSO_username = ''
$VSO_password = ''
$VSO_URL = ''
# DO NOT EDIT ANY OF THE CODE BELOW
$WAWS_URL = 'https://' + $WAWS + '.scm.azurewebsites.net/deploy'
$BODY = '
{
"format": "basic",
"url": "https://' + $VSO_username + ':' + $VSO_password + '#' + $VSO_URL + '"
}'
$authorization = "Basic "+[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($WAWS_username+":"+$WAWS_password ))
$bytes = [System.Text.Encoding]::ASCII.GetBytes($BODY)
$webRequest = [System.Net.WebRequest]::Create($WAWS_URL)
$webRequest.Method = "POST"
$webRequest.Headers.Add("Authorization", $authorization)
$webRequest.ContentLength = $bytes.Length
$webRequestStream = $webRequest.GetRequestStream();
$webRequestStream.Write($bytes, 0, $bytes.Length);
$webRequest.GetResponse()
I hope that what I wrote here makes sense. The last thing you would need is to bind this script to a hook in Git, so when you perform a push the script gets automatically triggered after it and the site is deployed. I haven't figured this piece yet tho.
This should also work to deploy a PHP/Node.js and similar code.
The easiest way would be to add them to an empty ASP .NET project, set them to be copied to the output folder, and then "build" the project.
Failing that, you could modify the build process template, but that's a "last resort" option.