I currently have a couple of concurrency issues with the Task-based asynchronous API in the Azure SDK for .Net version 3.0.2-prerelease.
I have a list of web site names
var webSites = new [] { "website1", "website2" };
and from these, I'm using the task based API to create or delete the WebSites. Both occasionally fail:
await Task.WhenAll(webSites.Select(x => webSiteClient.WebSites.CreateAsync(
"westeuropewebspace",
new WebSiteCreateParameters
{
SiteMode = WebSiteMode.Limited,
ComputeMode = WebSiteComputeMode.Shared,
Name = x
WebSpaceName = "something"
}
)));
Seldom, I get an exception complaining that the Server Farm "Default1" already exists. I get that this server farm is implicitly created for Free web sites, but there is currently no way to create this Server Farm through the API before creating the WebSites (only the "DefaultServerFarm" can be).
When deleting, something similar happens:
await Task.WhenAll(webSites.Select(x => webSiteClient.WebSites.DeleteAsync(
"westeuropewebspace",
x,
new WebSiteDeleteParameters
{
DeleteAllSlots = true,
DeleteEmptyServerFarm = true,
DeleteMetrics = true,
}
)));
Often (about every second time), I get an Exception that "website2" could not be found, although it definitely existed. The WebSite is deleted, though.
Update:
I have serialized this second Task.WaitAll into a foreach-loop and I still get the exception. The only difference now is that when deleting "website1" fails, "website2" still exists in the cloud (because the second delete request is not sent) and I have to delete it manually through the portal.
You are right - the create site api also tries to create a server farm implicitly and if called concurrently that may cause conflicts. A safer way is to create a server farm explicitly using API and then use that server farm when creating web sites. That way you explicitly control the placement of sites to server farms and there are no implicit server farm creations.
The Azure SDK API contain a method to create server farm explicitly.
https://github.com/Azure/azure-sdk-for-net/blob/master/src/WebSiteManagement/Generated/ServerFarmOperations.cs
Related
We use the Microsoft.SharePoint.Client library to automate SharePoint work from our workflow engine but yesterday, one of our client informed us they wanted to disable the Legacy Authentication (LegacyAuthProtocolsEnabled to false).
Once I tried it on our end, I ended up getting an Unauthorised exception.
All in good wanting to disable the Legacy Authentication for obvious security reason, but the problem with the Modern Authentication is that it requires user interaction which is clearly not a solution since we are running tasks in the background.
I've been googling this for quite some time but I haven't found a solution as of yet on how to handle automatic authentication for background work.
Is there a way to "authenticate" to SharePoint without any user interaction while LegacyAuthProtocolsEnabled is set to false?
I found an article that suggested using the App Authentication but after reading more about it, I believe this is considered an old method to authenticate and is likely to be deprecated as well over time, but I thought I'd still give it a go just in case but it did not work. When I got to
https://tenant.sharepoint.com/_layouts/15/appregnew.aspx
Where tenant is our company domain name, and I click on the "Create" button after filling in all the relevant fields, I get the following error, which is completely useless:
Sorry, something went wrong
An unexpected error has occurred.
TECHNICAL DETAILS
According to this article HOW TO HARDEN YOUR SHAREPOINT ONLINE ENVIRONMENT BY DISABLING LEGACY AUTHENTICATION, Legacy Authentication was no longer be an option as of the 13/10/2020, yet here we are, and the option is still available in SharePoint 365 and while the article is interesting explain why Legacy Authentication should be switched off, etc... it does not get into any details as to how automated solutions should be handled.
Also found an old thread "LegacyAuthProtocolsEnabled" and Scripted Logons to SharePoint Online? where #DeanWang suggests leaving it turned on as:
All custom CSOM, PowerShell code will stop working
This may also prevent third-party apps from accessing SharePoint
Online resources.
I'm going to stop here as I could keep going and the question is already too long for my liking and bottom line is, does anyone know if there is a way, and what is the best way, to authenticate to SharePoint while running automated "scripts/code" from a background task without requiring any user interaction while the Legacy Authentication is switch off?
Thanks
Update-1
After reading articles after articles, I've yet to connect to SharePoint 365.
I also spend more time on the PnP Framework as recommended by numerous articles. I created a dummy app with the following sample code which is used again in various articles, including this one:
Secure Authentication of SharePoint with PnP Framework with C#(Code)
My code is identical as you can see:
var clientContext = new AuthenticationManager().GetACSAppOnlyContext(
"https://mycompany.sharepoint.com/sites",
"MyClientid",
"MySecretId");
using (clientContext)
{
//Get Lists
var web = clientContext.Web;
var lists = web.Lists;
clientContext.Load(lists);
clientContext.ExecuteQuery();
foreach (var list in lists)
{
}
}
And even though I've granted full control in Azure for the specific test app that's using the specific ClientId and SecretId
I'm still getting the following error (401 - unauthorized):
System.Exception
HResult=0x80131500
Message=Token request failed.
Source=PnP.Framework
StackTrace:
at SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2S2SClient.Issue(String securityTokenServiceUrl, OAuth2AccessTokenRequest oauth2Request) in /_/src/lib/PnP.Framework/Utilities/OAuth/OAuth2S2SClient.cs:line 18
at PnP.Framework.Utilities.TokenHelper.GetAppOnlyAccessToken(String targetPrincipalName, String targetHost, String targetRealm) in /_/src/lib/PnP.Framework/Utilities/TokenHelper.cs:line 116
at PnP.Framework.Utilities.ACSTokenGenerator.GetToken(Uri siteUrl) in /_/src/lib/PnP.Framework/Utilities/ACSTokenGenerator.cs:line 37
at PnP.Framework.AuthenticationManager.<GetContextAsync>b__59_0(String site) in /_/src/lib/PnP.Framework/AuthenticationManager.cs:line 971
at PnP.Framework.AuthenticationManager.<>c__DisplayClass75_0.<GetAccessTokenContext>b__0(Object sender, WebRequestEventArgs args) in /_/src/lib/PnP.Framework/AuthenticationManager.cs:line 1336
at Microsoft.SharePoint.Client.ClientRuntimeContext.OnExecutingWebRequest(WebRequestEventArgs args)
at Microsoft.SharePoint.Client.ClientContext.FireExecutingWebRequestEventInternal(WebRequestEventArgs args)
at Microsoft.SharePoint.Client.ClientContext.GetWebRequestExecutor()
at Microsoft.SharePoint.Client.ClientContext.GetFormDigestInfoPrivate()
at Microsoft.SharePoint.Client.ClientContext.EnsureFormDigest()
at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()
at ConsoleApp5.Program.Main(String[] args) in C:\Users\myuser\source\repos\ConsoleApp5\ConsoleApp5\Program.cs:line 23
This exception was originally thrown at this call stack:
[External Code]
SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2WebRequest.GetResponse() in OAuth2WebRequest.cs
SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2S2SClient.Issue(string, SharePointPnP.IdentityModel.Extensions.S2S.Protocols.OAuth2.OAuth2AccessTokenRequest) in OAuth2S2SClient.cs
Inner Exception 1:
WebException: The remote server returned an error: (401) Unauthorized.
Is there another section I should be looking at (and change) in the App Registration in Azure
Since it's the SharePoint Online that we are talking about, one easy way to connect to different SharePoint Sites is by using the Azure AD App-Only approach and since you are talking about a Deamon Service you can easily use Application Permissions when registering the App Registration.
You can, and you should, read more about it from the linked Microsoft Docs article.
You can also loggin via certificate or app registration secret as it is discribed in the Log in to Microsoft 365 in order to create automated CI CD SPFx pipelines, for example.
Hope the above helps, if not feel free to ask :)
Update: Please read below in order to have a better understanding.
Firstly, in your code segment you are using a wrong method from the PnP.Framework package.
AuthenticationManager().GetACSAppOnlyContext()
The above method refers to a completely different method of obtaining an authentication token, more specifically the Sharepoint App-Only model, which... well.... more or less is not being used nowadays quite so ofte. I think I read somewhere that MS is thinking of retiring this kind of Authentication and going onwards on the path of Azure Active Directory authentication, but, unfotunately, I cannot seem to find the link.
Furthermore, I have collected three projects and uploaded them to github for you to see. You can simply clone the repo and run the projects as-is from HERE.
As you will be able to see for yourself, there are three projects in the solution, which you can run each one individually from VSCode or Vs.
More in detail:
ConsoleApp1
(sorry for the name but forgot to switch it :) )
This is a Deamon Console Project that references the PnP.Framework namespace and tries to utilize all of the goodies that the good folks form the PnP Community have contributed.
The procedure is straight forward and is the same for all three projects ->
Read the AppConfiguration
Request the Access Token with appropriate scopes (Depending the service that i am referencing)
Declare the Token to be used by our Client Context.
In the PnP.Framework-related project the above cycle can be seen as below
AuthenticationConfiguration config = AuthenticationConfiguration.ReadFromJsonFile("appsettings.json");
var authManager = new PnP.Framework.AuthenticationManager(config.ClientId, config.Certificate.CertificateDiskPath, config.Certificate.CertificatePassword, config.Tenant);
using (var cc = authManager.GetAccessTokenContext("https://<REPLACE:name of tenant>.sharepoint.com/sites/testsite2", (string siteURL) => authManager.GetAccessToken(siteURL)))
ConsoleAppMSGraph
As the name suggests this Deamon Console App utilizes GraphServiceClient graphClient in order to get all the information that you request through the graph endpoint.
Subsequntly, you will notice that for this porject the scope name changes to
string[] scopes = new string[] { "https://graph.microsoft.com/.default" };
In addition, we request a collection of all the lists that currently reside in our SharePoint Root Site with the below segment:
var lists = await graphClient.Sites["root"].Lists
.Request()
.GetAsync();
ConsoleAppSPClient
This app is the default and most simple way of accessing data on Sharepoint.
The projects utilizes MSAL.Net and Microsoft.Sharepoint.Client namespaces in order to fetch an access token and, subsequently, embed that token in all our next requests.
In order to keep the answer a bit short, please refer to here in order to see how we initiate a Confidential App Client, request for a token and, later on, embedd it in our ClientContext object.
Notes
I have listed in the Readme.md of the repo, which permissions you should give to your app registration. You can view them Here.
I am using the Sites.FullControl.All but you can narrow down the list of sites that the app registration will have access by using the Sites.Selected.
All of the above projects, reference a common class library that serves as a strongly typed configuration object.
IMPORTANT you should always use a certificate to authenticate the client app as it is mentioned here. The previous link also describes the way you can create a certificate and upload it to the store of the app registration.
Amazing! Thank you very much #Jimas13. For the last 2 weeks I was struggling to find solution to my problem!! You saved me!! If you ever been in Greece let me buy you a drink!
I'm looking to use the Change Feed Processor SDK to monitor for changes to an Azure Cosmos DB collection, however, I have not seen clear documentation about whether the host can be run as an Azure Web Job. Can it? And if yes, are there any known issues or limitations versus running it as a Console App?
There are a good number of blog posts about using the CFP SDK, however, most of them vaguely mention running the host on a VM, and none of them or any examples running the host as an azure web job.
Even if it's possible, as a side question is, if such a host is deployed as a continuous web job and I select the "Scale" setting of the web job to Multi Instance, what are the approaches or recommendations to make the extra instances run with a different instance name, which the CFP SDK requires?
According to my research,Cosmos db trigger could be implemented in the WebJob SDK.
static async Task Main()
{
var builder = new HostBuilder();
builder.ConfigureWebJobs(b =>
{
b.AddAzureStorageCoreServices();
b.AddCosmosDB(a =>
{
a.ConnectionMode = ConnectionMode.Gateway;
a.Protocol = Protocol.Https;
a.LeaseOptions.LeasePrefix = "prefix1";
});
});
var host = builder.Build();
using (host)
{
await host.RunAsync();
}
}
But it seems only Nuget for c# sdk could be used,no clues for other languages.So,you could refer to the Compare Functions and WebJobs to balance your needs and cost.
The Cosmos DB Trigger for Azure Functions it's actually, a WebJobs extension: https://github.com/Azure/azure-webjobs-sdk-extensions/tree/dev/src/WebJobs.Extensions.CosmosDB
And it uses the Change Feed Processor.
Functions run over WebJob technology. So to answer the question, yes, you can run Change Feed Processor on WebJobs, just make sure that:
Your App Service is set to Always On
If you plan to use multiple instances, make sure to set the InstanceName accordingly and not a static/fixed value. Probably something that identifies the WebJob instance.
I have a WebJob that needs to create a JWT token to talk with an external service. The following code works when I run the WebJob on my local machine:
public static string SignES256(byte[] p8Certificate, object header, object payload)
{
var headerString = JsonConvert.SerializeObject(header);
var payloadString = JsonConvert.SerializeObject(payload);
CngKey key = CngKey.Import(p8Certificate, CngKeyBlobFormat.Pkcs8PrivateBlob);
using (ECDsaCng dsa = new ECDsaCng(key))
{
dsa.HashAlgorithm = CngAlgorithm.Sha256;
var unsignedJwtData = Base64UrlEncoder.Encode(Encoding.UTF8.GetBytes(headerString)) + "." + Base64UrlEncoder.Encode(Encoding.UTF8.GetBytes(payloadString));
var signature = dsa.SignData(Encoding.UTF8.GetBytes(unsignedJwtData));
return unsignedJwtData + "." + Base64UrlEncoder.Encode(signature);
}
}
However, when I deploy my WebJob to Azure, I get the following exception:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: NotificationFunctions.QueueOperation ---> System.Security.Cryptography.CryptographicException: The system cannot find the file specified. at System.Security.Cryptography.NCryptNative.ImportKey(SafeNCryptProviderHandle provider, Byte[] keyBlob, String format) at System.Security.Cryptography.CngKey.Import(Byte[] keyBlob, CngKeyBlobFormat format, CngProvider provider)
It says it can't find a specified file, but the parameters I am passing in are not looking at a file location, they are in memory. From what I have gathered, there may be some kind of cryptography setting I need to enable to be able to use the CngKey.Import method, but I can't find any settings in the Azure portal to configure related to this.
I have also tried using JwtSecurityTokenHandler, but it doesn't seem to handle the ES256 hashing algorithm I need to use (even though it is referenced in the JwtAlgorithms class as ECDSA_SHA256).
Any suggestions would be appreciated!
UPDATE
It appears that CngKey.Import may actually be trying to store the certificate somewhere that is not accessible on Azure. I don't need it stored, so if there is a better way to access the certificate in memory or convert it to a different kind of certificate that would be easier to use that would work.
UPDATE 2
This issue might be related to Azure Web Apps IIS setting not loading the user profile as mentioned here. I have enabled this by setting WEBSITE_LOAD_USER_PROFILE = 1 in the Azure portal app settings. I have tried with this update when running the code both via the WebJob and the Web App in Azure but I still receive the same error.
I used a decompiler to take a look under the hood at what the CngKey.Import method was actually doing. It looks like it tries to insert the certificate I am using into the "Microsoft Software Key Storage Provider". I don't actually need this, just need to read the value of the certificate but it doesn't look like that is possible.
Once I realized a certificate is getting inserted into a store somewhere one the machine, I started thinking about how bad of a think that would be from a security standpoint if your Azure Web App was running in a shared environment, like it does for the Free and Shared tiers. Sure enough, my VM was on the Shared tier. Scaling it up to the Basic tier resolved this issue.
We're moving most of our web presence to our SharePoint server in the cloud. Our current setup uses a MVC Web API for data retrieval from DB. We do not want to host the API under a separate domain and thus need to move the API under SharePoint domain as well. There is no relaxation in this requirement.
Is there a way to publish my API to SharePoint? Or is there a SharePoint specific API project template in Visual Studio? If not what are my options?
EDIT Initially I have asked that MVC API needs to be part of the SharePoint 2013. But now things are such that API can reside anywhere - inside or outside - of SharePoint, as long as it is accessible from the root domain - which so far it seems not allowed (Error message: Calls to WebProxy without an app context are not allowed."). Still trying to see if this is possible, and if yes, how?
It sounds like the proxy you want to create is already part of SharePoint JSOM. Have a look at these:
http://msdn.microsoft.com/en-us/library/office/fp179895(v=office.15).aspx
http://msdn.microsoft.com/en-us/library/office/jj245162(v=office.15).aspx
This will allow you to overcome cross origin issues. The SP.WebProxy and SP.WebRequestInfo allow you to use javascript to make a call outside of the domain where the javascript executes.
What really happens behind the scenes is that SharePoint's javascript API sends the request to your sharepoint.com tenancy server, which will then invoke the service from the SharePoint server, and return the response back to your javascript. You can implement it like so in a sharepoint-hosted app:
// this javascript executes from my-company.sharepoint.com
var responseDocument = undefined;
$('#cross').click(function () {
var ctx = SP.ClientContext.get_current();
var request = new SP.WebRequestInfo();
request.set_url('https://www.somewebapi.com/my/custom/route');
request.set_method("GET");
responseDocument = SP.WebProxy.invoke(ctx, request); // executes on sp server
ctx.executeQueryAsync(onSuccess, onError);
});
function onSuccess() {
var response = responseDocument.get_body();
alert('success ' + response);
}
function onError(err) {
alert(JSON.stringify(err));
}
...and since the remote api hosted at the other domain is called from the server, you don't have to worry about any of the cross-domain issues.
Update
To answer your update, please check the results from this link.
Have you added the remote endpoint to your AppManifest.xml?
SharePoint doesn't give you a chance to define you own routes. Thats why you can not use old fashioned SharePoint solution to publish asp.net web api. You may consider using apps for SharePoint. It's like separate App with some connections to SharePoint.
Ultimately switched to JSONP solution. Installed the WebApiContrib.Formatting.JsonP in my MVC Web API project in Visual Studio, and modified SharePoint JavaScript, that calls the API, to include ?callback=? (callback is equal to question mark). Everything stays the same. No SharePoint's proxy caller needed! No SharePoint app needed!
This is a question for a WSS/SharePoint guru.
Consider this scenario: I have an ASP.Net web service which links our corporate CRM system and WSS-based intranet together. What I am trying to do is provision a new WSS site collection whenever a new client is added to the CRM system. In order to make this work, I need to programmatically add the managed path to the new site collection. I know that this is possible via the Object Model, but when I try it in my own web service, it fails. Sample code extract below:
Dim _ClientSiteUrl As String = "http://myintranet/clients/sampleclient"
Using _RootWeb As SPSite = New SPSite("http://myintranet")
Dim _ManagedPaths As SPPrefixCollection = _RootWeb.WebApplication.Prefixes
If Not (_ManagedPaths.Contains(_ClientSiteUrl)) Then
_ManagedPaths.Add(_ClientSiteUrl, SPPrefixType.ExplicitInclusion)
End If
End Using
This code fails with a NullReferenceException on SPUtility.ValidateFormDigest(). Research suggested that this may be due to insufficient privileges, I tried running the code within an elevated privileges block using SPSecurity.RunWithElevatedPrivileges(AddressOf AddManagedPath), where AddManagedPath is a Sub procedure containing the above code sample.
This then fails with an InvalidOperationException, "Operation is not valid due to the current state of the object."
Where am I going wrong?
One workaround I have managed to do is to call out to STSADM.EXE via Process.Start(), supplying the requisite parameters, and this works.
Update: whilst developing the web service, I am running it using the built-in Visual Studio 2005 web server - what security context will this be running under? Can I change the security context by putting entries in web.config?
Update: I think the problem is definitely to do with not running the web service within the correct SharePoint security context. I decided to go with the workaround I suggested and shell out to STSADM, although to do this, the application pool identity that the web service runs under must be a member of the SharePoint administrators.
Update
I think you have proved that the issue is not with the code.
SPSecurity.RunWithElevatedPrivileges: Normally the code in the SharePoint web application executes with the privileges of the user taking the action. The RunWithElevatedPrivileges runs the code in the context of the SharePoint web application pools account (i think)
The description on MSDN could go into the details a tiny bit more.
The issue with the call may be that the web service is not actually running the code within a SharePoint process, so explaining why it cannot elevate (wild guess alert).
Have a crack at changing the user of your web services application pool and see if that gives any joy.
It is likely to be a permissions issue.
Maybe try:
Dim clientSiteUrl As String = "http://myintranet/clients/sampleclient"
Using SPSite = new SPSite(clientSiteUrl)
webApp As SPWebApplication = SPWebApplication.Lookup(new Uri(clientSiteUrl));
If Not (webApp.Prefixes.Contains(clientSiteUrl)) Then
webApp.Prefixes.Add(clientSiteUrl, SPPrefixType.ExplicitInclusion)
End If
End Using
This is not exact code.
Since the above code is not the exact code, here is the exact working code for a Web Application scopped feature in the Feature Activated event:
On feature activation at the Mange web application features page, activate feature will create a new Explicit managed path in the specified web application (I want to replace the hard coding, maybe with Properties.Feature.Parent, or something similar.)
using (SPSite site = new SPSite("http://dev-moss07-eric/PathHere")) {
SPWebApplication webApp = SPWebApplication.Lookup(new Uri("http://dev-moss07-eric"));
if (webApp.Prefixes.Contains("PathHere"))
{
//
}
else
{
webApp.Prefixes.Add("PathHere", SPPrefixType.ExplicitInclusion);
}
}
Code can probably be improved, but its my attempt at converting the above code.
If you want to create a managed path (explicit) and a site collection at that path, do the following:
using (SPSite site = new SPSite("http://dev-moss07-eric")) {
SPWebApplication webApp = SPWebApplication.Lookup(new Uri("http://dev-moss07-eric"));
if (webApp.Prefixes.Contains("ManagedPathHere"))
{
//
}
else
{
webApp.Prefixes.Add("ManagedPathHere", SPPrefixType.ExplicitInclusion);
}
using (SPWeb web = site.OpenWeb())
{
SPWebApplication webApplication = web.Site.WebApplication;
try
{
webApplication.Sites.Add("ManagedPathHere","Site Title Here","This site is used for hosting styling assets.", 1033, "STS#1", "6scdev\\eric.schrader", "Eric Schrader", "eric.schrader#6sc.com");
}
catch (Exception ex)
{
//ex.ToString;
}
}
}