I have 2 sharepoint sites running out of one sharepoint installation. One site has claims based enabled and the other has classic auth enabled. Both sites also both use Kerberos.
I am using ManifoldCF to connect to these sites to extract the all content as well as the permissions. The ManifoldCF connector connects to the site with classic auth enabled and works as expected. However, trying to crawl the claims based site generates a 401 unauthorised error.
There is a web service package supplied with ManifoldCF that is accessed called MCPermissions.asmx. This file contains the following block of code which sets the user's credentials:
try
{
// Only handle requests for "item". Send all other requests to the SharePoint web service.
if (objectType.Equals(itemType))
{
retVal = GetItemPermissions(objectName);
}
else
{
ServicePointManager.ServerCertificateValidationCallback +=
new RemoteCertificateValidationCallback(ValidateCertificate);
using (SPPermissionsService.Permissions service = new SPPermissionsService.Permissions())
{
service.Url = SPContext.Current.Web.Url + "/_vti_bin/Permissions.asmx";
service.Credentials = System.Net.CredentialCache.DefaultNetworkCredentials;
retVal = service.GetPermissionCollection(objectName, objectType);
}
}
}
catch (SoapException soapEx)
{
throw soapEx;
}
catch (Exception ex)
{
SPDiagnosticsService.Local.WriteTrace(0, new SPDiagnosticsCategory("MCPermissions.asmx", TraceSeverity.Unexpected, EventSeverity.Error), TraceSeverity.Unexpected, "Error: "+ex.Message+"; SPContext.Current.Web.Url='"+SPContext.Current.Web.Url+"'", ex.StackTrace);
throw RaiseException(ex.Message, "1000", ex.Source);
}
This code is accessed via ManifoldCF correctly, but it seems its the request made in the plugin to /_vti_bin/Permissions.asmx that is causing the 401 issue.
I have tried setting pre defined credentials in the code above using NetworkCredential
("username", "password", "domain") but no luck.
Example:
string webUrl = SPContext.Current.Web.Url;
NetworkCredential myCredentials = new NetworkCredential("DOMAIN\\user", "mypassword", "DOMAIN");
CredentialCache credCache = new CredentialCache();
credCache.Add(new Uri(webUrl), "Negotiate", myCredentials);
service.Url = webUrl + "/_vti_bin/Permissions.asmx";
service.Credentials = credCache;
With the claims based authentication, the username that i enter into ManfoldCF or in the browser authentication gets changed from the regular \ format to the claims username format (e.g. i:0#.w||/).
Does anybody know why claims based would be causing the 401 issue where the classic authentication does not?
Turns out this was a bug with the ManifoldCF 1.2 release. It has been fixed in newer versions.
Related
I'm trying to use Microsoft.Identity.Client and Microsoft.SharePoint.Client libraries to authenticate to an On-premise SharePoint server and then query it.
I obtain the Azure AD access token from which the SharePoint server is a part of like following:
private readonly string[] m_scopes = { "user.read", "https://sql.azuresynapse-dogfood.net/user_impersonation" };
var publicAppBuilder = PublicClientApplicationBuilder.Create("MyClientId").WithAuthority("https://login.microsoftonline.com/a******com.onmicrosoft.com");
publicAppBuilder.WithRedirectUri("https://login.microsoftonline.com/common/oauth2/nativeclient");
var app = publicAppBuilder.Build();
AuthenticationResult result = null;
result = app.AcquireTokenInteractive(m_scopes).ExecuteAsync().GetAwaiter().GetResult();
if (result != null)
{
m_mediator.AccessToken = result.AccessToken;
}
When I get the access token I put it in the request header as follows:
args.WebRequestExecutor.RequestHeaders["Authorization"] = "Bearer " + m_mediator.AccessToken;
Which is inside the ClientContext.ExecutingWebRequest subscribed method:
clientContext.ExecutingWebRequest += (sender, args) =>
which is triggered by
context.ExecuteQuery();
The remote server returned an error: (401) Unauthorized.
or
The remote server returned an error: (403) Forbidden.
How can I establish the connection? I want to avoid using app-only registration, I want to authenticate using Azure AD MFA (Interactive) method.Please note that I have all the permissions needed and I am an admin on both Azure AD where SharePoint is joined, as well on the SharePoint server itself. I authenticate through the browser just fine.
I've tried multiple things so far:
I tried creating a separate request where I forward the previously acquired accessToken as Authorization: Bearer token
I tried reading the FedAuth from the authentication connection window, so I can forward it in my HTTP request but with no success
I tried creating a "Web browser" using a WebBrowser C# class and reading the cookies that are on a browser level like the following: cookieContainer = webBrowser1.Document.Cookie; but I had no success.
I'm expecting to Authenticate via Azure AD and then connect to SharePoint in order to query it
To resolve the error "The remote server returned an error: (401)
Unauthorized", please try checking the following:
Check whether your URL is correct:
The SharePoint Online URL must always start with HTTPS.
$SiteURL` `=` `"https://crescent.sharepoint.com/sites/marketing"`
Check if you have the right permissions to the site:
Check whether you have sufficient permissions and you are able to open the site in the browser. Make sure to have SharePoint Online Administrator Role.
Check whether the Legacy authentication protocol is enabled:
Make sure to enable Legacy authentication protocol in your tenant, if it is not enabled.
Reference : SharePoint Online: Fix "The remote server returned an error (401) Unauthorized" Error in PowerShell - SharePoint Diary
To resolve the error "The remote server returned an error: (403)
Forbidden.", please try checking the following:
Make sure whether you have provided correct URL and credentials.
Make sure whether you have installed latest version of SharePoint Online Client Component SDK.
Try adding yourself to the site explicitly
Check the lock status of your site and unlock if it is locked.
Please check if any conditional access policies is enabled in your tenant.
If you try to connect to the Tenant Admin site, make sure the Tenant Admin URL like below:
https://YourDomain-admin.sharepoint.com
Reference : SharePoint Online: Fix "The remote server returned an error: (403) Forbidden." Error in PowerShell - SharePoint Diary.
I've found a solution.
I basically iterate through all cookies whenever a browser navigates through a new page and parse all the cookies until I get the fedAuth cookie:
I created a web browser from System.Windows.Forms.WebBrowser
In the WebBrowserNavigatedEventHandler for Navigated I do the following:
if (webBrowser1.Url.AbsoluteUri == "about:blank")
{
return;
}
var cookieData = GetWebBrowserCookie.GetCookieInternal(webBrowser1.Url, false);
if (string.IsNullOrEmpty(cookieData) == false)
{
var dict = ParseCookieData(cookieData);
if (dict.ContainsKey("FedAuth") && !string.IsNullOrEmpty(dict["FedAuth"]))
{
m_mediator.FedAuthCookie = dict["FedAuth"];
if (dict.ContainsKey("rtFa") && !string.IsNullOrEmpty(dict["rtFa"]))
{
m_mediator.RtFaCookie = dict["rtFa"];
}
m_mediator.UpdateConfiguration();
this.Close();
}
}
The ParseCookieData method looks like this:
private IDictionary<string, string> ParseCookieData(string cookieData)
{
var cookieDictionary = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrEmpty(cookieData))
{
return cookieDictionary;
}
var values = cookieData.TrimEnd(';').Split(';');
foreach (var parts in values.Select(c => c.Split(new[] { '=' }, 2)))
{
var cookieName = parts[0].Trim();
var cookieValue = parts.Length == 1 ? string.Empty : parts[1];
cookieDictionary[cookieName] = cookieValue;
}
return cookieDictionary;
}
and GetWebBrowserCookie class looks like this:
[SecurityCritical]
public static string GetCookieInternal(Uri uri, bool throwIfNoCookie)
{
uint pchCookieData = 0;
string url = UriToString(uri);
uint flag = (uint)NativeMethods.InternetFlags.INTERNET_COOKIE_HTTPONLY;
//Gets the size of the string builder
if (NativeMethods.InternetGetCookieEx(url, null, null, ref pchCookieData, flag, IntPtr.Zero))
{
pchCookieData++;
StringBuilder cookieData = new StringBuilder((int)pchCookieData);
//Read the cookie
if (NativeMethods.InternetGetCookieEx(url, null, cookieData, ref pchCookieData, flag, IntPtr.Zero))
{
DemandWebPermission(uri);
return cookieData.ToString();
}
}
int lastErrorCode = Marshal.GetLastWin32Error();
if (throwIfNoCookie || (lastErrorCode != (int)NativeMethods.ErrorFlags.ERROR_NO_MORE_ITEMS))
{
throw new Win32Exception(lastErrorCode);
}
return null;
}
private static void DemandWebPermission(Uri uri)
{
string uriString = UriToString(uri);
if (uri.IsFile)
{
string localPath = uri.LocalPath;
new FileIOPermission(FileIOPermissionAccess.Read, localPath).Demand();
}
else
{
new WebPermission(NetworkAccess.Connect, uriString).Demand();
}
}
private static string UriToString(Uri uri)
{
if (uri == null)
{
return string.Empty;
}
UriComponents components = (uri.IsAbsoluteUri ? UriComponents.AbsoluteUri : UriComponents.SerializationInfoString);
return new StringBuilder(uri.GetComponents(components, UriFormat.SafeUnescaped), 2083).ToString();
}
This way we open up a pop-up C# web browser, authenticate the user through the web using MFA and then close the browser when we acquire an authentication cookie so we can continue working with HTTP requests towards the Sharepoint server.
Source: https://github.com/OceanAirdrop/SharePointOnlineGetFedAuthAndRtfaCookie
I am using the following approach as the basis of this (https://learn.microsoft.com/en-us/azure/active-directory/develop/active-directory-devquickstarts-webapi-dotnet).
I got all this example working after setting up azure. But now we need to port it to an actual existing mobile app and web api app. The mobile app can get the Bearer token, but when we pass it to the web api, we pass this in a CSOM request as follows, but we still get a 401 Unauthroised response.
public static ClientContext GetSharepointBearerClientContext(this JwtTokenDetails tokenDetails)
{
var context = new ClientContext(tokenDetails.SiteUrl);
//context.AuthenticationMode = ClientAuthenticationMode.Anonymous;
context.ExecutingWebRequest += new EventHandler<WebRequestEventArgs>((s, e) =>
{
e.WebRequestExecutor.RequestHeaders["Authorization"] = "Bearer " + tokenDetails.BearerToken;
});
return context;
}
Our web api doesn't use any of the tech as in the example above, as I presume that we should just be able to pass the token through the CSOM request in the header, but this is not working, what else could I look at?
I have assigned the Office 365 Sharepoint Online (Microsoft.Sharepoint) permission and set the following
I have also done the same for the app registration, which we don't really use! Still not sure how the app registration comes into it)...
So this was possible, it was just microsoft telling us to put in an incorrect value. All the documentation says put the APP ID URI in the Resource. But in our case it needed to be the sharepoint url.
So we have the tenant name which on azure id the domain name e.g. srmukdev.onmicrosoft.com
Tenant: srmukdev.onmicrosoft.com
Application Id: This is the guid for the app registered in azure active directory.
RedirectUri: This can be any url(URI), its not actually used as a url for a mobile app as far as I can see.
ResourceUrl: srmukdev.sharepoint.com
The code I am using to get a token is as follows for a WPF example. The aadInstance is https://login.microsoftonline.com/{0}
private static string authority = String.Format(CultureInfo.InvariantCulture, aadInstance, tenant);
public async void CheckForCachedToken(PromptBehavior propmptBehavior)
{
//
// As the application starts, try to get an access token without prompting the user. If one exists, populate the To Do list. If not, continue.
//
AuthenticationResult result = null;
try
{
result = await authContext.AcquireTokenAsync(resourceUrl, applicationId, redirectUri, new PlatformParameters(propmptBehavior));
TokenTextBox.Text = result.AccessToken;
// A valid token is in the cache - get the To Do list.
GetTokenButton.Content = "Clear Cache";
}
catch (AdalException ex)
{
if (ex.ErrorCode == "user_interaction_required")
{
// There are no tokens in the cache. Proceed without calling the To Do list service.
}
else
{
// An unexpected error occurred.
string message = ex.Message;
if (ex.InnerException != null)
{
message += "Inner Exception : " + ex.InnerException.Message;
}
MessageBox.Show(message);
}
return;
}
}
I am trying to set up a C# console app that can send notifications/reminders to users via Skype for Business online from a generic AD account. I was excited to see the other day that according to this page, UCWA is now supported in Skype for Business online: https://msdn.microsoft.com/en-us/library/office/mt650889.aspx.
I've been trying to follow this tutorial to get this set up: https://msdn.microsoft.com/en-us/library/office/mt590891(v=office.16).aspx. So far I haven't really had much luck... I have my application set up in Azure AD but I get stuck at the "Requesting an access token using implicit grant flow" step of that article (not 100% certain I'm taking the correct actions before that either)... so far I have this:
string clientId = "xxxxxxxx"
string resourceUri = "https://webdir.online.lync.com";
string authorityUri = "https://login.windows.net/common/oauth2/authorize";
AuthenticationContext authContext = new AuthenticationContext(authorityUri);
UserCredential cred = new UserCredential("username", "password");
string token = authContext.AcquireToken(resourceUri, clientId, cred).AccessToken;
var poolReq = CreateRequest("https://webdir.online.lync.com/autodiscover/autodiscoverservice.svc/root", "GET",token);
var poolResp = GetResponse(poolReq);
dynamic tmp = JsonConvert.DeserializeObject(poolResp);
string resourcePool = tmp._links.user.href;
Console.WriteLine(resourcePool);
var accessTokenReq = CreateRequest("https://login.windows.net/common/oauth2/authorize"
+ "?response_type=id_token"
+ "&client_id=" + clientId
+ "&redirect_uri=https://login.live.com/oauth20_desktop.srf"
+ "&state=" + Guid.NewGuid().ToString()
+ "&resource=" + new Uri(resourcePool).Host.ToString()
, "GET",token);
var accessTokenResp = GetResponse(accessTokenReq);
my GetResponse and CreateRequest methods:
public static string GetResponse(HttpWebRequest request)
{
string response = string.Empty;
using (HttpWebResponse httpResponse = request.GetResponse() as System.Net.HttpWebResponse)
{
//Get StreamReader that holds the response stream
using (StreamReader reader = new System.IO.StreamReader(httpResponse.GetResponseStream()))
{
response = reader.ReadToEnd();
}
}
return response;
}
public static HttpWebRequest CreateRequest(string uri, string method, string accessToken)
{
HttpWebRequest request = System.Net.WebRequest.Create(uri) as System.Net.HttpWebRequest;
request.KeepAlive = true;
request.Method = method;
request.ContentLength = 0;
request.ContentType = "application/json";
request.Headers.Add("Authorization", String.Format("Bearer {0}", accessToken));
return request;
}
accessTokenResp is an office online logon page, not the access token I need to move forward... so I'm stuck. I've tried quite a few variations of the above code.
I've been scouring the net for more examples but can't really find any, especially since UCWA support for Office 365 is so new. Does anyone have an example of how to do what I am trying to do or can point me to one? Everything I've found so far hasn't really even been close to what I'm trying. I can't use the Skype for Business client SDK unfortunately either as it doesn't meet all of my requirements.
I came to a working solution using ADAL (v3), with the help of steps outlined at
Authentication using Azure AD
Here the steps, which involve requesting multiple authentication tokens to AAD using ADAL
Register your application, as Native Application, in Azure AD.
Perform autodiscovery to find user's UCWA root resource URI.
This can be done by performing a GET request on
GET https://webdir.online.lync.com/Autodiscover/AutodiscoverService.svc/root?originalDomain=yourdomain.onmicrosoft.com
Request an access token for the UCWA root resource returned in the autodiscovery response, using ADAL
For instance, your root resource will be at
https://webdir0e.online.lync.com/Autodiscover/AutodiscoverService.svc/root/oauth/user?originalDomain=yourdomain.onmicrosoft.com
you'll have to obtain a token from AAD for resource https://webdir0e.online.lync.com/
Perform a GET on the root resource with the bearer token obtained from ADAL
GET https://webdir0e.online.lync.com/Autodiscover/AutodiscoverService.svc/root/oauth/user?originalDomain=yourdomain.onmicrosoft.com
This will return, within the user resource, the URI for applications resource, where to create your UCWA application. This in my case is:
https://webpoolam30e08.infra.lync.com/ucwa/oauth/v1/applications
Residing then in another domain, thus different audience / resource, not included in the auth token previously obatained
Acquire a new token from AAD for the host resource where the home pool and applications resource are (https://webpoolam30e08.infra.lync.com in my case)
Create a new UCWA application by doing a POST on the applications URI, using the token obtained from ADAL
Voilá, your UCWA application is created. What I notice at the moment, is that just few resources are available, excluding me / presence. So users' presence can be retrieved, but self presence status can't be changed.
I've been able however to retrieve my personal note, and the following resources are available to me:
people
communication
meetings
Show me some code:
Function to perform the flow obtaining and switching auth tokens
public static async Task<UcwaApp> Create365UcwaApp(UcwaAppSettings appSettings, Func<string, Task<OAuthToken>> acquireTokenFunc)
{
var result = new UcwaApp();
result.Settings = appSettings;
var rootResource = await result.Discover365RootResourceAsync(appSettings.DomainName);
var userUri = new Uri(rootResource.Resource.GetLinkUri("user"), UriKind.Absolute);
//Acquire a token for the domain where user resource is
var token = await acquireTokenFunc(userUri.GetComponents(UriComponents.SchemeAndServer, UriFormat.SafeUnescaped));
//Set Authorization Header with new token
result.AuthToken = token;
var usersResult = await result.GetUserResource(userUri.ToString());
//
result.ApplicationsUrl = usersResult.Resource.GetLinkUri("applications");
var appsHostUri = new Uri(result.ApplicationsUrl, UriKind.Absolute).GetComponents(UriComponents.SchemeAndServer, UriFormat.SafeUnescaped);
//Acquire a token for the domain where applications resource is
token = await acquireTokenFunc(appsHostUri);
//Set Authorization Header with new token
result.AuthToken = token;
//
var appResult = await result.CreateApplicationAsync(result.ApplicationsUrl, appSettings.ApplicationId, appSettings.UserAgent, appSettings.Culture);
return result;
}
Usage code ato retrieve OAuth tokens using ADAL
var ucSettings = new UcwaAppSettings
{
UserAgent = "Test Console",
Culture = "en-us",
DomainName = "yourdomain.onmicrosoft.com",
ApplicationId = "your app client id"
};
var acquireTokenFunc = new Func<string, Task<OAuthToken>>(async (resourceUri) =>
{
var authContext = new AuthenticationContext("https://login.windows.net/" + ucSettings.DomainName);
var ar = await authContext.AcquireTokenAsync(resourceUri,
ucSettings.ApplicationId,
new UserCredential("myusername", "mypassword"));
return new OAuthToken(ar.AccessTokenType, ar.AccessToken, ar.ExpiresOn.Ticks);
});
var app = await UcwaApp.Create365UcwaApp(ucSettings, acquireTokenFunc);
It should be of course possible to avoid hard-coding username and password using ADAL, but this was easier for PoC and especially in case of Console Application as you asked
I've just blogged about this using a start-to-finish example, hopefully it will help you. I only go as far as signing in, but you can use it with another post I've done on sending IMs using Skype Web SDK here (see day 13 and 14) and combine the two, it should work fine.
-tom
Similar to Massimo's solution, I've created a Skype for Business Online C# based console app that demonstrates how to sign and use UCWA to create/list/delete meetings and change user presence. I haven't gotten around to extending it to send IM's, but you're certainly welcome to clone my repository and extend it to your needs. Just drop in your Azure AD tenant name and native app ID into the code.
I think they just turned this on today - I was doing something unrelated with the Skype Web SDK samples and had to create a new Azure AD app, and noticed that there are two new preview features for receiving conversation updates and changing user information.
Now everything in the Github samples works for Skype For Business Online.
I'm creating an MVC app that requires me to nab the group users from a Sharepoint site. I'm getting a 401 error when I publish and run it via IIS (localhost). When I run it through the debugger though, everything works as intended.
I'm assuming it's a permissions error based on the context. Is there a way to run the app normally using my current NTLM credentials programmatically (instead of what seems like the IIS account/user)?
Code
(I'm on mobile right now, so I'm sorry if anything looks wonky...)
// Constructor
clientContext = new ClientContext("http://sharepointsite");
clientContext.Credentials = CredentialCache.DefaultNetworkCredentials;
//clientContext.Credentials = new NetworkCredentials("username", "password", "domain"); <- this works just fine if I hard code my current credentials
// Calling method
UserCollection users = GetUsersCollection();
clientContext.Load(users);
try {
clientContext.ExecuteQuery(); // 401 error here
return true;
}
catch {
return false;
}
You can specify the credentials you connect to SharePoint with CSOM it would look something like this:
clientContext.Credentials = new NetworkCredential("Username", "Password", "Domain");
You can do MVC impersonation like this:
string currentUser = System.Security.Principal.WindowsIdentity.GetCurrent().Name;
System.Security.Principal.WindowsImpersonationContext impersonationContext;
impersonationContext =
((System.Security.Principal.WindowsIdentity)User.Identity).Impersonate();
currentUser += "Impersonate:" + System.Security.Principal.WindowsIdentity.GetCurrent().Name;
impersonationContext.Undo();
ViewBag.U = currentUser;
return View();
I found that example in this thread:
http://forums.asp.net/t/1961337.aspx?Asp+Net+MVC+5+how+to+impersonate+user+on+IIS
I'm trying to read a value from a list in a remote SharePoint site (different SP Web App). The web apps are set up with Claims Auth, and the client web app SP Managed account is configured with an SPN. I believe Kerberos and claims are set up correctly, but I am unable to reach the remote server, and the request causes an exception: "The remote server returned an error: (401) Unauthorized."
The exception occurs in the line ctx.ExecuteQuery(); but it does not catch the exception in the if (scope.HasException) instead, the exception is caught by the calling code (outside of the using{} block).
When I look at the traffic at the remote server using Wireshark, it doesn't look like the request is even getting to the server; it's almost as if the 401 occurs before the Kerberos ticket is exchanged for the claim.
Here's my code:
using (ClientContext ctx = new ClientContext(contextUrl))
{
CredentialCache cc = new CredentialCache();
cc.Add(new Uri(contextUrl), "Kerberos", CredentialCache.DefaultNetworkCredentials);
ctx.Credentials = cc;
ctx.AuthenticationMode = ClientAuthenticationMode.Default;
ExceptionHandlingScope scope = new ExceptionHandlingScope(ctx);
Web ctxWeb = ctx.Web;
List ctxList;
Microsoft.SharePoint.Client.ListItemCollection listItems;
using (scope.StartScope())
{
using (scope.StartTry())
{
ctxList = ctxWeb.Lists.GetByTitle("Reusable Content");
CamlQuery qry = new CamlQuery();
qry.ViewXml = string.Format(ViewQueryByField, "Title", "Text", SharedContentTitle);
listItems = ctxList.GetItems(qry);
ctx.Load(listItems, items => items.Include(
item => item["Title"],
item => item["ReusableHtml"],
item => item["ReusableText"]));
}
using (scope.StartCatch()) { }
using (scope.StartFinally()) { }
}
ctx.ExecuteQuery();
if (scope.HasException)
{
result = string.Format("Error retrieving content<!-- Error Message: {0} | {1} -->", scope.ErrorMessage, contextUrl);
}
if (listItems.Count == 1)
{
Microsoft.SharePoint.Client.ListItem contentItem = listItems[0];
if (SelectedType == SharedContentType.Html)
{
result = contentItem["ReusableHtml"].ToString();
}
else if (SelectedType == SharedContentType.Text)
{
result = contentItem["ReusableText"].ToString();
}
}
}
I realize the part with the CredentialCache shouldn't be necessary in claims, but every single example I can find is either running in a console app, or in a client side application of some kind; this code is running in the codebehind of a regular ASP.NET UserControl.
Edit: I should probably mention, the code above doesn't even work when the remote URL is the root site collection on the same web app as the calling code (which is in a site collection under /sites/)--in other words, even when the hostname is the same as the calling code.
Any suggestions of what to try next are greatly appreciated!
Mike
Is there a reason why you are not using the standard OM?
You already said this is running in a web part, which means it is in the context of application pool account. Unless you elevate permissions by switching users, it won't authenticate correctly. Maybe try that. But I would not use the client OM when you do have access to the API already.