The remote server returned an error: (401) Unauthorized - CSOM - OAuth - after specific time - sharepoint

Question

I am trying to get permissions assigned to each document from the document library using csom in console application. Console app is running fine for almost one hour and post that I get error : "The remote server returned an error: (401) Unauthorized."
I registered an app in SharePoint and I'm using Client ID & Client Secret Key to generate Client Context. Here is the code to get the client context
public static ClientContext getClientContext()
{
ClientContext ctx = null;
try
{
ctx = new AuthenticationManager().GetAppOnlyAuthenticatedContext(SiteUrl, ClientId, ClientSecret);
ctx.RequestTimeout = 6000000;
ctx.ExecutingWebRequest += delegate (object sender, WebRequestEventArgs e)
{
e.WebRequestExecutor.WebRequest.UserAgent = "NONISV|Contoso|GovernanceCheck/1.0";
};
}
catch (Exception ex)
{
WriteLog("Method-getClientContext, Error - " + ex.Message);
}
return ctx;
}
Here is the code which is getting all the assignments assigned to document in document library.
static StringBuilder excelData = new StringBuilder();
static void ProcessWebnLists()
{
try
{
string[] lists = { "ABC", "XYZ", "DEF", "GHI"};
foreach (string strList in lists)
{
using (var ctx = getClientContext())
{
if (ctx != null)
{
Web web = ctx.Site.RootWeb;
ListCollection listCollection = web.Lists;
ctx.Load(web);
ctx.Load(listCollection);
ctx.ExecuteQuery();
List list = web.Lists.GetByTitle(strList);
ctx.Load(list);
ctx.ExecuteQuery();
Console.WriteLine("List Name:{0}", list.Title);
var query = new CamlQuery { ViewXml = "<View/>" };
var items = list.GetItems(query);
ctx.Load(items);
ctx.ExecuteQuery();
foreach (var item in items)
{
var itemAssignments = item.RoleAssignments;
ctx.Load(item, i => i.DisplayName);
ctx.Load(itemAssignments);
ctx.ExecuteQuery();
Console.WriteLine(item.DisplayName);
string groupNames = "";
foreach (var assignment in itemAssignments)
{
try
{
ctx.Load(assignment.Member);
ctx.ExecuteQuery();
groupNames += "[" + assignment.Member.Title.Replace(",", " ") + "] ";
Console.WriteLine($"--> {assignment.Member.Title}");
}
catch (Exception e)
{
WriteLog("Method-ProcessWebnLists, List-" + list.Title + ", Document Title - " + item.DisplayName + ", Error : " + e.Message);
WriteLog("Method-ProcessWebnLists, StackTrace : " + e.StackTrace);
}
}
excelData.AppendFormat("{0},{1},ITEM,{2}", list.Title, (item.DisplayName).Replace(",", " "), groupNames);
excelData.AppendLine();
}
}
excelData.AppendLine();
}
}
}
catch (Exception ex)
{
WriteLog("Method-ProcessWebnLists, Error : " + ex.Message);
WriteLog("Method-ProcessWebnLists, StackTrace : " + ex.StackTrace.ToString());
}
}
Can anyone suggest what else I am missing in this or what kind of changes do I need to make to avoid this error?
The process is taking too much time so I want my application to keep running for couple of hours.
Thanks in advance!

Answer 1

Is there a reason why getClientContext() is inside the foreach loop?
I encountered this error before.
I added this to solve mine.
public static void ExecuteQueryWithIncrementalRetry(this ClientContext context, int retryCount, int delay)
{
int retryAttempts = 0;
int backoffInterval = delay;
if (retryCount <= 0)
throw new ArgumentException("Provide a retry count greater than zero.");
if (delay <= 0)
throw new ArgumentException("Provide a delay greater than zero.");
while (retryAttempts < retryCount)
{
try
{
context.ExecuteQuery();
return;
}
catch (WebException wex)
{
var response = wex.Response as HttpWebResponse;
if (response != null && response.StatusCode == (HttpStatusCode)429)
{
Console.WriteLine(string.Format("CSOM request exceeded usage limits. Sleeping for {0} seconds before retrying.", backoffInterval));
//Add delay.
System.Threading.Thread.Sleep(backoffInterval);
//Add to retry count and increase delay.
retryAttempts++;
backoffInterval = backoffInterval * 2;
}
else
{
throw;
}
}
}
throw new MaximumRetryAttemptedException(string.Format("Maximum retry attempts {0}, have been attempted.", retryCount));
}
Basically, this code backs off for a certain time/interval before executing another query, to prevent throttling.
So instead of using ctx.ExecuteQuery(), use ctx.ExecuteQueryWithIncrementalRetry(5, 1000);
Further explanation here
https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online