Copy Sharepoint online csv to datalake using Azure Data Factory - azure

I am trying to copy CSV file from Sharepoint online to Data lake using Data Factory. I followed the below documentation.
https://learn.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory#copy-file-from-sharepoint-online
My pipeline looks below
I am getting errors in the copy data activity. My connection to Sharepoint is successful, and even I can download the CSV file using the base & relative URL.
Below is my error
Error code 2200
ErrorCode=HttpRequestFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Http request failed with client error, status code 403 Forbidden, please check your activity settings. If you configured a baseUrl that includes path, please make sure it ends with '/'.
Request URL: https://xxxx.sharepoint.com/sites/39421875/_api/web/GetFileByServerRelativeUrl('/sites/39421875/Shared%20Documents/Test.csv')/$value.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=System,'
My base URL: https://xxx.sharepoint.com/
Relative URL: sites/39421875/_api/web/GetFileByServerRelativeUrl('/sites/39421875/Shared Documents/Test.csv')/$value
Not sure why the above error is coming? I added "/" in the base URL.
I tried multiple times, and the same error is happening. Even I tried to give a complete URL in the base URL but with no success. I followed the documentation for the Additional Header.
Can anyone advise?

The above error is mainly because of Authorization Failure. Please make sure to check whether the below permissions are granted or not.
Go to Azure Active directory -> App registration and open app -> API permission -> Add this permissions.
And also please follow the below reference: It has detailed information about app permission on specific site collection and Authorization of SharePoint.
Reference:
DataFactory: Extract data from SharePoint Online by javiBI
Access SharePoint Online using Postman by Shantha Kumar

Related

Getting error while copying the file from sharepoint to blob storage in ADF

I am getting this error ,
"ErrorCode=UserErrorFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Upload file failed at path sapoutput.,Source=mscorlib,''Type=System.Net.WebException,Message=The remote server returned an error: (403) Forbidden.,Source=System,"
While copying a file from sharepoint site to blob storage
My web activity to get access token is running fine but there is error in copy activity.
If you don’t have required permission and you will get this error.
For granting SharePoint access permission you need to register application
Open SharePoint online site -> Search application id and fill the empty fields –> add domain name and register URL
Please follow this Microsoft document for more information.

External status checks in gitlab

I want to add external status checks for my project on gitlab. For that I followed this doc but for my project the option to add status checks is not showing in the settings -> General -> merge requests section. Also When I am trying to add it through api as shown in this doc, I am getting "Unauthorized" as response. I have maintainer access for the project and I have given complete api access to my token. I even tried with project access token but the response was same.
Any help from the community would be appreciated.
The external status checks is a Ultimate tier feature. If the setting is not displayed, this is most likely because you do not have an Ultimate tier subscription entitlement on this project.

Microsoft Azure - failed to create table: StorageError: Server failed to authenticate the request

I have a node.js with AngularJs Web application. And in production environment it is failing. In App logs it shows as
/home/site/wwwroot/node_modules/connect-azuretables/lib/connect-azuretables.js:86
throw ('failed to create table: ' + error);
"failed to create table: Storage Error: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature."
I am new to Microsoft Azure.
Please check few points:
The issue maybe due to time zone difference. Please set the proper time and timezone of your pc and execute.
Also please check if accesss key is correct .Note that the key is case sensitive and make sure you have not included white spaces.Also check if SAS has expired.SAS url may not require "?" in url .Please check if "?" is present in sas url creation.
If not generate the keys or connection string again and try to use them.
Please make sure you have Storage Blob Data Contributor role in azure. Navigate to Storage account -> Access Control (IAM) -> Add role assignment to do the same
Check if any firewall is blocking the request .
Other wise see if it is the issue with azure node js sdk version.

Microsoft Graph API file permissions are not visible/delete

I am trying to create Read permissions for a single file in the sharepoint online site document library without any issues.
I am capturing the permissionID while creating the permission on the specific file. immediately I am trying to delete the same permission and I am getting the below exception:
HTTP Response Code:404; Response Message: The resource could not be found
Below URL I am using to delete a permission:
https://graph.microsoft.com/v1.0/drives/{driveId}/root:/{pathToFile}:/permissions/{permissionID}.
any help can be appreciated.

Blueprism Code Producing 403 Forbidden Error

I am trying to use Blueprism to download pdf files from Sharepoint.
I am using the below code. Url and Path are both variables which get passed through. This is producing a 403 forbidden error. I believe that Blueprism is required to pass credentials to Sharepoint before it will be allowed to download the file. Is this possible?
Using wc As New System.Net.WebClient()
wc.DownloadFile(Url, Path)
End Using
I don't think you have a password related issue, likely the SharePoint is recognising your account access via AD group or SSO of some kind. 403 means the server has understood your request but is denying to fulfil it because of an access reason on the profile you have. Make sure you can manually download the file on the profile you are emulating for a start then also check the profile the bot is operating under (not necessarily yours in some cases ) has access to said file.
Literally 403 is a request that relates to an access issue so somewhere something doesn't have the correct access lined up
First of all, is this a Sharepoint Online or OnPremises version? As you understand, you are connecting to web resource via an API and as such you have to get authenticated and authorized to access those resources.
If it's SPO, you can use the API component for SharePoint integration from DX and configure.
In case of SP On prem, you have to customize a lot to achieve your results.

Resources