I have an azure trigger function and I want the service to take the files from folder which is in on-premise then process them. The folder is that we only can access via VPN. The service has a VNET configuration for this but still cannot access the folder.
What should I do?
Please let me know why I am getting this error or do I need to change something in my process? Thanks in advance.
tried to ping the folder address, got this exception
System.Net.NetworkInformation.PingException: An exception occurred during a Ping request. ---> System.ComponentModel.Win32Exception (5): Access is denied. at System.Net.NetworkInformation.Ping.InitialiseIcmpHandle() at System.Net.NetworkInformation.Ping.DoSendPingCore(IPAddress address, Byte[] buffer, Int32 timeout, PingOptions options, Boolean isAsync) at System.Net.NetworkInformation.Ping.Send(IPAddress address, Int32 timeout, Byte[] buffer, PingOptions options) --- End of inner exception stack trace --- at System.Net.NetworkInformation.Ping.Send(IPAddress address, Int32 timeout, Byte[] buffer, PingOptions options) at System.Net.NetworkInformation.Ping.Send(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options) at ....
tried to access to the path, got the exception below
System.UnauthorizedAccessException: Access to the path '\\\xxxxxx\test' is denied. at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound) at System.IO.Enumeration.FileSystemEnumerator`1.Init() at ....
You cannot initiate the connection from Azure Functions to on-prem, network connections can only be initiated by clients connecting to the Private endpoint, Service providers (in this case Azure Functions) do not have any routing configuration to initiate connections into service consumers. Connections can only be establish in a single direction.
If the onprem folder is a OneDrive folder you have the option of using the Microsoft Graph API to send to a webhook which could be your Function App. Details are here.
https://learn.microsoft.com/en-us/graph/webhooks
This is only one option with the limited information I have on your scenario.
Related
We use Azure DevOps for building and deploying our applications and services in Azure. Right now we are also switching our on premise deployments to Azure DevOps as well.
Unfortunately we have one issue with our servers which are on the one hand protected by a firewall (outgoing internet traffic is mostly blocked) and on the other hand are part of Azure DevOps deployment groups. As soon as we create a new release and start the deployment to a stage the artifact download process fails since the server is not permitted to download the artifacts from the blob storage where the artifacts are stored.
Warning, https://elgvsblobprodsu6weus10.blob.core.windows.net/dbcXXXXXXX/XXXX
Try 1/5, retryable exception caught. Retrying in 00:00:01. Details:
2019-10-21T19:59:02.7071549Z No LastRequestResponse on exception HttpRequestException: An error occurred while sending the request. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: Error 12029 calling WINHTTP_CALLBACK_STATUS_REQUEST_ERROR, 'A connection with the server could not be established'.
2019-10-21T19:59:02.7071940Z at System.Threading.Tasks.RendezvousAwaitable`1.GetResult()
2019-10-21T19:59:02.7072055Z at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
2019-10-21T19:59:02.7072112Z --- End of inner exception stack trace ---
2019-10-21T19:59:02.7074791Z at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
2019-10-21T19:59:02.7074976Z at Microsoft.VisualStudio.Services.Common.TaskCancellationExtensions.EnforceCancellation[TResult](Task`1 task, CancellationToken cancellationToken, Func`1 makeMessage, String file, String member, Int32 line)
2019-10-21T19:59:02.7075133Z at Microsoft.VisualStudio.Services.BlobStore.WebApi.DedupStoreHttpClient.<>c__DisplayClass56_0.<<GetRedirectResponseAsync>b__0>d.MoveNext()
2019-10-21T19:59:02.7075213Z --- End of stack trace from previous location where exception was thrown ---
2019-10-21T19:59:02.7075312Z at Microsoft.VisualStudio.Services.Content.Common.AsyncHttpRetryHelper`1.InvokeAsync(CancellationToken cancellationToken)
So is there any list of endpoints which are used for storing Azure Pipeline artifacts that can be added to our firewall policy, do we have to exclude *.blob.core.windows.net in general or is there any different approach?
Unfortunately the Microsoft docs do only provide a list of Urls which are required for establishing a connection with Azure DevOps in general, but not with the artifacts store.
Is there any list of endpoints which are used for storing Azure
Pipeline artifacts that can be added to our firewall policy.
Of course yes. We store these data in the corresponding region's Azure Data center.
Download the Json file in this link, the file list the IP Address range of different regions. But, here is one important thing you need pay attention to. For security, we update these IP address range regularly. (Also, publish it to users regularly)
If you choose this approach to achieve you want, you need to download the JSON file regularly, then make necessary updates in your access rules:
[!IMPORTANT] IP address ranges for Azure services can change, and
updates are published weekly. Download the JSON file regularly, and
make necessary updates in your access rules. If your scenario involves
configuring network security group rules in an Azure virtual network
to access Azure Container Registry, use the AzureContainerRegistry
service tag instead.
See this doc: Allow access by IP address range.
But, if you think this method is inconvenient for you and your team. Another unique method is the one you are using now: allow access to all Azure blob storage accounts using the wildcard *.blob.core.windows.net.
I've been trying to get to through this issue for some time now, and unfortunately, I'm hitting a wall.
We have Azure AD configured with Azure Domain Services. VM's are joined to this domain. We also have Azure SQL Database that we're attempting to connect to from the IIS application running on the VM.
I've followed the official MS docs on this (https://learn.microsoft.com/en-us/azure/sql-database/sql-database-aad-authentication-configure) that helped verify Azure AD identities can connect to the Azure SQL DB (I can connect via SSMS to Azure SQL using an Azure AD Admin user. I can then use that ID to generate other contained users, which I have done).
The problem exists when I attempt to authenticate with the database with Active Directory Integration in my connection string. I get two errors, which i'm going to include below.
Now, I have not been able to get past the last one (stating that federation is needed), and in fact, have read numerous things (albeit nothing directly from MS) that says this is simply not supported and that I need to create a federation to get this working, which just seems completely unreasonable given that this is an Azure-only environment.
Connection string included below as well.
I've tried various iterations of that connection string to no avail. If anyone can provide some insight into this, I'd appreciate it greatly!
One last thing, the IIS application pool is running as a user that is both part of the AAD Domain group on Azure AD, and a contained user created on the DB.
Thanks!
I've tried various iterations of the connection string that include removing much of the additional params on the connection string, and the quotes.
I've tried different users on the application pool
End goal is to remove all plain-text PW's from appearing within the connection string to Azure SQL.
First error:
Unable to load adalsql.dll (Authentication=ActiveDirectoryPassword). Error code: 0x2.
Second Error:
[AdalException: Integrated Windows authentication supported only in federation flow.]
ADALNativeWrapper.ADALGetAccessToken(String username, IntPtr password, String stsURL, String servicePrincipalName, ValueType correlationId, String clientId, Boolean* fWindowsIntegrated, Int64& fileTime) +829
System.Data.SqlClient.<>c__DisplayClass2_0.<AcquireTokenAsync>b__0() +132
System.Threading.Tasks.Task`1.InnerInvoke() +121
System.Threading.Tasks.Task.Execute() +47
Connection String:
name="LocalSqlServer" connectionString="Server=tcp:XXXXX;Initial Catalog=XXXXX;Persist Security Info=False;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Authentication='Active Directory Integrated';" />
I am trying to use terraform to create a Service Fabric Cluster via combination of VMSS, VNETs, NSGs, etc. Unfortunately there isn't an official terraform provider for service fabric, so I am using an ARM template via azurerm_template_deployment.
All resources get created successfully via terraform apply and I am able to RDP into one of the VMs in the scale set. Unfortunately, even after 30-60 minutes, the Azure Portal for Service Fabric resource Status reads Waiting for nodes with no nodes showing up in the table.
When I RDP into the VM, I see several warning/error events in Event Viewer. All events come from ServiceFabricNodeBootstrapAgent:
1.
Info: Bootstrapping local node
2.
Info: PUT https://westus.servicefabric.azure.com/runtime/clusters/GUID_redacted/nodes/node_name_redacted
{"nodeTypeRef":"fronend","ipAddress":"192.168.0.8","faultDomain":"fd:/4","upgradeDomain":"4"}
3. Info
Using client certificate: [Version]
V3
[Subject]
CN=AzureServiceFabric-AnonymousClient
Simple Name: AzureServiceFabric-AnonymousClient
DNS Name: AzureServiceFabric-AnonymousClient
[Issuer]
CN=AzureServiceFabric-AnonymousClient
Simple Name: AzureServiceFabric-AnonymousClient
DNS Name: AzureServiceFabric-AnonymousClient
... etc ...
4.
ERROR: System.Net.WebException: The remote server returned an error: (401) Unauthorized.
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.Azure.ServiceFabric.Extension.Core.RestClient.Invoke(Uri requestUri, String method, String requestBody, X509Certificate2 clientCertificate)
5.
System.Exception: System.Net.WebException: The remote server returned an error: (401) Unauthorized.
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.Azure.ServiceFabric.Extension.Core.RestClient.Invoke(Uri requestUri, String method, String requestBody, X509Certificate2 clientCertificate)
at Microsoft.Azure.ServiceFabric.Extension.Core.RestClient.Invoke(Uri requestUri, String method, String requestBody, List`1 clientCertificates)
at Microsoft.Azure.ServiceFabric.Extension.Core.RestClient.Invoke(Uri requestUri, String method, String requestBody, List`1 clientCertificates)
at Microsoft.Azure.ServiceFabric.Extension.Core.WrpTopologyService.UpdateNodeInfo(String machineName, NodeDescription nodeDescription)
at System.Threading.Tasks.Task.Execute()
6.
ERROR: Microsoft.Azure.ServiceFabric.Extension.Core.AgentException: Unable to send node information
at Microsoft.Azure.ServiceFabric.Extension.Core.NodeBootstrapAgent.<RunOnce>d__11.MoveNext()
It seems that ServiceFabricNodeBootstrapAgent, a Windows service gets installed, but that's it. None of the service fabric runtime services ever get installed and I am not sure even why that API is used, or why it is returning 401. Anyone out there seen this problem?
Creating a cluster from the marketplace as well as via exported ARM template with powershell works just fine.
My test site has after a deploy started to get 403 forbidden back when trying to access files from the azure blob storage.
This is only a problem on our test environment, the new release works just fine in production. Both production and test is hosted in azure, and both use their own azure blob storage.
I have tried regenerating the access keys for the blob storage, without any luck.
I can use the access keys locally and connect to the test blob storage and access the files just fine.
If i try to change the test environment to use the production blob storage, i still get the 403 forbidden error.
Microsoft.WindowsAzure.Storage.StorageException: The remote server
returned an error: (403) Forbidden. ---> System.Net.WebException: The
remote server returned an error: (403) Forbidden. at
System.Net.HttpWebRequest.GetResponse() at
Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand1 cmd, IRetryPolicy policy, OperationContext operationContext) in c:\Program Files (x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line 677 --- End of inner exception stack trace --- at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSync[T](RESTCommand1
cmd, IRetryPolicy policy, OperationContext operationContext) in
c:\Program Files
(x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Core\Executor\Executor.cs:line
604 at
Microsoft.WindowsAzure.Storage.Blob.CloudBlobClient.GetBlobReferenceFromServer(StorageUri
blobUri, AccessCondition accessCondition, BlobRequestOptions options,
OperationContext operationContext) in c:\Program Files
(x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlobClient.cs:line
563 at
Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.GetBlobReferenceFromServer(String
blobName, AccessCondition accessCondition, BlobRequestOptions options,
OperationContext operationContext) in c:\Program Files
(x86)\Jenkins\workspace\release_dotnet_master\Lib\ClassLibraryCommon\Blob\CloudBlobContainer.cs:line
818 at ASP.XYZ in d:\XYZ.cshtml:line 27 Request Information
RequestID:fc137321-0001-00ce-02d2-de5736000000 RequestDate:Tue, 06 Jun
2017 14:41:21 GMT StatusMessage:Server failed to authenticate the
request. Make sure the value of Authorization header is formed
correctly including the signature.
It must be an issue on the azure app service that runs the test site. I just cannot see what it could be.
See gist here for some very simplified code https://gist.github.com/Skaanning/5cddf95a0d1ff210482d99a683e0be9c .
Edit
I have multiple other blob stores, but none of them work on the test site. They work fine on other environments though.
I tried scaling it up and down, without any luck. But then i tried changing from 32 bit to 64 bit and now it works! I will keep the question open, in case someone can explain what on earth happened here
var img = container.GetBlobReference("someimage.png"); // this works just fine
var img2 = container.GetBlobReferenceFromServer("someimage.png"); // this throws a 403
The first line will work fine. The reason is that it will not send request to server when we execute GetBlobReference method. When executing GetBlobReferenceFromServer method, it will send a request to server to get the blob content. It will throw 404 not found exception if the blob can't be found in blob server.
My test site has after a deploy started to get 403 forbidden back when trying to access files from the azure blob storage.
The 403 forbidden exception often caused by a wrong access key is used. Please print out the connection string which was used in your test environment.
Trace.Write(CloudConfigurationManager.GetSetting("blob.storage"));
A Azure Storage connection string uses following format. Please make sure you have provided a right account name and key pair for the connection string. If you set account name1 and account key for account name2, it will cause the exception.
DefaultEndpointsProtocol=https;AccountName=[accountName];AccountKey=[accountKey];EndpointSuffix=core.windows.net
Another thing would cause this issue is that the request to your storage server is rejected by your test server. Please check whether you have configured Dynamic IP Security in your web.config.
Edit 2017/6/7 4:26 PM
I suggest you take my upper suggestion. Please print out the connection string at runtime. If you set the connections string in both app setting in Azure portal and web.config. settings in Azure portal will override the settings in web.config.
Please also print out the current date time at runtime to check whether system time of your app instance is right. If the date or time of your system has been changed, 404 Forbidden also will happen.
The storage services ensure that a request is no older than 15 minutes by the time it reaches the service. This guards against certain security attacks, including replay attacks. When this check fails, the server returns response code 403 (Forbidden).
Reference: Authentication for the Azure Storage Services
Edit 2017/6/7 5:55 PM
Have you tried removing all the original files at your test server when deploying your release?
Review your server datetime. That worked for me.
I had this same error the only thing that fixed the error was by switching the platform to x64 from any cpu. What voodo is this?
I then changed back to "Any CPU" deleted all local file manually this time instead of using rebuild or clean from Visual Studio. Rebuilt from clean, I no longer get the 403.
My servertime was wrong, somehow my application set the time back 1 hour 20 mins and that would result in a 403, I'm speculating that the authorization from azure blobstorage uses server time for its authentication.
At least setting the time back to correct worked for me.
I had faced similar issue: Microsoft.WindowsAzure.Storage.StorageException: The remote server returned an error: (403) Forbidden. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
Fix: I have configured the access policy readonly on the azure portal.
I was using the readonly access policy to generate the SAS url in my c# code, as the was no readonly key configured the error was happening.
follow below fix:
Azure storage container Access policy
Azure storage container Access policy Set readonly/write
I tried the other one of the two access keys and it worked.
My fix was - Change access level to Blob,not a container
Here I'm trying to install "Microsoft Azure Active Directory Sync Service" but it's displays an error saying that "Unable to install the Synchronization Service. Please see the event log for additional details" so can any one help me out to get this done.
In Event Logs the error shown as
"System.Exception: Unable to install the Synchronization Service. Please see the event log for additional details. ---> Microsoft.Azure.ActiveDirectory.Synchronization.Framework.ProcessExecutionFailedException: Exception: Execution failed with errorCode: 1. Details: Sqlcmd: Error: Microsoft SQL Server Native Client 11.0 : SQL Server Network Interfaces: The specified LocalDB instance does not exist. [x89C50107]. . Sqlcmd: Error: Microsoft SQL Server Native Client 11.0 : Login timeout expired. Sqlcmd: Error: Microsoft SQL Server Native Client 11.0 : A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.. at Microsoft.Azure.ActiveDirectory.Synchronization.Framework.ProcessAdapter.StartProcessCore(String fileName, String arguments, String workingDirectory, NetworkCredential credential, Boolean loadUserProfile, Boolean hideWindow, Boolean waitForExit) at Microsoft.Azure.ActiveDirectory.Synchronization.Framework.ProcessAdapter.StartBackgroundProcessAndWaitForExit(String fileName, String arguments, String workingDirectory, NetworkCredential credential, Boolean loadUserProfile) at Microsoft.Azure.ActiveDirectory.Synchronization.Framework.SqlCmdAdapter.ExecuteCommand(String arguments, NetworkCredential credential) at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SynchronizationServiceSetupTask.<>c__DisplayClass19.b__17() at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SynchronizationServiceSetupTask.<>c__DisplayClass19.b__18() at Microsoft.Azure.ActiveDirectory.Synchronization.Framework.ActionExecutor.Execute(Action action, String description) at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SynchronizationServiceSetupTask.InitializeSqlSharedInstance(String sqlLocalDbInstanceOwnerSid, NetworkCredential sqlLocalDbInstanceOwnerCredential, SetupConfig config) at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SynchronizationServiceSetupTask.InstallCore() at Microsoft.Azure.ActiveDirectory.Synchronization.Framework.ActionExecutor.Execute(Action action, String description) at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SetupBase.Install() --- End of inner exception stack trace --- at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SetupBase.ThrowSetupTaskFailureException(String exceptionFormatString, String taskName, Exception innerException) at Microsoft.Azure.ActiveDirectory.Synchronization.Setup.SetupBase.Install() at Microsoft.Azure.ActiveDirectory.Synchronization.UserInterface.SetupAdapter.TypeDependencies.GenericDirectorySyncSetupInstall(String pathToSetupFiles, String installationPath, ProgressChangedEventHandler progressChangedEventHandler) at Microsoft.Azure.ActiveDirectory.Synchronization.UserInterface.UI.WizardPages.InstallOrUpgradePageViewModel.SetupTask(Object sender, DoWorkEventArgs args) at Microsoft.Azure.ActiveDirectory.Synchronization.UserInterface.UI.Controls.Wizards.ProgressReportingTaskViewModel.ExecuteAction(Action action, Boolean isProgressIndeterminate)"
Something during the installation has gone wrong. The official documentation says:
You need an account with local administrator privileges on your computer to install Azure AD Sync.
Azure AD Sync requires a SQL Server database to store identity data. By default a SQL Express LocalDB (a light version of SQL Server Express) is installed and the service account for the service is created on the local machine.
SQL Server Express has a 10GB size limit that enables you to manage approximately 100.000 objects.
Source: https://msdn.microsoft.com/en-us/library/azure/dn757602.aspx
Note: Azure AD Sync is replaced by Azure AD Connect. You can read more about Azure AD Connect here and here.