ImageMagick on Azure Functions cannot load native library - azure

I'm trying to use Magick.NET library to do image processing on Azure Functions.
I have tried the same code in a Console App (.NET Core 3.1) which works without any issues. However when running the same code in an Azure Functions Project, I get the following error.
Code:
var image = new MagickImage(File.ReadAllBytes(#"<My Image Path>"));
Exception:
System.TypeInitializationException: 'The type initializer for 'NativeMagickSettings' threw an exception.'
StackTrace:
at ImageMagick.MagickSettings.NativeMagickSettings..ctor()
at ImageMagick.MagickSettings..ctor()
at ImageMagick.MagickImage..ctor()
at ImageMagick.MagickImage..ctor(Byte[] data)
at FunctionApp2.Function1.Run(TimerInfo myTimer, ILogger log) <-- My Code
Inner Exception:
Unable to load DLL 'Magick.Native-Q8-x64.dll' or one of its dependencies: The specified module could not be found. (0x8007007E)
Inner Exception StackTrace
at ImageMagick.Environment.NativeMethods.X64.Environment_Initialize()
at ImageMagick.Environment.NativeEnvironment.Initialize()
at ImageMagick.Environment.Initialize()
at ImageMagick.MagickSettings.NativeMagickSettings..cctor()
I believe this is due to the way Azure Functions package is organized and the path Magick.NET is looking for its native binary to load. I can confirm that the native binary is written at \bin\Debug\netcoreapp3.1\runtimes\<platform>\native when the Function app is built.
I have traced the apps using ProcessMonitor for each app and can see Function app looks in the wrong location.
It looks in
\bin\Debug\netcoreapp3.1\ bin\ runtimes\[platform]\native
instead of
\bin\Debug\netcoreapp3.1\runtimes\[platform]\native
(Below traces are filtered to only show path contains Magick.Native and process name filtered to the Console app exe and func.exe)
Console App Trace
Function App Trace
I have also tried setting MagickAnyCPU.CacheDirectory to Directory.GetCurrentDirectory() but it hasn't helped.
UPDATE:
After building, if I copy \netcoreapp3.1\runtimes folder to \bin this fixes the problem, but I current don't want to rely on that trick if there's a proper workaround.
UPDATE 2:
This issue appears to happen only when using
<TargetFramework>netcoreapp3.1</TargetFramework>
and Microsoft.NET.Sdk.Functions version 3.0.x
It does not appear when using older versions. I have successfully used
<TargetFramework>netcoreapp2.1</TargetFramework>
and Microsoft.NET.Sdk.Functions version 1.0.x
without this issue.
So, is there a workaround to get ImageMagick to load native lib from the right place?
Potential Workaround is to get msbuild to copy the runtimes folder after the build completes.
Add following to csproj
<Target Name="PostBuild" AfterTargets="PostBuildEvent">
<Exec Command="xcopy /Y /E $(OutDir)runtimes\ $(OutDir)bin\runtimes\" />
</Target>
or right click project > properties > Build Events and paste the above command to post build event command line
This will copy all of the runtimes binaries, not just the one we want, increasing the size of final build.

UPDATE:
This may be a bug in Microsoft.NET.Sdk.Functions version 3.0.8.
These issues in the github repo are tracking this.
Version 3.0.8 places the runtimes folder incorrectly
Microsoft.Data.SqlClient is not supported on this platform with Microsoft.NET.Sdk.Functions 3.0.8
This seems to only happen when using Microsoft.NET.Sdk.Functions version 3.0.8 which seems to have only just appeared in nuget on Friday. I will instead downgrade to 3.0.7.
Alternatively if you don't want to downgrade,
You could use MagickNET.SetNativeLibraryDirectory as a work around

Related

Azure Functions V1 linking wrong version of the RestSharp.dll

Guys I know that the azure functions cli has a dependency with RestSharp.dll and I think that is conflicting with one of my Azure Functions.
I am getting a runtime type exception
System.TypeLoadException: 'Could not load type 'RestSharp.IAuthenticator' from assembly 'RestSharp, Version=105.2.3.0, Culture=neutral, PublicKeyToken=null'.'
Now my azure function is dependent on RestSharp nuget Version 104.4.0.0. There is no reference in my project to Version 105.2.3.0. Here is the interesting thing. In my despair I searched my entire computer for the culprit dll version 105.2.3.0 and I found it at AppData\local\Azure.Functions.Cli\1.0.12
Is that it ? Is the Azure functions runtime trying to link with its RestSharp.dll version instead of the dll version of my project ?
Runtime should technically load version 104.4.0.0. However it is still loading the version referred by the runtime (105.2.3.0). I was able to verify this behavior. Have filed a bug for this https://github.com/Azure/azure-functions-host/issues/2832.
In the meantime is it possible to do one of the following:
Update the code to use 105.2.3.0, I see RestSharp.IAuthenticator type is present. It is under a different namespace. There should be another method exposing the same functionality
If the function app is not being used in prod. You could use the beta runtime. You should not encounter this issue in beta runtime (v2.x)

.NET Core Runtime Package Store on Linux

I created a console application and added Newtonsoft.json as reference for test.
After following the steps, I published the application (netcoreapp2.0) to ubuntu.16.04-x64 version and deployed it to a Ubuntu(16.04) machine, when I run the console application, it always shows below error even the package exists in /usr/local/share/dotnet/store
An assembly specified in the application dependencies manifest (RuntimePackageConsole.deps.json) was not found:
package: 'Newtonsoft.Json', version: '11.0.1'
path: 'lib/netstandard2.0/Newtonsoft.Json.dll'
This assembly was expected to be in the local runtime store as the application was published using the following target manifest files: artifact.xml
After I changed the path of Newtonsoft.Json.dll in dependency json file to absolute path, then it works.
Anybody had the same issue on Linux? Or how can I find out the reason, why can't application load the package by relative path in dependency json file?
It looks like you are targeting a manifest when you publish your code.
Starting with .NET Core 2.0, it's possible to package and deploy apps against a known set of packages that exist in the target environment. The benefits are faster deployments, lower disk space use, and improved startup performance in some cases.
This feature is implemented as a runtime package store, which is a directory on disk where packages are stored (typically at /usr/local/share/dotnet/store on macOS/Linux and C:/Program Files/dotnet/store on Windows). Under this directory, there are subdirectories for architectures and target frameworks. The file layout is similar to the way that NuGet assets are laid out on disk:
A target manifest file lists the packages in the runtime package store. Developers can target this manifest when publishing their app. The target manifest is typically provided by the owner of the targeted production environment.
https://learn.microsoft.com/en-us/dotnet/core/deploying/runtime-store
So you may want to take a look at your environment and ensure that your package stores do in fact have the required libraries, or that you publish without a manifest.

Azure Function fails to bind ILogger

My function is referencing an assembly that references Microsoft.Extensions.Logging.Abstractions 2.0.0. If I add a nuget reference to that version to the function's assembly, function execution fails with:
[1/25/2018 11:14:46 PM] Microsoft.Azure.WebJobs.Host: Error indexing
method 'TrainingFunction.Run'. Microsoft.Azure.WebJobs.Host: Cannot
bind parameter 'log' to type ILogger. Make sure the parameter Type is
supported by the binding. If you're using binding extensions (e.g.
ServiceBus, Timers, etc.) make sure you've called the registration
method for the extension(s) in your startup code (e.g.
config.UseServiceBus(), config.UseTimers(), etc.).
Is it possible to use the newer logger in Azure functions ?
(SDK 1.0.7)
What is probably happening is that the SDK is binding to version X of the ILogger assembly and your user code is binding to version Y. The binding engine then doesn't recognize your parameter's type as being the same since they're from different assemblies. (this can happen with any other type too).
Generally the fix is to:
See the nugget references used by the SDK
Use those existing references and don't add the same dll with a different version.
I was somehow also having the same error, but it was the package version for Microsoft.EntityFrameworkCore.SqlServer what is causing the issue.
Downgrading Microsoft.EntityFrameworkCore.SqlServer v2.2.0 to v2.1.4 did the trick.
I assume that there is a version mismatch between logging.abstractions libraries for this package.
For me, the problem was that I needed to explicitly declare the Azure Functions Version in my .csproj file.
I added <AzureFunctionsVersion>v2</AzureFunctionsVersion> after the <TargetFramework> element:
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<AzureFunctionsVersion>v2</AzureFunctionsVersion>
</PropertyGroup>
Hope that helps someone :-)
Also binding-order could cause this failure.
Changing from parameter order ...
[FunctionName("SomeFunction")]
public static async Task Run([BlobTrigger("path", Connection = "conn")]
ILogger logger, ExecutionContext context, Stream stream, string name) {}
... to ...
[FunctionName("SomeFunction")]
public static async Task Run([BlobTrigger("path", Connection = "conn")]
Stream stream, string name, ILogger logger, ExecutionContext context) {}
... fixed my problem. (Microsoft.NET.Sdk.Functions v1.0.24)
For me, I was using NuGet package Microsoft.Extensions.Logging on a project referenced by my Azure function. Uninstalled this package and my error (this exact error) went away.
I didn't actually need the NuGet package to use ILogger in my Azure function.
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
ILogger log)
Edit: Also downgrading to Microsoft.Extensions.Http 2.1.0 worked on another occurrence of this error. If you still need the Extensions package (for IHttpClientFactory for example) it's possible to get this logging error because of an SDK version conflict with 3.1.1.
As stated by one of MS employees the cause could be:
we don't quite yet support .NET Core 2.2, but we have the work underway and should ship in January.
https://github.com/Azure/azure-functions-host/issues/3854#issuecomment-449034719
Another reason for the same error...
Somehow I ended up with a using statement referencing Microsoft.Build.Framework which has its own version of ILogger the fix is just to replace this with Microsoft.Extensions.Logging which the functions app is expecting
This will also happen when updating Azure Functions/Durable Functions from v2 to v3. You need to manually edit .csproj to resolve the ILogger reference problem.
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<AzureFunctionsVersion>v3</AzureFunctionsVersion>
</PropertyGroup>
<ItemGroup>
...
<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.2" />
</ItemGroup>
The function will continue to report a run-time module error. Then you have to try to create new Azure Function v3 project in Visual Studio to download the templates for run-time modules. More information on the procedure is provided in the official documentation: Azure Functions runtime versions overview
I had to uninstall Microsoft.Extensions.Logging nuget package from the solution.
I suddenly started getting the ILogger binding error after auto-updating my Nuget packages.
I couldn't find any explict references to ILogger extensions in my solution files. It turned out the culprit was a different extension Microsoft.Extensions.Caching.Memory that updated to v5.00 (.NET 5?). Downgrading that package to the prior version 3.1.15 fixed the error (I was originally at v3.1.4). I wonder if any v5.x extension package would trigger this issue.
What worked for me was downgrading "Microsoft.Extensions.Http" from version 5.x.x.x to "3.1.20", which implicitly downgrades "Microsoft.Extensions.Logging" to "3.1.20"
It looks like there is an implicit dependency with the version 5.x of Http library and "Microsoft.Extensions.Logging" version 5.x , and that version of logging has some issue with it.

Azure Web Deploy "Could not find a part of the path 'D:\home\site\wwwroot\bin\roslyn\csc.exe'."

I've been browsing the web for a couple hours now looking for an answer to my problem. I am trying to deploy a Web API on Azure Web App Service using VS2017. Everything builds and works fine when running locally but once deployed on Azure (through VS2017) I get this error:
D:\home\site\wwwroot\bin\roslyn\csc.exe
My project is an ASP.NET Web Application (Using Azure Web API template) .NET Framework 4.6.1. I use Microsoft.CodeDom.Providers.DotNetCompilerPlatform Version=1.0.6.0
I also made sure that csc.exe is located in:
Visual Studio 2017\Projects\DeviceManagementAPI\DeviceManagementAPI\bin\roslyn
Just had the same problem and it seems it's a known issue with Microsoft.CodeDom.Providers.DotNetCompilerPlatform 1.0.6 and 1.0.7.
Downgrading to 1.0.5 solves the problem.
Upgrading Microsoft.CodeDom.Providers.DotNetCompilerPlatform to 1.08 worked for me
After a while I simply uploaded manually the roslyn file directly into the server through Kudu. It seems to solve the problem but I still don't know why it won't upload automatically.
Same issue might be caused by missing or incorrect relative packages path. If you're changing solution folder structure make sure all imports have proper path's to avoid missing Roslyn files.
Generally suggest to replace auto-generated ../../../packages/... rabit hole with parameter which will point to correct Nuget folder.
<Import Project="$(NugetPackagesPath)\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.1.0.8\build\net45\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.props" Condition="Exists('$(NugetPackagesPath)\packages\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.1.0.8\build\net45\Microsoft.CodeDom.Providers.DotNetCompilerPlatform.props')" />

How to choose a custom node version for Azure WebJobs

I'm working on a website that deploys its custom version of node (x64) that we upload and configure via iisnode.yml. We have a dependency in a binary package (zopfli) that we build locally and deploy as well. So far so good.
The problem is with one of the webjobs we have. It seems like it is always using the x86 version of the node version set in WEBSITE_NODE_DEFAULT_VERSION no matter the value of issnode.yml or the platform I select via the portal.
So my question is:
How can I tell my webjobs to use the same version I specify in iisnode.yml (bin\node.exe)?
The WebJob won't look at iisnode.yml. The simplest solution is to create a run.cmd file in your WebJobs folder, and have it explicitly run what you want. e.g. it could have:
bin\node.exe foo.js

Resources