Extremely slow package restores for Azure/Kudu build - azure

Question

We have issues with very long deployment times in Azure/Kudu due to extremely slow package restores for our .NET Core projects. Here is output from the log:
2017-05-15T08:33:50.4762761Z,Command: deploy.cmd,,0
2017-05-15T08:33:51.2574560Z,Restoring packages for MySolution.sln,,0
2017-05-15T08:59:16.3007973Z, Restoring packages for D:\home\site\repository\src\MyProject\MyProject.csproj...,,0
2017-05-15T08:59:22.6581127Z, Lock file has not changed. Skipping lock file write. Path: D:\home\site\repository\src\MyProject\obj\project.assets.json,,0
2017-05-15T08:59:22.6737370Z, Restore completed in 6.33 sec for D:\home\site\repository\src\MyProject\MyProject.csproj.,,0
2017-05-15T08:59:22.6737370Z, ,,0
2017-05-15T08:59:22.6737370Z, NuGet Config files used:,,0
2017-05-15T08:59:22.6908155Z, D:\local\AppData\NuGet\NuGet.Config,,0
2017-05-15T08:59:22.6908155Z, ,,0
2017-05-15T08:59:22.7049839Z, Feeds used:,,0
2017-05-15T08:59:22.7049839Z, https://api.nuget.org/v3/index.json,,0
2017-05-15T08:59:22.7362324Z,Building and publishing MyProject.csproj,,0
2017-05-15T08:59:23.2362247Z,Microsoft (R) Build Engine version 15.1.1012.6693,,0
2017-05-15T08:59:23.2520644Z,Copyright (C) Microsoft Corporation. All rights reserved.,,0
2017-05-15T08:59:23.2520644Z,,,0
2017-05-15T09:12:28.8924730Z, MyProject -> D:\home\site\repository\src\MyProject\bin\Release\netcoreapp1.1\MyProject.dll,,0
As you can see, it takes almost 30 minutes between these steps alone:
2017-05-15T08:33:51.2574560Z,Restoring packages for MySolution.sln,,0
2017-05-15T08:59:16.3007973Z, Restoring packages for D:\home\site\repository\src\MyProject\MyProject.csproj...,,0
Help is much appreciated.
UPDATE
It doesn't seem to be an issue only with package restore. Here's an example of when the build itself also takes an hour:
* 2017-09-06T11:27:45.4325294Z,Restoring packages for MySolution.sln,,0
* 2017-09-06T12:10:47.8765057Z, Restore completed in 974.85 ms for D:\home\site\repository\src\MyProject\MyProject.csproj.,,0
* 2017-09-06T12:10:47.9397198Z,Building and publishing MyProject.csproj,,0
* 2017-09-06T13:09:33.8740387Z,Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core,,0
2017-09-06T13:09:33.8896654Z,Copyright (C) Microsoft Corporation. All rights reserved.,,0
2017-09-06T13:09:33.9052887Z,,,0
2017-09-06T13:29:16.1438980Z, MyProject -> D:\home\site\repository\src\MyProject\bin\Release\netcoreapp1.1\MyProject.dll,,0
2017-09-06T13:32:51.0677707Z, MyProject -> D:\local\Temp\8d4f51a4b579642\,,0
Nothing fancy is going on in the csproj:
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>netcoreapp1.1</TargetFramework>
<PreserveCompilationContext>true</PreserveCompilationContext>
<AssemblyName>MyProject</AssemblyName>
<OutputType>Exe</OutputType>
<PackageId>MyProject</PackageId>
<RuntimeFrameworkVersion>1.1.1</RuntimeFrameworkVersion>
<PackageTargetFallback>$(PackageTargetFallback);dotnet5.6;portable-net45+win8</PackageTargetFallback>
</PropertyGroup>
<ItemGroup>
<Compile Remove="*.ts;app\**\*" />
<None Update="rewrite-rules.xml">
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</None>
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.Diagnostics" Version="1.1.1" />
<PackageReference Include="Microsoft.AspNetCore.Server.IISIntegration" Version="1.1.1" />
<PackageReference Include="Microsoft.AspNetCore.Server.Kestrel" Version="1.1.1" />
<PackageReference Include="Microsoft.Extensions.Logging.Console" Version="1.1.1" />
<PackageReference Include="Microsoft.AspNetCore.StaticFiles" Version="1.1.1" />
<PackageReference Include="Microsoft.AspNetCore.Rewrite" Version="1.0.1" />
<PackageReference Include="Microsoft.AspNetCore.ResponseCompression" Version="1.0.1" />
<PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="1.0.2" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="1.1.2" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="1.1.2" />
<PackageReference Include="Microsoft.Extensions.Configuration.FileExtensions" Version="1.1.2" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="1.1.2" />
<PackageReference Include="Microsoft.AspNetCore.WebUtilities" Version="1.1.2" />
</ItemGroup>
</Project>