General: I develop an Azure DevOps extension with tasks and pipeline decorators. Testing on local Azure DevOps Server instance. Extension loaded through manage extensions from local hard drive. Let's say that I installed the extension first time with version 1.0.0 and a node_modules dependency "3rdPartyDep" with version 2.0.0, which has transitive dependencies with vulnerabilities.
Scenario:
Upgrade "3rdPartyDep" to version 3.0.0 with fixed vulnerabilities. Build new version of my extension, say 1.0.1. Create the .vsix, update the extension in the Azure DevOps Server.
Run a pipeline, which fails because I did not check the "3rdPartyDep" changes and there are breaking changes and the extension fails to run.
Rollback the "3rdPartyDep" library to 2.0.0 because I have no time now to check what is broken in there right now as I have other things to debug and implement, repackage the extension, increase version to 1.0.2, update extension in Azure DevOps Server.
Run the pipeline. It fails with the same exception, as if I didn't rollback. I look into the agent taks folder and I see that the node_modules with the "3rdPartyDep" library is pointing to 3.0.0, which is wrong because I rolled back the version.
I open the generated .vsix archive and check that the node_modules inside contains the correct 2.0.0 version, so no problems of packaging or building from my side.
I make a conclusion that Azure DevOps stores somewhere a cached version of the extension with the node_modules including the wrong version of the "3rdPartyDep". I search that cache folder over internet to find out where it is, and I also search with a search tool all my machine, including words in file. Nowhere to be found. There is no location on my machine with such node_modules containing the 3.0.0 version. It might be stored in some encrypted DB?
I uninstall completely the extension, and install it back. I see that Azure DevOps has a history for the extension, and the cache is not cleared. Any pipeline fails, even if my .vsix does not contain this dependency.
I'm stuck.
Questions:
Where extensions are actually cached inside Azure DevOps Server?
Why updating, uninstalling and installing does not fix the problem?
Is there any way to fix this? What can I do? I do not want to reinstall the server completely. Moreover, this raises concerns about how node_modules are managed and cached and what happens at the clients and the cloud.
You could try the following items:
Try to clean the browser cache, and check whether you have increase the version number in the task.json.
Try to perform Delete task -- Save definition -- add task again process.
Delete Azure DevOps Server cache, which can be followed in this link.
Uninstall the extension from CollectionSettings, remove the extension from local Manage Extensions. Then upload again the extension and install it in the collection.
Related
I have permanently deleted a package from Artifacts.
Is there a way to restore it?
Now if I try to push I am getting this error:
Response status code does not indicate success: 409 (Conflict - The version 3.18.1 of Google.Protobuf has been deleted. It cannot be restored or pushed. (DevOps Activity ID: 014272E1-B940-428D-B3DF-4E67E5890D4B))
To add a Artifacts deleted nupkg
I am afraid there is no such way to restore it.
According to the document Understand immutability of packages:
Once you publish a particular version of a package to a feed, that version number is permanently reserved. You cannot upload a newer revision package with that same version number, or delete it and upload a new package at the same version.
That because many package clients, including NuGet, keep a local cache of packages on your machine. Once a client has cached a particular package#version, it will return that copy on future install/restore requests. If, on the server, you replace package#version (rev 1) with a new package#version (rev 2), the client is unable to tell the difference. This can lead to indeterminate build results from different machines.
So we could not overwrite all packages that have already been uploaded, even we could not delete them and re-uploaded. To resolve this issue, you could try to upload the packages with a new version or create a new feed to save the package.
I am new to Azure DevOps and trying to create my first Azure pipeline. I have a ASP.NET MVC project and there are a few NuGet packages that need to be restored before the MSBuild step.
Unfortunately, the NuGet restore is failing with the following error:
The pipeline is not valid. Job Job_1: Step 'NuGetCommand' references
task 'NuGetCommand' at version '2.194.0' contains an execution handler
that relies on NodeJS version '6' which is restricted by your
administrator.
NodeJS 6 came disabled out of the box so we are not going to enable it.
My Questions:
Is there an alternative to NuGet restore that does not use NodeJS?
Is there a way to update the NodeJS6 to a higher version?
update 23-Nov-2021
I have found a work around for the time being. I am using a custom PowerShell script to restore NuGet Packages and build Visual Studio project
$msBuildExe = 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\MSBuild.exe'
Write-Host "Restoring NuGet packages" -foregroundcolor green
& "$($msBuildExe)" "$($path)" /p:Configuration=Release /p:platform=x86 /t:restore
Note: $path here is the path to my .csproj file
Apparently, other people are also getting the same issue and it is just a matter of time that the task is updated by the OpenSource community.
Here are some similar issues being faced in other tasks as well:
https://github.com/microsoft/azure-pipelines-tasks/issues/15526
https://github.com/microsoft/azure-pipelines-tasks/issues/15511
https://github.com/microsoft/azure-pipelines-tasks/issues/15516
https://github.com/microsoft/azure-pipelines-tasks/issues/15525
It's AzureDevOps' NuGetCommand task that uses NodeJS, not NuGet itself. Therefore, you can find a way to restore without using Azure DevOps' NuGetCommand task.
Idea 1: use DotnetCoreCli task instead. However, this probably won't work for you since you said your project is ASP.NET MVC, rather than ASP.NET Core. Also, it also appears to need NodeJS to run.
Idea 2: Use MSBuild restore. You can test on your local machine whether or not this works by clearing your global packages folder, or temporarily configuring NuGet to use a different path, and then running msbuild -t:restore My.sln from a Developer PowerShell For Visual Studio prompt. If your project uses packages.config, rather than PackageReference, you'll need to also pass -p:RestorePackagesConfig=true (although maybe this is currently broken). I'm not an expert on Azure Pipelines tasks, so I don't know what it means that this task defines both PowerShell and Node execution entry points, but maybe it means it will work even if your CI agent doesn't allow NodeJS.
Idea 3: Don't use any of the built-in tasks, just use - script: or - task: PowerShell#2, but even that is a little questionable whether it'll work since even the powershell task defines a Node execution entry point. I'm guessing it will work, but I don't have access to a CI agent where NodeJS is forbidden, so I couldn't test even if I wanted to. Anyway, if this works, then you can run MSBuild yourself (but it might also be your responsibility to find msbuild.exe if it's not on the path). Or you can download nuget.exe yourself and execute it in your script. The point is, if you can get Azure Pipeline's script task working, you can run any script and do everything you need yourself.
Idea 4: Use Microsoft Hosted agents. They have documented all the software they pre-install on the machines, which includes Node JS. Downside is that once you exceed the free quota it costs money, and I've worked for companies where it's easier to get money to buy hardware once-off, and pretend that maintenance of the server is free, even though it reduces team productivity, rather than pay for a monthly service. So, I'll totally understand if this is not an option for you.
Idea 5: Talk to whoever maintains your CI agents and convince them to allow & install NodeJS. It's clearly a fundamental part of Azure Pipelines. The tasks are open source on github, and you can see that pretty much all of them use NodeJS to orchestrate whatever work it does. Frankly, I thought the agent software itself was a NodeJS application, so I'm surprised that it runs without NodeJS.
I'm working on a project in Azure DevOps and, as recommended in the doc, I created an Artifacts Feed with maven central as upstream source to store all my dependencies (I don't really need to publish artifacts for now).
So I configured my local maven to fetch all the dependencies from my feed instead of maven central and it all works fine, except that it's very slow compared to maven central.
When I start from an empty .m2 on my local machine, it takes 1 min 15 secs to build my project when downloading the dependencies from maven central, but it takes over 8 minutes to do the same when downloading the dependencies from the Feed (which contains already all the dependencies).
I could live with that, since the download of everything happens only on the first build.
But the issue is that it's also slower when building my project from Azure Pipelines, which I really didn't expect since it's a connection from Azure to Azure and within the same organization. In this case, it takes at least twice the time when using the feed rather than maven central. And this will be true every time since Azure Pipelines gives you a fresh VM each time you build (I'm using a hosted agent), so there's no dependencies caching in this case.
It's really annoying since my project is just a HelloWorld so far, so it will only get worse over time.
Using a repository manager/feed is the best practice according to both Maven and Azure, but at this point I'm really thinking of going for the bad practice of getting everything from maven central instead of my feed, at least in my pipeline, to improve the performance.
Am I the only one having this issue ? What are your thoughts about this ?
Finally, after diving into the documentation for Azure Pipelines recently, I found out there is a way to cache the maven repository between runs so it partially solves my issue since the full download of the dependencies will happen only once.
Here is the doc in question for those who are interested.
I'm trying to use EventHub from a WebJob as described https://github.com/Azure/azure-webjobs-sdk/wiki/EventHub-support and Any Example of WebJob using EventHub?.
The problem is that the SO question refers to the nuget package Microsoft.Azure.WebJobs.ServiceBus.1.2.0-alpha-10291 and as far as I can see, that package isn't available anymore.
Does anybody know if there's a new release in the pipe or any other workaround?
BR,
Max.
That library is available from MyGet repository and is being updated sometimes. For adding that, you need to add that repository to NuGet settings:
Package Manager Settings => create a new package source => Use https://www.myget.org/F/azure-appservice/ as a link for the repository.
Now, you should have the library and its dependencies.
I have an Azure Website configured to deploy from a Bitbucket repository. This works fine.
Since the application is still in active development, I update the nuget packages it uses quite frequently. This causes the packages folder to keep growing indefinitely, unless I go and manually delete the packages.
Now, in my local machine this is not a big issue. Space is cheap. But in Azure, this makes us go over the quota really fast, as old packages accumulate.
How can I customize the Azure deploy process so that it deletes all the packages after a successful deployment?
(I am open to other solutions as well)
You can utilize the custom deployment script feature where you add a step that cleans up the packages directory.
You can read about it here:
http://blog.amitapple.com/post/38418009331/azurewebsitecustomdeploymentpart2/
Another option is to add a post deployment action, by adding a script file (.cmd/.bat) that has the clenup logic to the following directory in your site: d:\home\site\deployments\tools\PostDeploymentActions\, this script will run after the deployment completes successfully.
Read more about it here:
https://github.com/projectkudu/kudu/wiki/Post-Deployment-Action-Hooks