I have to connect Octopus-Deploy to an external feed located on a Azure Devops Artifact Feed. I've successfully connected to the feed, but seeing how I packaged my SSIS project (.ispac) and SSDB project (.dacpac) as .dacpac and .ispac, it won't find them as it expects them to be in .NuGet format.
I haven't been able to try much as I'm pretty stuck.
N/A
I'm constrained to put whatever type of package I make on this feed to connect to from Octopus and deploy from. I have to have either have all my packages be NuGet Packages or find and alternate way that Octopus Deploy can find my packages on the Azure External Feed w/out them being NuGet. I'm only deploying SSDT things: i.e. dacpac, ispac, sql-agents, sql scripts etc.
You can make a NuSpec file which contains the things you need to package into NuGet to use a NuGet external feed in Octopus with. In the case of SSIS, you will want to reference the .ispac file in your nuspec file like so:
{
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
<metadata>
<id>SSIS.ODSToDW</id>
<version>2.0.0</version>
<authors>blah</authors>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description>ODSToDW ispac</description>
</metadata>
<files>
<file src="bin/Development/ODS-To-DW.ispac" target="ODS-To-DW.ispac" />
</files>
</package>
}
You'll then do a "build" pipeline in DevOps which builds your solution. In the case of ispac you will need to use SSIS Build (you can get it from the market place).
Upon building the solution so that the latest .ispac is formed, you then use NuGet pack pointed to your .nuspec file like so:
{
- task: NuGetCommand#2
inputs:
command: 'pack'
packagesToPack: '**/*.nuspec'
configuration: 'Release'
versioningScheme: 'byPrereleaseNumber'
majorVersion: '1'
minorVersion: '0'
patchVersion: '2'
packTimezone: 'local'
}
Then you can do a "NuGet push" to your Artifact feed in azure. From which Octopus will be able to connect using "external NuGet feed" which will allow you to using "Deploy Package" from the Octopus Process options from which you can point to your specific artifact using it's name and a post-deployment powershell script will allow you to deploy to your sql server.
Related
I'm doing a zip deploy of a .NET Framework web app to an Azure App Service via a GitHub workflow.
I have set WEBSITE_RUN_FROM_PACKAGE to 1 in the Azure console's Settings / Configuration / Application settings page. I've also tried setting WEBSITE_RUN_FROM_ZIP to 1 there just in case (although I think this is an obsolete flag).
The package is building correctly in GitHub and I can see it showing up in my Kudu debug console, under C:\home\site\wwwroot (as MyPackageName.zip) as well as in C:\home\data\SitePackages (as 20220512205318.zip, for example).
The deploy portion of my YAML is:
deploy:
runs-on: windows-latest
needs: build
environment:
name: 'Test'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: ASP-app
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: ${{ env.AZURE_WEBAPP_NAME }}
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_XYZsecret }}
package: .
And the .PublishSettings I've uploaded to GitHub looks like:
<publishData>
<!-- Which one of these 3 profiles is my YAML using? I don't actually know. -->
<publishProfile profileName="mywebappname-test - Web Deploy" publishMethod="MSDeploy" etc="foobar">
<databases/>
</publishProfile>
<publishProfile profileName="mywebappname-test - FTP" publishMethod="FTP" etc="foobar">
<databases/>
</publishProfile>
<publishProfile profileName="mywebappname-test - Zip Deploy" publishMethod="ZipDeploy" etc="foobar">
<databases/>
</publishProfile>
</publishData>
The zip package is not getting automatically unpacked. The MSFT support rep I talked to suggested that this was the problem, and indeed when I download the package to my machine and drop it into Kudu's Tools/Zip Push Deploy page, I see that the package is unpacked, and I can get the site to work by setting the appropriate Physical Path to match the '/' Virtual path. Specifically the Kudu Tools Zip Push causes my web.config and favicon.ico etc. files to show up in:
C:\home\site\wwwroot\Content\D_C\a\foo\bar\good\boy\obj\Test\Package\PackageTmp
and I can go to the Azure console for my app service, navigate to Settings / Configuration/ Path Mappings, Virtual applications and directories, and edit the existing entry to:
Virtual path: /
Physical Path: site\wwwroot\Content\D_C\a\foo\bar\good\boy\obj\Test\Package\PackageTmp
Type: Application
and then see my site come up in a browser.
However, when don't do anything to unpack the archive, and I leave the entry as:
Virtual path: /
Physical Path: site\wwwroot
Type: Application
I can't see my site in a browser and instead just see "You do not have permission to view this directory or page." When I then dig in to the logs in Kudo, I see 403.14 - Forbidden errors on my main site and a 404.0 - Not Found error on C:\home\site\wwwroot\favicon.ico. (Like the rest of my files, favicon.ico is still inside the zip archive at [...]\foo\bar\good\boy\obj\Test\Package\PackageTmp\favicon.ico.)
My questions are:
Should my web app be able to run at all with just my zip file sitting there as C:\home\site\wwwroot\MyPackageName.zip? Or does it really need to be unpacked as the MSFT rep indicated?
If it is supposed to run this way, any ideas on what am I missing? I assume it's something in my YAML (which of the 3 publishProfile settings is it actually choosing here?) or in Settings / Configuration/ Path Mappings or Application settings, but I have no idea what at this point and I'm running out of ideas.
Thanks, Eric
Should my web app be able to run at all with just my zip file sitting there as C:\home\site\wwwroot\MyPackageName.zip?
Pretty much, yes, just not in wwwroot. When WEBSITE_RUN_FROM_PACKAGE is enabled, the application is run from the archive directly as a read only directory mount. Nothing is copied to wwwwroot or anywhere else.
If it is supposed to run this way, any ideas on what am I missing?
My understanding is package deploy from GitHub is not supported or rather GitHub archives are incompatible with run from package on App Service.
dwellman's response was the correct answer to my original question, but I'll add some more detail here on how I used this information to get my deployment to work properly. I feel like the inability to read the zip archive's internal index XML file(s) to find the correct relative path is an Azure flaw, but until it's addressed, I hope others may find this useful.
My first step was to abandon the idea of deploying as a zip file. It's possible I could have still made this work by doing some post-processing to zip things up in a different format without the nested folders, but I decided in my case that the benefits of a single-file deployment weren't worth the cost. To stop deploying as a zip file, I manually edited the .pubxml file I was passing in as msbuild option /p:PublishProfile=AzureCI.pubxml. The changes I made were to change PackageAsSingleFile from true to false, and change DesktopBuildPackageLocation from a zip file path to a folder path.
This alone was enough to get my site to get deployed to Azure as individual files instead of a zip archive. The files were still buried in an ugly folder structure, but I could at least see them in Kudu and get the site to work by applying the same Settings / Configuration/ Path Mappings, Virtual applications and directories adjustment I describe in my original question.
I could have stopped there, but I wanted to be able to just use the default virtual path and not have my Azure configuration be so dependent on my upstream processes. In other words, I wanted to just have my web.config and favicon.ico etc land directly in C:\home\site\wwwroot instead of deep in the weeds of a subfolder structure. To make this work, I changed the package argument to webapps-deploy in my YAML from . to the appropriate path as follows:
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: ${{ env.AZURE_WEBAPP_NAME }}
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_XYZsecret }}
package: .\Archive\Content\D_C\a\foo\bar\good\boy\obj\Test\Package\PackageTmp
This caused the deployment process to pick off just the files I needed from the build and drop them into C:\home\site\wwwroot. I could then revert the path mapping kludge and be on my way.
I'm Trying to setup a Release CD for D365 (or Power Platform) using the "Power Platform Deploy Package" task:
https://learn.microsoft.com/en-us/power-platform/alm/devops-build-tool-tasks
I can see the build and release are flowing correct except for the very last part.
I can tell it works all the way to the "Download Artifact" part "
Download Artifact
Do I need anything else beside those 2?:
pipeline
I get this "Package File not specified or not found" error:
error
same error with several combinations of env variable paths. tried the exact path and still does not work
Am I forgetting anything?
path
The code is produced with the VS CRM Package and compiled just fine. I only updated the .net framework version to 4.7.2.
vs template
If you are using Hosted agent instead of Self-hosted agent, we need to specify the path using predefined variables $(Build.ArtifactStagingDirectory) instead of D:\a\r1\a, then it should be work.
Also we could add the task Copy files to filter the .dll file and copy them to another path, then specify the Package File path.
Note: I just share the yaml sample, you could enter the variable to class edit mode(UI)
- task: CopyFiles#2
displayName: 'Copy Files to: $(Build.ArtifactStagingDirectory)'
inputs:
Contents: |
$(Build.ArtifactStagingDirectory)/_Demo-CI/drop/bin/Debug/*.dll
TargetFolder: '$(Build.ArtifactStagingDirectory)'
Below is Azure error log from the Azure release pipeline.
2020-07-24T05:17:13.7712974Z ##[section]Starting: Deploy Azure App Service
2020-07-24T05:17:14.0464143Z ==============================================================================
2020-07-24T05:17:14.0465719Z Task : Azure App Service deploy
2020-07-24T05:17:14.0469391Z Description : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2020-07-24T05:17:14.0469849Z Version : 3.163.5
2020-07-24T05:17:14.0470410Z Author : Microsoft Corporation
2020-07-24T05:17:14.0470921Z Help : https://aka.ms/azureappservicetroubleshooting
2020-07-24T05:17:14.0471274Z =============================================================================
2020-07-24T05:17:16.0650643Z Got connection details for Azure App Service:'XXXXX'
2020-07-24T05:17:17.6576087Z (node:764) Warning: Use Cipheriv for counter mode of aes-256-ctr
**2020-07-24T05:17:17.6627737Z ##[error]Error: No package found with specified pattern: D:\a\r1\a\Drop<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.**
2020-07-24T05:17:22.9168975Z Successfully added release annotation to the Application Insight : XXXXX
2020-07-24T05:17:24.8686063Z Successfully updated deployment History at https://XXXXX.scm.azurewebsites.net/api/deployments/231595567842919
2020-07-24T05:17:24.8933576Z ##[section]Finishing: Deploy Azure App Service
P.S. I have checked build pipeline, I can download build Artificats, but it displays below warning.
[warning]Directory 'D:\a\1\a' is empty. Nothing will be added to build artifact 'drop'.
Assuming you meant you have solved the build artifact empty issue, but the release pipeline still had issue in Azure App Service deploy task.
As the error mentioned, there is "No package found with specified pattern: D:\a\r1\a\Drop", you need to check how you define the package, try format $(System.DefaultWorkingDirectory)/**/*.zip instead of $(System.DefaultWorkingDirectory)\Drop.
As task "Publish Build Artifact" publish artifact from $(build.artifactstagingdirectory), so we need add a task "Copy files" to copy your files ** from $(system.defaultworkingdirectory) to $(build.artifactstagingdirectory) **. which copied my build to stagingdirectory.
We have dotnet core application with Nuget.config file . As part of azure devops CI pipeline, we want to read and replace environment variables in Nuget.config file. Environment/Pipeline variables(USERNAME and PWD) are configured in "Build Docker Image" Task in CI pipeline. The azure devops CI build server runs on Ubuntu machine.
Environment variables in MAC/Linux should be replaced in following way as per this documentation.
The syntax "$USERNAME" in Nuget.config is not working here. When we replace static username and password, it works fine.
Nuget.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="abcFeed" value="https://somedevops.com/xyz/packaging/someApps/nuget/v3/index.json" />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
</packageSources>
<packageSourceCredentials>
<abcFeed>
<add key="Username" value="$USERNAME"/>
<add key="ClearTextPassword" value="$PWD"/>
</abcFeed>
</packageSourceCredentials>
</configuration>
We have tried below syntax with following documentation, none of them are working: $USERNAME, $(USERNAME), %USERNAME%, $USERNAME$
https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file#example-config-file
https://intercom.help/appcenter/en/articles/2216043-restore-private-azuredevops-package-management-feeds-using-pat.
Setting an environment variable in a NuGet.Config file
What is correct syntax or way of replacing environment variables in Nuget.config file with azure devops pipeline?
You could use a Replace Tokens task instead, and configure this to process your nuget.config file.
However, you may want to consider using a Nuget service connection as that will avoid having to save the Nuget credentials as pipeline variables and also make them more secure as they will be exposed in plain text when replaced in the config file.
For more information about using Service Connections in Azure DevOps, see the Microsoft documentation at: https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-nuget
I believe our nuget docs need correcting. Use the same characters for environment variable replacement, regardless of platform: %myPW%
I tested successfully with dotnet.exe 3.0.100 on Ubuntu 18.*
Please engage with me on the tracking issue I have assigned to me: https://github.com/NuGet/Home/issues/8662
I work in a dev environment where the computers aren't part of a domain. I'm trying to set up TeamCity 8.05 on one of the computers to publish check-ins to a test server for our QA team. I've got it working the way I want when I deploy to local directories on the TeamCity server, but when I try to publish to the QA server I run in to permissions problem. I have a share on the QA server with a local user who I want to authenticate with when doing my publish. Is this possible with MSBuild?
Command line switches to my MSBuild runner in TeamCity look like this:
/p:PublishProfile=./Alpha.publish /p:DeployOnBuild=true
/p:Configuration=Alpha /p:VisualStudioVersion=11.0
/p:UserName=QAServer\BuildUser /p:Password=somepass
The Alpha.publish file looks like this:
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<WebPublishMethod>FileSystem</WebPublishMethod>
<LastUsedBuildConfiguration>Alpha</LastUsedBuildConfiguration>
<LastUsedPlatform>Any CPU</LastUsedPlatform>
<SiteUrlToLaunchAfterPublish />
<LaunchSiteAfterPublish>False</LaunchSiteAfterPublish>
<ExcludeApp_Data>False</ExcludeApp_Data>
<publishUrl>\\QAServer\Alpha</publishUrl>
<DeleteExistingFiles>True</DeleteExistingFiles>
<PrecompileBeforePublish>True</PrecompileBeforePublish>
<EnableUpdateable>True</EnableUpdateable>
<DebugSymbols>False</DebugSymbols>
<WDPMergeOption>DonotMerge</WDPMergeOption>
</PropertyGroup>
</Project>
I get an 'Access to the path \\QAServer\Alpha is denied' message when this step runs. I assume this is because the build agent runs as a system account which can't access the QAServer.
I'm trying to avoid the overhead of using Web Deploy, but it looks like that would work for me if this isn't possible.
From Sayed I. Hashimi on twitter:
FileSystem relies on windows auth. Passing in password is ignored.
Your build user will need write access
That's what I thought. My solution is to publish to local staging directory first, then use the SMB Deployer from the TeamCity Deployer plugin to move the files across the network. The plugin allows you to specify credentials.