I have a project that is built with an azure pipeline, the artifacts are sent to both the File Share and Azure DevOps. A self hosted agent is currently used for the pipeline that builds my project. Now I want to deploy my artifacts on a physical computer in the network of my company. Artifacts are installation files that support "silent installation mode", so I can install everything by executing corresponding files with PowerShell.
My question is: What should I do to achieve my goal? I considered that I would simply create a release pipeline (I use Microsoft's classic UI editor for pipelines) and add the necessary tasks. In the end everything will look something like this:
Copy artifacts from the computer on which the product was built (A)
to the computer on which the product is to be installed (B).
Perform the installation of services that are required for the main
application.
Perform the installation from the main application. Is this the
right course of action or should I choose something else?
One more question. It is not entirely clear to me how I should divide the deployment stages, because: Step 1 is still done by the agent on the computer A. Steps 2 and 3 should already do another agent. Hence, an agent must be also present on the computer B (in total I need two agents, one for the build pipeline on the computer A and one on the target computer B). Is that correct?
Deploy artifacts to physical machine with Azure Pipelines
I think you are very close to the answer, but I am not sure whether the advice I provided is the best, because it is more a matter of taste, you can check my suggestions below.
We could achieve this by one agent on the computer A:
Use the Remote Copy task to copy the artifact to the computer B by the agent on the computer A.
Use the Powershell execute install files with command line arguments on remote computer the agent on the computer A.
In this case, we do not need to create a new agent on the computer B, and also no need to divide the deployment stages.
We have a web application that has a similar requirement, we need to include location specific files when we deploy our application to that location.
I'm not sure this is the best route, but we incorporated using Ansible in our deployment pipeline. Our Ansible playbook utilizes the hosts file which contains variables for the location and the jar version to be deployed.
Again, our situation is different, but the same should work.
For instance, you may be able to something like this...
Your Ansible inventory (hosts) file may look like this
[servers]
xxx.xxx.xxx.xxx
[all:vars]
app_name=mfgweb
env=qa
app_version=2020.03.4
*** If your artifact is a jar file
jar_group=com.somegroup
jar_name=some_jar <-- note: no extentsion
jar_version=2019.11.3
Your playbook may look something like this
---
- hosts: localhost
connection: local
tasks:
- name: Find old working directories
find:
paths: /tmp
patterns: 'ansible.*'
file_type: directory
age: 2d
register: tmp_dirs
- name: Cleanup old working directories
file:
path: "{{ item }}"
state: absent
loop: "{{ tmp_dirs.files | map(attribute='path') | list }}"
- name: Create working directory
tempfile:
state: directory
suffix: work
register: workdir
- name: Download artifact
maven_artifact:
group_id: "{{ jar_group }}"
artifact_id: "{{ jar_name }}"
version: "{{ jar_version }}"
repository_url: "{{ artifacts_url }}"
username: "{{ artifacts_user }}"
password: "{{ artifacts_password }}"
verify_checksum: never
dest: "{{ workdir.path }}/{{ app_name }}/"
This is just a sample, but you would be able to install required software in you playbook using yum and/or pip.
Related
I'm doing a zip deploy of a .NET Framework web app to an Azure App Service via a GitHub workflow.
I have set WEBSITE_RUN_FROM_PACKAGE to 1 in the Azure console's Settings / Configuration / Application settings page. I've also tried setting WEBSITE_RUN_FROM_ZIP to 1 there just in case (although I think this is an obsolete flag).
The package is building correctly in GitHub and I can see it showing up in my Kudu debug console, under C:\home\site\wwwroot (as MyPackageName.zip) as well as in C:\home\data\SitePackages (as 20220512205318.zip, for example).
The deploy portion of my YAML is:
deploy:
runs-on: windows-latest
needs: build
environment:
name: 'Test'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: ASP-app
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: ${{ env.AZURE_WEBAPP_NAME }}
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_XYZsecret }}
package: .
And the .PublishSettings I've uploaded to GitHub looks like:
<publishData>
<!-- Which one of these 3 profiles is my YAML using? I don't actually know. -->
<publishProfile profileName="mywebappname-test - Web Deploy" publishMethod="MSDeploy" etc="foobar">
<databases/>
</publishProfile>
<publishProfile profileName="mywebappname-test - FTP" publishMethod="FTP" etc="foobar">
<databases/>
</publishProfile>
<publishProfile profileName="mywebappname-test - Zip Deploy" publishMethod="ZipDeploy" etc="foobar">
<databases/>
</publishProfile>
</publishData>
The zip package is not getting automatically unpacked. The MSFT support rep I talked to suggested that this was the problem, and indeed when I download the package to my machine and drop it into Kudu's Tools/Zip Push Deploy page, I see that the package is unpacked, and I can get the site to work by setting the appropriate Physical Path to match the '/' Virtual path. Specifically the Kudu Tools Zip Push causes my web.config and favicon.ico etc. files to show up in:
C:\home\site\wwwroot\Content\D_C\a\foo\bar\good\boy\obj\Test\Package\PackageTmp
and I can go to the Azure console for my app service, navigate to Settings / Configuration/ Path Mappings, Virtual applications and directories, and edit the existing entry to:
Virtual path: /
Physical Path: site\wwwroot\Content\D_C\a\foo\bar\good\boy\obj\Test\Package\PackageTmp
Type: Application
and then see my site come up in a browser.
However, when don't do anything to unpack the archive, and I leave the entry as:
Virtual path: /
Physical Path: site\wwwroot
Type: Application
I can't see my site in a browser and instead just see "You do not have permission to view this directory or page." When I then dig in to the logs in Kudo, I see 403.14 - Forbidden errors on my main site and a 404.0 - Not Found error on C:\home\site\wwwroot\favicon.ico. (Like the rest of my files, favicon.ico is still inside the zip archive at [...]\foo\bar\good\boy\obj\Test\Package\PackageTmp\favicon.ico.)
My questions are:
Should my web app be able to run at all with just my zip file sitting there as C:\home\site\wwwroot\MyPackageName.zip? Or does it really need to be unpacked as the MSFT rep indicated?
If it is supposed to run this way, any ideas on what am I missing? I assume it's something in my YAML (which of the 3 publishProfile settings is it actually choosing here?) or in Settings / Configuration/ Path Mappings or Application settings, but I have no idea what at this point and I'm running out of ideas.
Thanks, Eric
Should my web app be able to run at all with just my zip file sitting there as C:\home\site\wwwroot\MyPackageName.zip?
Pretty much, yes, just not in wwwroot. When WEBSITE_RUN_FROM_PACKAGE is enabled, the application is run from the archive directly as a read only directory mount. Nothing is copied to wwwwroot or anywhere else.
If it is supposed to run this way, any ideas on what am I missing?
My understanding is package deploy from GitHub is not supported or rather GitHub archives are incompatible with run from package on App Service.
dwellman's response was the correct answer to my original question, but I'll add some more detail here on how I used this information to get my deployment to work properly. I feel like the inability to read the zip archive's internal index XML file(s) to find the correct relative path is an Azure flaw, but until it's addressed, I hope others may find this useful.
My first step was to abandon the idea of deploying as a zip file. It's possible I could have still made this work by doing some post-processing to zip things up in a different format without the nested folders, but I decided in my case that the benefits of a single-file deployment weren't worth the cost. To stop deploying as a zip file, I manually edited the .pubxml file I was passing in as msbuild option /p:PublishProfile=AzureCI.pubxml. The changes I made were to change PackageAsSingleFile from true to false, and change DesktopBuildPackageLocation from a zip file path to a folder path.
This alone was enough to get my site to get deployed to Azure as individual files instead of a zip archive. The files were still buried in an ugly folder structure, but I could at least see them in Kudu and get the site to work by applying the same Settings / Configuration/ Path Mappings, Virtual applications and directories adjustment I describe in my original question.
I could have stopped there, but I wanted to be able to just use the default virtual path and not have my Azure configuration be so dependent on my upstream processes. In other words, I wanted to just have my web.config and favicon.ico etc land directly in C:\home\site\wwwroot instead of deep in the weeds of a subfolder structure. To make this work, I changed the package argument to webapps-deploy in my YAML from . to the appropriate path as follows:
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: ${{ env.AZURE_WEBAPP_NAME }}
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_XYZsecret }}
package: .\Archive\Content\D_C\a\foo\bar\good\boy\obj\Test\Package\PackageTmp
This caused the deployment process to pick off just the files I needed from the build and drop them into C:\home\site\wwwroot. I could then revert the path mapping kludge and be on my way.
Need to handle YUM package installation deployement process with different versions/packages, for target environments(dev/prod/systest) using ansible playbook.
NOTE: I have gone through groups_var and hosts_var concept but did not understand if multiple packages with different versions can handled for deployment in multiple environment based on input
As you found out, this separation can be achieved by using group_vars and host_vars. These are loaded in relation to the path of inventory file.
Simple example tasks like below will install different versions in dev and prod environments as explained below.
Example playbook1.yml:
- hosts: appservers
tasks:
- name: install app-a
yum:
name: 'app-a-{{ app_a_version }}'
- name: install app-b
yum:
name: 'app-b-{{ app_b_version }}'
Consider the example directory structure separating each environment's inventory:
dev/hosts
prod/hosts
systest/hosts
Each inventory file will contain hosts/groups for that environment.
Dev environment:
Example dev/hosts:
[appservers]
appserver1.dev
appserver2.dev
Then we can have variables specific to this environments in dev/group_vars/appservers.yml:
---
app_a_version: 1.1
app_b_version: 5.5
Will install app-a-1.1 and app-b-5.5 when run as:
ansible-playbook playbook1.yml -i dev/hosts
Prod environment:
Example prod/hosts:
[appservers]
appserver1.prod
appserver2.prod
And variables defined in prod/group_vars/appservers.yml:
app_a_version: 1.0
app_b_version: 5.0
But in prod it will install app-a-1.0 and app-b-5.0 when run as:
ansible-playbook playbook1.yml -i prod/hosts
host_vars work in similar way, and can be used to provide variables specific to each host of the inventory rather than groups in inventory.
I wanted to deploy multiple VMS from my ansible-playbook, so I used split function and I am getting the error of dict does not have user_inout error. Please have a look at my code.
Code:
- name: os system
pause:
prompt: |
Which os do you want to use?
1- Windows Server
2- CentOS_7
3- CentOs_8
4- Ubuntu
5- Others
register: os_system
- set_fact:
o_name: "{{ os_system.user_input.split(',') }}"
- name: Domain Decision
pause:
prompt: Do you want your PC in Domain
register: decision
when: 'item|string == "1"'
with_items:
- "{{ o_name }}"
- set_fact:
dec: "{{ decision.user_input.split(',') }}"
Now my real issue, if the user chooses option 1,2 then it will be split by set_fact for os_system and according to that input it will decide for the domain decision my main issue is that while the task is at set_fact for the decision, it will give me the error like this:
Error:
fatal: [x.x.x.x]: FAILED! => {
"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'user_input'\n\nThe error appears to be in '/home/x.x.x.x/sites/playbook.yml': line 82, column 6, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - set_fact:\n ^ here\n"
}
IF I remove the loop from task domain decision, then set_fact works very perfectly.
When you want to collect user input, it can be in two ways:
Interactive (prompted), using vars_prompt.
Through variables (non-prompted). For example, you could have users create a file with variables and load with vars_file.
I would prefer the second approach if possible.
However, to get user input interactively through prompts:
- hosts: my_hosts
vars_prompt:
- name: os_system
prompt: |
Which os do you want to use?
1- Windows Server
2- CentOS_7
3- CentOs_8
4- Ubuntu
5- Others
private: no
tasks:
- set_fact:
o_name: "{{ os_system.split(',') }}"
You should note that by splitting the input of 1,2 - you will not get OS name. You will again have to set a fact based on the number (1 = Windows Server). IMHO this is unnecessary complication.
A much better option would be to have users create a variables file like below:
my_vars.yml:
os_choices:
- { name: Windows Server, domain: yes }
- { name: CentOS_7, domain: no }
And load it in play using:
- hosts: my_hosts
vars_file:
- my_vars.yml
So we are currently moving away from our current deployment provider: Beanstalk, which is great but we are on the top tier and we keep running out of space or hitting our repository limits. So we are moving away so please do not suggest any other SaaS provider.
I personally use Gitlab for my own projects and a few company projects and it's amazing we use a self hosted version on our local server in our company building.
We have CI setup and currently are using the following deployment code (I have minified the bits just to the deployment for development) - this uses the shell executer for deploying as we deploy to an existing linux server.
variables:
HOSTNAME: '<hostname>'
USERNAME: '<username>'
PASSWORD: '<password>'
PATH_DEV: '/path/to/www'
# Define the stages (we can add as many as we want)
stages:
# - build
- deploy
# The code for development deployment
deploy_dev:
stage: deploy
script:
- echo "Deploying to development environment..."
- rm .gitlab-ci.yml
- rsync -urltvz --filter=':- .gitignore' --exclude=".git" -e "sshpass -p"$PASSWORD" ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" * $USERNAME#$HOSTNAME:$PATH_DEV
- echo "Finished deploying."
environment:
name: Development
url: http://dev.domain.com
only:
- envdev
The Problem:
When we use the above code to deploy it's perfect and works really well, and it deploys all the code after optimisation etc, but we have found a little bug here.
When you delete a file then the rsync command will not delete the file, now I did some searching and found the --remove flag you can add, and it worked - but it deleted all the user uploaded content as well. Now I added the .gitignore in to the filtering, so it would ignore some the files in their (which are usually user generated) or configuration files or/and libraries (npm, etc.). This is fine until a user started uploading files using the media manager in our framework which stores in a folder that is not in the .gitignore file and it can't because it contains other files, as we also add our own files in there so they're editable by the user, so now I am unsure how to manage this.
What we are looking for is a CI setup, which will upload file changes to the server, so it would search through the latest commits, and find the latest files that have been changed and then push only them files up. Of course I would like to do this with the Gitlab CI still, so any ideas examples or tutorials would be amazing.
Thanks in advance.
~ Danny
May it helps: https://github.com/banago/PHPloy
Looks this tool designed for php project, but I think it can use other web deployment.
how it works:
PHPloy stores a file called .revision on your server. This file contains the hash of the commit that you have deployed to that server. When you run phploy, it downloads that file and compares the commit reference in it with the commit you are trying to deploy to find out which files to upload. PHPloy also stores a .revision file for each submodule in your repository.
New to Ansible I'm experimenting with setting up a website under IIS.
I can create and configure an application pool, but I'm struggling with the website. The basic site works, HTTPS/SSL is still troublesome, but I read there are some bugs in the win_iis_website/win_iis_webbinding scripts that are being worked on. The part I'm stuck with are IIS' features per site.
In IIS (in the GUI) there are sub-features that can be configured for a site:
I was unable to find how to configure these using Ansible (more specifically Ansible's win_iis_website module).
I'm looking to configure ASP, Handler mappings, URL rewrites and Default documents.
Is there any way to do so?
My current yml for creating the site looks like this:
- name: create new website {{ websitename}}
win_iis_website:
name: "{{ websitename}}"
state: started
port: 443
ip: *
ssl: true
hostname: "{{ websitename }}"
application_pool: "{{ websitename }}"
physical_path: c:\inetpub\wwwroot\{{ websitename }}
parameters: logfile.directory:c:\inetpub\logs\
register: website
I am currently making Playbooks for IIS and indeed to perform the configuration there is no particular module that allows you to modify the functions of the sections, I looked in some places and the information was very scarce, there are modules for applicationPool, but for this you have to use win_shell as follows
- name: Name of playbook
win_shell: |
<PowerShell command>
You can base on the CIS BENCHMARK guide of IIS.
Check the win_feature module:
- name: Install IIS Web-Server with sub features and management tools
win_feature:
name: Web-Server
state: present
restart: True
include_sub_features: True
include_management_tools: True
if you want to do in a more controlled manner, check the installed features with the command:
Get-WindowsFeature
And add like:
- name: Install IIS
win_feature:
name: "Web-Filtering,Web-Dir-Browsing,Web-Default-Doc"
state: present
restart: no
include_sub_features: no
include_management_tools: yes