File Transform task fails to transform XML configurations on zipped package - azure

I'm working on Release pipeline, which will perform transformation on App Service Worker configs, then will publish workers + web application.
My input package is a zip package produced out of MsBuild publish (from ASP.NET build pipeline).
...\PackageTmp\app_data\jobs\triggered\BillingWorker\App.Prod.config
...\PackageTmp\app_data\jobs\triggered\BillingWorker\App.Test.config
...\PackageTmp\app_data\jobs\triggered\BillingWorker\BillingWorker.exe.config
...\PackageTmp\app_data\jobs\triggered\EtlWorker\App.Prod.config
...\PackageTmp\app_data\jobs\triggered\EtlWorker\App.Test.config
...\PackageTmp\app_data\jobs\triggered\EtlWorker\EtlWorker.exe.config
...\PackageTmp\Web.config
...\PackageTmp\Web.Test.config
...\PackageTmp\Web.Prod.config
...\PackageTmp\many other files
Transformation of Web.config is done correctly by Publish to Azure Web App task. However, workers configs aren't transformed automatically, so I added a File Transform task with following config:
This step doesn't work and here is the output:
2019-08-14T15:41:01.1435779Z ##[section]Starting: File Transform: config
2019-08-14T15:41:01.1576716Z ==============================================================================
2019-08-14T15:41:01.1576853Z Task : File transform
2019-08-14T15:41:01.1576932Z Description : Replace tokens with variable values in XML or JSON configuration files
2019-08-14T15:41:01.1576994Z Version : 1.156.0
2019-08-14T15:41:01.1600786Z Author : Microsoft Corporation
2019-08-14T15:41:01.1600885Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/file-transform
2019-08-14T15:41:01.1600986Z ==============================================================================
2019-08-14T15:41:01.6339900Z ##[warning]Unable to apply transformation for the given package. Verify the following.
2019-08-14T15:41:01.6351367Z ##[warning]Unable to apply transformation for the given package. Verify the following.
2019-08-14T15:41:01.8369297Z Initiated variable substitution in config file :
...
... many lines about variable subsitution
...
This output looks wrong, as it produces warning without declared explanation. How to workaround this warning?

The problem is that File Transform task strongly relies on names of both files - the one being transformed and the one containing transformation rules. Strict naming convention is required which can be described in following words:
A template named Name.xml can be transformed only by files named Name.Debug.xml, Name.Release.xml, and more general - Name.{anything-here}.xml.
What's happening here is that App.config file is renamed to {YourApplicationName}.exe.config during build thus the tranformation using App.Debug.config fails.
I see 2 workarounds:
1. Preserve the original name App.config
a. In a project file, set App.config file's property to Copy to output directory: Copy always
b. Setup "File Transform task" with args -transform *.Debug.config -xml *.config -result {YourApplicationName}.exe.config
c (optional). If you didn't specify -result in task, you need to setup another task to rename App.config to {YourApplicationName}.exe.config after the transformation has finished (for example a Command Line task with command copy App.config {YourApplicationName}.exe.config /Y)
2. Write custom transformator script
a. Unzip package into temp folder
b. Transform file using Powershell (make use of Microsoft.Web.XmlTransform.dll installed on agent)
c. Zip again and replace the original zip.

The native step in the official task doesn't support transformation in zip files. You can use another task for do it before the deploy task.
I used this and it worked fine to me:
https://marketplace.visualstudio.com/items?itemName=solidify-labs.vsts-task-tokenize-in-archive

Related

copy specific files from a container using wildcard filename in ADF pipeline

I'm trying to copy specific files from a container in a Storage Account using an ADF pipeline. lets say the container has the following files
aa_aaa_01_yyyymmdd.csv
aa_abb_01_yyyymmdd.csv
aa_aaa_02_yyyymmdd.csv
aa_aaa_03_yyyymmdd.csv
aa_abb_02_yyyymmdd.csv
ab_abc_01_yyyymmdd.csv
My pipeline has to copy all the files beginning with 'aa_aaa_'. I tried using the * wildcard at the time of creating the source dataset - like "aa_aaa_*.csv" but didn't work; the validation fails.
please help. Thanks
You can use prefix option like below in copy activity

Using ADF to get a subset of files from the directory in Azure File Share

For example my Azure file share directory contains the following files:
abc_YYYYMMDD.txt
def_YYYYMMDD.txt
ijk_YYYYMMDD.txt
I'm only interested to get abc_YYYYMMDD.txt and ijk_YYYYMMDD.txt
Currently, I have a Get Metadata activity that gets a list of files (childItems property) inside a File Share directory.
Then I have Filter activity that has this dynamic content:
#startswith(item().name, variables('filename_filter')) OR startswith(item().name,
variables('filename2filter')))
Unfortunately, it has an error:
Position 54 'startswith' is a primitive and doesn't support nested properties
How do I resolve this if I have multiple conditions inside the Dynamic Content for Filter activity?
Your Filter expression should be like this:
#or(startswith(item().name,variables('filename_filter')),startswith(item().name,variables('filename2filter'))
The expression doesn't support "or" directly, you should use or() function.
This is my source folder:
I create a pipeline, using parameters to filter the filename which start with "test1" and "test2":
Run the pipeline:
Output:
HTH.

How to set path of step Definition file , if my feature file ,stepDefinition file and runner file resides in separate folders?

I am using maven project for cucumber execution and below is my project explorer view.
Project Explorer
I want to instruct "SimpleDataDriverRunner.java" file to execute "SimpleDataDriverStepDefinition.java" class. Both these files are present in different folder structure (as shown in image above).
Below is my runner class
#RunWith(Cucumber.class)
#CucumberOptions(
features="dir path\\com\\features\\SimpleDataDriven.feature",
glue= {"stepDefinition"},
monochrome=true
)
Can someone please guide me how to achieve this and if this is correct way to deal with multiple runner file and step definition files ?
If there is any BDD cucumber sample project having multiple feature, step Definition and runner files is much appreciated .
I never understood why multiple runners are needed, if is just to run different scenarios it doesn't seem very efficient.
I would use just one runner and select what to run by tags.
Annotate your feature with some tag like #someTag and run it.
E.g.:
mvn test -Dcucumber.filter.tags="#someTag"
From cucumber documentation:
cucumber.features= # command separated paths to feature files. example: path/to/example.feature, path/to/other.feature
cucumber.glue= # comma separated package names. example: com.example.glue
Try adding in glue the package for runners also:
glue= {"stepDefinition", "runner"},

Locate .ipa with FAKE script after TeamCity build (Xamarin.iOS Ad-Hoc)

I am trying to setup a CI build for my Xamarin.iOS application using TeamCity and FAKE and upload the result (.ipa file) to HockeyApp. I am running in a small problem.
I managed to trigger the FAKE build script from TeamCity and also build my application successfully. The next step would be to call the "HockeyApp" method from the "HockeyAppHelper" module from FakeLib, but todo this I need the path to the .ipa file. All examples I find just hardcode the path (something like bin/iPhone/Release/*.ipa) however in my case the .ipa will by dropped in a folder containing the a timestamp (like bin/iPhone/Release/MyApp 2017-01-24 17-16-56/MyApp.ipa).
Question is how do I get hold of the .ipa file in the build script (because of the timestamp I cannot hardcode the path)?
Below is the section of the FAKE script responsible for building and uploading the .ipa:
Target "ios-adhoc" (fun () ->
RestorePackages "RidderCRM.iOS.sln"
UpdatePlist version versionNumber "RidderCRM.iOS"
iOSBuild (fun defaults ->
{defaults with
ProjectPath = "RidderCRM.iOS.sln"
Configuration = "Release"
Platform = "iPhone"
Target = "Build"
BuildIpa = true
Properties = [ "CodesignKey", "iPhone Distribution"; "CodesignProvision", "Automatic:AdHoc" ]
})
let appPath = Directory.EnumerateFiles(Path.Combine("RidderCRM.iOS", "bin", "iPhone", "Release"), "*.ipa").First()
TeamCityHelper.PublishArtifact appPath
HockeyApp (fun p ->
{p with
ApiToken = Environment.GetEnvironmentVariable("HockeyAppApiToken")
File = appPath
}) |> ignore
)
Seeing that this is more of an issue of what the actual MSBuild Task is doing in Xamarin.iOS.Common.targets, there are many things you can do.
You can edit the Xamarin.iOS.Common.targets file to remove the date time stamp. (Not so great)
You can use a custom Target which invokes the <Copy> Task. (Better)
You can customize the IpaPackageDir property to specify what the directory should be. (Best)
In short the problem of the TimeStamp comes directly from this element:
<IpaPackageDir Condition="'$(IpaPackageDir)' == ''">$(DeviceSpecificOutputPath)$(_AppBundleName) $([System.DateTime]::Now.ToString('yyyy-MM-dd HH-mm-ss'))</IpaPackageDir>
How would we do each one of these?
You would edit the code snippet above in your Xamarin.iOS.Common.targets file and remove the timestamp. It's usually a good idea to note edit .targets if you aren't familiar with them. Also updating Xamarin can override these.
You can add a <Target Name="AfterBuild"> definition with a simple copy task inside: <Copy SourceFiles="$(IpaPackagePath)" DestinationFolder="$(OutputPath)" />
You can edit the property of <IpaPackageDir> directly via:
<PropertyGroup>
<IpaPackageDir>$(OutputPath)</IpaPackageDir>
</PropertyGroup>
Note on #3:
A new MSBuild property IpaPackageDir has been added to make it easy to customize the .ipa file output location. If IpaPackageDir is set to a custom location, the .ipa file will be placed in that location instead of the default timestamped subdirectory.
https://developer.xamarin.com/releases/ios/xamarin.ios_9/xamarin.ios_9.8/#New_MSBuild_property_IpaPackageDir_to_customize_.ipa_output_location

Artefact folder structure does not contain empty directories

I'm trying to store whole the output of my build, this includes some empty folders. These aren't included by the artefact mechanism in teamcity:
What doesn't work:
OAR\=> OAR.zip
OAR->OAR.zip
OAR
Inside of OAR i have a folder structure that needs to be stored. I know i could put a placeholder file in each but that is not the answer i'm after. Otherwise ill have to zip it myself?
Unfortunately TeamCity, by design, searches for files and uploads them as artifacts which means that empty folders are never included. Given the open and very old issue in the TeamCity tracker I doubt they are going to fix it any time soon.
I would recommend zipping the folder yourself, that is the approach we have taken. How you implement that depends on the build technology you are using. For example, if you are building using Nant you could add the zip task to your build, there are similar options for MSBuild and Ant.
If you don't want to rely on the build performing the zip I would recommend installing 7zip on your build agents and using the command line to perform the zip. Just remember if you want 7zip to include empty directories use * as the wildcard rather than *. * like so:
7z a -r OAR.zip *
Technically you could use powershell to do the zipping, which would be better than having to install something on your agents. I haven't tried this option myself.
Apologies for not linking all my references above. Apparently, and understandably so, I need at least 10 reputation to post more than 2 links.

Resources