Can't package Azure DevOps Extension, cause a dependency includes a file with spaces in the name - azure-pipelines-build-task

After updating QueryAzureDevOpsExtensionVersion#3 to version 4 and the same for PackageAzureDevOpsExtension, I started getting errors in PackageAzureDevOpsExtension.
All errors looked something like that:
error: Error: Part Name 'Myproject/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/openssl/OpenSSL License.txt' is invalid. Please check the following: 0 [
"error: Error: Part Name 'Myproject/node_modules/azure-pipelines-tasks-azure-arm-rest-v2/openssl/OpenSSL License.txt' is invalid. Please check the following: ",
'error: 1. No whitespace or any of these characters: #^[]<>?',
'error: 2. Cannot end with a period.',
'error: 3. No percent-encoded / or \\ characters. Additionally, % must be followed by two hex characters.',
''
That part of the pipeline now looks like this:
- task: QueryAzureDevOpsExtensionVersion#4
name: QueryVersion
displayName: 'Query Extension Version'
inputs:
connectTo: 'VsTeam'
connectedServiceName: 'Clipper-Marketplace-Admin'
publisherId: '$(publisherId)'
extensionId: '$(extensionId)'
versionAction: ${{ parameters.updateKind }}
- task: PackageAzureDevOpsExtension#4
inputs:
rootFolder: '$(Build.SourcesDirectory)/AzurePipelinesTasks'
patternManifest: 'my-project-vss-extension.json'
publisherId: '$(publisherId)'
extensionId: '$(extensionId)'
extensionName: '$(extensionName)'
extensionVersion: '$(QueryVersion.Extension.Version)'
updateTasksVersion: true
updateTasksVersionType: ${{ parameters.updateKind }}
extensionVisibility: 'private' # Change to public if you're publishing to the marketplace
extensionPricing: 'free'
What can cause these errors in the pipeline? The reason why I'm updating the pipeline is because there's some issue with the latest typescript version which causes an issue with compiling the code. After updating packages, I started failing in the package stage because of errors like the above.

It looks like azure-pipelines-tasks-azure-arm-rest-v2 was updated to v3 in the past week. I just compared #^2 with #^3 and it indeed now includes a file with spaces in the name:
\node_modules\azure-pipelines-tasks-azure-arm-rest-v2\openssl
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 05/12/2022 15:27 2318 HashInfo.txt
-a--- 05/12/2022 15:27 2267136 libeay32.dll
-a--- 05/12/2022 15:27 6404 OpenSSL License.txt
-a--- 05/12/2022 15:27 548352 openssl.exe
-a--- 05/12/2022 15:27 2722 ReadMe.txt
-a--- 05/12/2022 15:27 385536 ssleay32.dll
I've personally solved that in 2 ways in the past.
Delete the offending files
Rename the offending file, replacing with _.
You have a 3rd option:
Downgrade to azure-pipelines-tasks-azure-arm-rest-v2#^2
I don't know what the limitation is in this case, I've just worked around it so far. Recommend commenting on the bug in the tfx repo.
Until then, add a pwsh script prior to running package or publish to remove the spaces:
dir -recurse | where {-Not $_.PsIscontainer -AND $_.name -match " "} | foreach {
$New=$_.name.Replace(" ","_")
Rename-Item -path $_.Fullname -newname $New -passthru
}
You can also comment on the bug with the repo that hosts azure-pipelines-tasks-azure-arm-rest-v2.

Related

Blazor WebAssembly Application fails to load due to integrity errors

We have developed a Blazor WebAssembly Application that has already gone into productive usage for a certain group of customers.
The Application works well in all Browsers with Standard Security settings. However, this morning I got a call from one of the customers, where the Application did not load at all in their Chrome Browser.
I saw the following Errors in the console:
Unknown error occurred while trying to verify integrity.
Failed to load resource: the server responded with a status of 403 (forbidden)
Failed to find a valid digest in the 'integrity' attribute for ressource '<somepath.dll>' with SHA-256 integrity <sha56>. the resource has been blocked
Now my question is, what could cause this? Is this a Browser Security setting, or another security setting e.g on server, in code etc.? How can I fix this?
Here's a picture of the errors mentioned above
The most likely reason why this is happening, is that some Antiviruses block the execution of downloaded .dll files. That's why it is working in some networks, but doesn't in some others.
What you can do, and what is also suggested as a Workaround by microsoft, is to rename all .dll to .bin - and also change the config json. it worked in my case.
I use the following PowerShell function for that:
Function Hide-BlazorDLL {
Param(
[string]$Path = (Get-Location).Path
)
<#
According to the following Links:
https://github.com/dotnet/aspnetcore/issues/19552
https://github.com/dotnet/aspnetcore/issues/5477#issuecomment-599148931
https://github.com/dotnet/aspnetcore/issues/21489
https://gist.github.com/Swimburger/774ca2b63bad4a16eb2fa23b47297e71
#>
# Test if path is correct and accessible
$WorkingDir = Join-Path $Path "_framework"
if (!(Test-Path $WorkingDir)) { Throw "Wrong path $Path. current location must be wwwroot folder of published application." }
# Get All Items
$AllItems = Get-ChildItem $WorkingDir -Recurse
$DLLs = $AllItems | Where-Object { $_.Name -like '*.dll*' }
$BINs = $AllItems | Where-Object { $_.Name -like '*.bin*' }
# End script if no .dll are found
if ($DLLs) {
# Delete all current .bin files
if ($BINs) {
Remove-item $BINs.FullName -Force
}
# Change .dll to .bin on files and config
$DLLs | Rename-item -NewName { $_.Name -replace ".dll\b",".bin" }
((Get-Content "$WorkingDir\blazor.boot.json" -Raw) -replace '.dll"','.bin"') | Set-Content "$WorkingDir\blazor.boot.json"
# Delete Compressed Blazor files
if (Test-Path "$WorkingDir\blazor.boot.json.gz") {
Remove-Item "$WorkingDir\blazor.boot.json.gz"
}
if (Test-Path "$WorkingDir\blazor.boot.json.br") {
Remove-Item "$WorkingDir\blazor.boot.json.br"
}
# Do the same for ServiceWorker, if it exists
$ServiceWorker = Get-Item "$Path\service-worker-assets.js" -ErrorAction SilentlyContinue
if ($ServiceWorker) {
((Get-Content $ServiceWorker.FullName -Raw) -replace '.dll"','.bin"') | Set-Content $ServiceWorker.FullName
Remove-Item ($ServiceWorker.FullName + ".gz")
Remove-Item ($ServiceWorker.FullName + ".br")
}
}
else {
Write-Host "There are no .dll Files to rename to .bin"
}
}
Basically you need to navigate to the wwwroot folder of your published application and run the function there. e.g:
PS D:\inetpub\wwwroot\<appname>\wwwroot> Hide-BlazorDLL
Solution for me was to delete the obj and the bin folder in both the client and the server project folder
This error for some reason kept happening for me when I tested my application in an anonymous browser window (Google Chrome).
Try using a normal browser window if you're getting integrity errors.
Also, if you're using Cloudflare CDN don't forget to "Purge Everything" in the cache.
We have experienced this issue using Cloudflare auto minify feature. That feature removes any comments from html, js and other files - which some of the blazor .js files seems to contain.
This means that the hash of the file contents no longer matches the hash found in blazor.boot.json -> an integrity issue will be thrown and stop the app from loading.
Disabling the auto minify feature fixed the issue.
Tech Stack:
.NET 6.0.11
I had a similar issue. In the local machine, it is working fine. But when it is deployed through GitHub Actions, I get integrity checks error. I got this issue for Blazor WebAssembly ASP.NET Core Hosted (WebAssemblyPrerendered) project. Here is the fix I followed.
Added the .gitattributes file to the solution root folder.
Added the below code at the end of the file.
# blazor dlls - treat all .dll files as binary
*.dll binary

eyaml is not decrypted on Puppet agent

We running a Puppet master server on CentOS 7.6 in combination with the repo from puppet (http://yum.puppetlabs.com/puppet6/el/7)
When I write am eyaml where I define a variable, I just see the encrypted value on the agent (Windows 2016).
On the Puppet master, I can edit the eyaml, everthing looks fine. The only difference to a standard installation is, that everything is in a different environment 'myenv'.
hiera.yaml:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
### Encrypted eyaml files
- name: "Secret data: per-node, per-datacenter, common"
lookup_key: eyaml_lookup_key
path: "/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc/%{::trusted.certname}.eyaml"
options:
pkcs7_private_key: /etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem
pkcs7_public_key: /etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem
# Environments
- name: "env2"
glob: "env/test/*/%{::trusted.certname}.yaml"
- name: "env1"
glob: "env/test/%{::trusted.certname}.yaml"
# Components
- name: "cmp2"
glob: "cmp/test/*/%{::trusted.certname}.yaml"
- name: "cmp1"
glob: "cmp/test/%{::trusted.certname}.yaml"
# Others
- name: "Other YAML hierarchy levels"
paths:
- "common.yaml"
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ puppetserver -v
puppetserver version: 6.3.0
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ hiera -v
3.5.0
[ root #pup-mst-srv-10:/etc/puppetlabs/puppet]-$ eyaml version
[hiera-eyaml-core] hiera-eyaml (core): 3.0.0
[ root #pup-mst-srv-10:/etc/yum.repos.d]-$ puppetserver gem list -e hiera-eyaml
*** LOCAL GEMS ***
hiera-eyaml (3.0.0)
[ root #pup-mst-srv-10:/etc/eyaml]-$ cat config.yaml
pkcs7_private_key: '/etc/puppetlabs/puppet/eyaml/private_key.pkcs7.pem'
pkcs7_public_key: '/etc/puppetlabs/puppet/eyaml/public_key.pkcs7.pem'
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ ls -l /etc/puppetlabs/puppet/eyaml/
total 8
-r--r--r--. 1 puppet puppet 1679 Jul 11 15:39 private_key.pkcs7.pem
-r--r--r--. 1 puppet puppet 1050 Jul 11 15:39 public_key.pkcs7.pem
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ eyaml edit rdc.eyaml
---
classes:
- win_ad_abcd
win_ad_abcd::testpassword : DEC(1)::PKCS7[test12]!
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/data/cmp/test/rdc]-$ cat rdc.eyaml
---
classes:
- win_ad_abcd
win_ad_abcd::testpassword : ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAS0E/Y3+QzFhRVZM+F+5kQ8ZQrvGddUno5sDeg3Np9P1/8I5Xetemrx5DTKQaD5C4DS3kgvxjrSqVk/GCCMtZUW5Ynlym1yvylHA7zXmn+g6pYbe5XW88y2Xv1IzdxHwPmgOlFAXJCRoieTrfph+Y4mQBWi2uyrTphHM/o31JcDREfzOeucTSGaHnq8SHeP7t5O7w5ZFG4++hasBLUTubG2ZOAgQRTlksmTK3oOJ0eLRDab4LpgBMaL/VaZgFiu3qmMb3IPtHlaSEAiTRQzdJW7WeHTJUqPSBNni1WmPXA3lFqmp8PFomxsLBTv7i9/gw7SQ2FHwpu5izH6iKwzmEcjA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBRpizv6doUY5DzpFaBg45lgBCJeK3Yi9qSUCulkHzBDzx6]
[ root #pup-mst-srv-10:/etc/puppetlabs/code/environments/myenv/modules/win_ad_abcd/manifests]-$ cat init.pp
class win_ad_abcd (
$testpassword = "Not Set",
)
{
notify{"eyaml --> ${testpassword} <--":}
file { 'C:\Windows\Temp\test.out':
content => $testpassword,
}
}
I expect an output in the test.out file on the agent with "test12", but I got always the encrypted output.
Would be great to have some hints.
Ivo
I spent a bit of time unsuccessfully trying to reproduce this. On the basis of this information, I also can't think of any reason why this would occur.
On attempting to create a minimal, complete example, I simplified the setup, and you could try using something similar to my setup to see if you can still reproduce the issue. Then, add in additional complexity similar to your original problem until you find the minimal, complete, verifiable example. We might be able to help further if you do this.
Simpler hiera.yaml:
---
version: 5
defaults:
datadir: data
data_hash: yaml_data
hierarchy:
- name: "Secret data: per-node, per-datacenter, common"
lookup_key: eyaml_lookup_key
path: "encrypted.yaml"
options:
pkcs7_private_key: ./keys/private_key.pkcs7.pem
pkcs7_public_key: ./keys/public_key.pkcs7.pem
spec/fixtures/hiera/data/encrypted.yaml:
---
test: >
ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEw
DQYJKoZIhvcNAQEBBQAEggEAedQX8FExcat6yk0zsUzNbzQ/07w8ghPOw4eY
ycrfz0H7Cr7KnuBMY0yloFmtWuhYcjXETfaU3U3zGr9IOl4Aiy7yD3ZIvH0Y
HoEWKiJeUzNGrpaH/QFk378cEbpd6LXG46nMzw6w21uhASmvVt3KmZBJwY29
sEk2MpZm32H4JxQQosns4SDMQ6tA5h1xSrgpBTKd1x5vKSTsNnLAahjW31aH
JbK7Se+hHJ4zi9P0/ZjT07OTq1X2rwnfNK8wgKJa/VEDSH+KoNub+4TDHfj/
CWyGQx3Y5U1J2R2/6P5Vp2zRaAf/0BT43Ud/M8H25BIjYosuGtDVCVbbxMNK
mXZITDA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBDEPKno3R1K0XNat4a9
uCuDgBAdp579qk9MbgWLXyXSBD80]
Created keys:
▶ eyaml createkeys
▶ ls keys
private_key.pkcs7.pem public_key.pkcs7.pem
Create a secret:
▶ eyaml encrypt -l 'test' -s 'mySecretString' >> \
spec/fixtures/hiera/data/encrypted.yaml
Which I then cleaned up by further editing that file manually to get the content shown above.
I tested using lookup:
▶ puppet lookup \
--hiera_config=spec/fixtures/hiera/hiera.yaml test
--- mySecretString
Theories:
The information above doesn't truly represent your setup.
You could have actually encrypted the encrypted string?
Things I ruled out:
An error would be seen if there was some problem accessing your eyaml keys. Or if the eyaml library wasn't available. The fact that no error is seen suggests the keys are ok, the eyaml_lookup_key function is found, and eyaml is working basically.
Otherwise, see if you can boil this down to a truly minimal, verifiable, complete example.
Thanks Alex, I create a new VM with puppet and set them up like you described. This helped me to find a stupid mistake.
The eyaml with the encrypted password was rdc.eyaml and I had a symlink to this file like myHostname.yaml
So: myHostname.yaml -> rdc.eyaml
After moving myHostname.yaml to myHostname.eyaml everything works fine.

Set-AzureRmDataFactoryV2IntegrationRuntime : Method not found

When I am trying "Set-AzureRmDataFactoryV2IntegrationRuntime" command constantly getting below error.
Set-AzureRmDataFactoryV2IntegrationRuntime : Method not found:
'Newtonsoft.Json.Serialization.IAttributeProvider
Newtonsoft.Json.Serialization.JsonProperty.get_AttributeProvider()'.
At C:\Users\ravi\source\repos\test.ps1:12 char:5
+ Set-AzureRmDataFactoryV2IntegrationRuntime `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : CloseError: (:) [Set-AzureRmData...egrationRuntime], MissingMethodException
+ FullyQualifiedErrorId : Microsoft.Azure.Commands.DataFactoryV2.SetAzureDataFactoryIntegrationRuntimeCommand
When I ran "Get-Module" command I see below list of modules
ModuleType Version Name
ExportedCommands
---------- ------- ---- ----------------
Script 0.5.10 AzureRM.DataFactoryV2
{Get-AzureRmDataFactoryV2, Get-AzureRmDataFactoryV2ActivityRun,
Get-AzureRmDat...
Script 5.6.0 AzureRM.profile
{Add-AzureRmEnvironment, Clear-AzureRmContext, Clear-AzureRmDefault,
Connect-A...
Manifest 3.1.0.0 Microsoft.PowerShell.Management
{Add-Computer, Add-Content, Checkpoint-Computer, Clear-Content...}
Manifest 3.0.0.0 Microsoft.PowerShell.Security
{ConvertFrom-SecureString, ConvertTo-SecureString, Get-Acl,
Get-AuthenticodeSi...
Manifest 3.1.0.0 Microsoft.PowerShell.Utility {Add-Member,
Add-Type, Clear-Variable, Compare-Object...}
Manifest 3.0.0.0 Microsoft.WSMan.Management
{Connect-WSMan, Disable-WSManCredSSP, Disconnect-WSMan,
Enable-WSManCredSSP...}
Script 0.2.0 PowerShellEditorServices.Commands
{ConvertFrom-ScriptExtent, ConvertTo-ScriptExtent, Find-Ast,
Get-Token...}
Script 0.2.0 PowerShellEditorServices.VSCode
{Close-VSCodeHtmlContentView, New-VSCodeHtmlContentView,
Set-VSCodeHtmlContent...
I have installed Azure PowerShell MSI link given on below page.
Page URL: https://learn.microsoft.com/en-us/powershell/azure/other-install?view=azurermps-6.9.0
MSI URL: https://github.com/Azure/azure-powershell/releases/download/v6.9.0-September2018/azure-powershell.6.9.0.msi
Update:
When I was trying to create IS manually I see that its created but with "Unavailable" status
Updated on 10/10 for Joy Wang
I test the Set-AzureRmDataFactoryV2IntegrationRuntime on my side, it works fine. I recommend you to remove the modules and install again, refer to this link.
My specific command:
Set-AzureRmDataFactoryV2IntegrationRuntime -ResourceGroupName 'joywebapp' -DataFactoryName 'joyfactoryv2' -Name 'integrationRuntime35' -Type "SelfHosted" -Description 'New description111'
My module :
When I was trying to create IS manually I see that its created but with "Unavailable" status
To fix this issue of Self-Hosted type runtime, you could install the Microsoft Integration Runtime Configuration Manager, in the runtime -> Edit -> Click here to launch the express setup for this computer, refer to the screenshot.
After installing it, you will see them like the screenshot.
Check in the portal:

How to resolve NAnt Error creating FileSet in TeamCity?

I'm using TeamCity to build and deploy into our demo site. We have one configuration called HTML Demo Site and one of the build step is using NAnt to deploy the HTML to the site.
The build file have defined a target:
<target name="deploy-html" description="Deploys the HTML to the demo server">
<echo message="Deploying HTML to the demo server..."/>
<copy todir="\\<server>\<dir>\<client>" includeemptydirs="true" overwrite="true">
<fileset basedir="..\html\_master">
<include name="**\*"/>
<exclude name="node_modules\**"/>
</fileset>
</copy>
</target>
Each time I run the build on TeamCity, it's failing with this error:
C:\tc\w\9149e011dfa8657d\build_scripts\website.build(27,14):
[NAnt output] Error creating FileSet.
[NAnt output] The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
So I tried running on PowerShell to get a list of files that exceed the max length:
Get-ChildItem -Recurse | Where-Object {$_.FullName.Length -gt 248}
But the only files returned are files under the node_modules directory. But in the build file, it's being excluded. So I'm not sure where else to look? Any ideas?
You could try a few things:
Delete the node_modules dir first
Use robocopy /mir in an <exec> task
try putting exclude first before include (not likely, but worth a try)
try changing the path expression to name="node_modules\**\*" or name="**\node_modules\**" or similar
Deleting first worked for me - but the built in nant delete task also has problems so I had to use the rmdir console command
<exec program="${environment::get-variable('WinDir')}\system32\cmd">
<arg value="/c "rmdir /q /s ${Build.BuildFolder}\WebApplication\node_modules"" />
</exec>

Powershell and System.Security.Cryptography.X509Certificates.X509Certificate2

i'm getting this error when i run the system.security namespace. This is what i am running after
$cert=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("C:\mycert.cer")
New-Object: Cannot find type [System.Security.Cryptography.X509Certificates.X509Certificate2("C:\mycert.cer")]: make sure the assembly containing this type is loaded.
At line:1 char:19
+ $cert = New-Object <<<<
+ CategoryInfo : InvalidType: (:) [New-Object], PSArgumentException
+ FullyQualifiedErrorId : TypeNotFound,Microsoft.PowerShell.Commands.NewObjectCommand**
What am i doing wrong?
Try running this to see if you have the System.dll loaded (should be by default):
[AppDomain]::CurrentDomain.GetAssemblies() |
Where {$_.Location -match '\\System\\'}
If it is loaded then this command should show the X509Certificate2 type:
[AppDomain]::CurrentDomain.GetAssemblies() |
Where {$_.Location -match '\\System\\'} |
%{$_.GetExportedTypes()} | Where {$_.Name -match 'X509Cert'}
If the System.dll isn't loaded (which would be odd) try loading it:
Add-Type -AssemblyName System
See: http://technet.microsoft.com/en-us/library/hh849914.aspx
FYI ... I got error:
Unable to find type [System.Security.Cryptography.x509Certificates.X509Certificate2UI]
when using:
$certSelect = [System.Security.Cryptography.x509Certificates.X509Certificate2UI]::SelectFromCollection($certCollection, $title, $msg, 0)
However, I had no error creating the collection earlier on in my script:
$certCollection = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2Collection
To make the error go away I had to include the following at some point earlier on:
Add-Type -AssemblyName System.Security
I've solved my problem. It's easily:
cd\
$cert=New-Object System.Security.Cryptography.X509Certificates.X509Certificate2("C:\mycert.cer")
cd\ is necessary
I ran into this in the ISE (but seems to apply to the normal command window too) and it seems that using autocomplete will automatically Add-Type for whatever you're looking for. If you start a new instance and run:
[AppDomain]::CurrentDomain.GetAssemblies() | grep Security
it will not return System.Security, but if you then type this and let intellisense do its thing:
[System.
You can then run this again:
[AppDomain]::CurrentDomain.GetAssemblies() | grep Security
And it will then return System.Security. So this is why you can write a script that works fine, and then revisit it later and it's broken. Using intellisense doesn't fix your script though, instead you have to add this line:
Add-Type System.Security
Or whatever library is not getting auto-added (it seems to need the dll filename, e.g. C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.Security.dll).
I'm pretty sure IseSteroids (a paid ISE add-in) can detect this, maybe others as well.

Resources