Here is my Azure batch configuration which I am trying to create from microsoft tutorials here and here
I am trying to get env variables defined here
CloudPool pool = batchClient.PoolOperations.CreatePool(
poolId: PoolId,
targetDedicatedComputeNodes: PoolNodeCount,
virtualMachineSize: PoolVMSize,
virtualMachineConfiguration: vmConfiguration);
// Specify the application and version to install on the compute nodes
pool.ApplicationPackageReferences = new List<ApplicationPackageReference>
{
new ApplicationPackageReference {
ApplicationId = "7Zip",
Version = "19.00" }
};
// Commit the pool so that it's created in the Batch service. As the nodes join
// the pool, the specified application package is installed on each.
await pool.CommitAsync();
CloudJob job = batchClient.JobOperations.CreateJob();
job.Id = JobId;
job.PoolInformation = new PoolInformation { PoolId = PoolId };
await job.CommitAsync();
string taskId = "blendertask01";
string commandLine =
#"cmd /c echo %AZ_BATCH_APP_PACKAGE_7Zip%";
CloudTask blenderTask = new CloudTask(taskId, commandLine);
batchClient.JobOperations.AddTask(JobId, blenderTask);
I am expecting the output of cmd /c echo %AZ_BATCH_APP_PACKAGE_7Zip% to give me the path where I can find my application 7zip so that i can install it however I dont get that.
Instead i get %AZ_BATCH_APP_PACKAGE_7Zip%
Instead of using AZ_BATCH_APP_PACKAGE_7Zip we should specify the version also
thus it would become %AZ_BATCH_APP_PACKAGE_7Zip#19.00%\7z1900-x64.exe
This information is not clearly defined however after few hit and tries I discovered it
Also that the env variables are not visible when we log on using remote account.
Related
I am using Azure Batch with a C# application. The overall C# application is responsible for doing many things. Unfortunately, 1 operation that works on my local machine does not work within an Azure Batch application.
The operation that works locally but not as an Azure C# application is programmatically starting a System.Diagnostics. A process that executes a FFmpeg argument to take a screenshot of a video. The process seems to run, but, the jpg file is not created. Locally, the code will create a jpg but as an Azure Batch app, the file is not created.
I have set the image file to be created in the AZ_BATCH_TASK_WORKING_DIR folder. I have reduced the code to just do the 1 operation that does not work as an Azure application and here it is:
using System;
using System.Diagnostics;
using System.IO;
namespace BatchAppWithFfmpegProcess
{
internal class Program
{
static void Main(string[] args)
{
#if DEBUG
string workingDirectory = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
#else
/*
Within each Tasks directory, the Batch service creates a working directory (wd) whose unique path is
specified by the AZ_BATCH_TASK_WORKING_DIR environment variable. This directory provides read/write access
to the task. The task can create, read, update, and delete files under this directory.
This directory is retained based on the RetentionTime constraint that is specified for the task.
The stdout.txt and stderr.txt files are written to the Tasks folder during the execution of the task.
*/
string workingDirectory = Environment.GetEnvironmentVariable("AZ_BATCH_TASK_WORKING_DIR");
#endif
var applicationPath = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
string ffmpegPath = Path.Combine(applicationPath, #"lib\");
string videoFilePath = Path.Combine(applicationPath, "MichaelJacksonSmoothCriminal_Trimmed.mp4");
string thumbnailFilePath = Path.Combine(workingDirectory, "MichaelJacksonSmoothCriminal_Trimmed.jpg");
if (File.Exists(thumbnailFilePath))
{
File.Delete(thumbnailFilePath);
}
string arguments = $"-i \"{videoFilePath}\" -ss 00:00:01.000 -frames:v 1 \"{thumbnailFilePath}\"";
var startInfo = new ProcessStartInfo
{
FileName = ffmpegPath + $"\\ffmpeg.exe",
Arguments = arguments,
RedirectStandardError = false,
RedirectStandardOutput = true,
CreateNoWindow = true,
UseShellExecute = false,
WorkingDirectory = ffmpegPath
};
using (var process = new Process { StartInfo = startInfo })
{
process.Start();
process.WaitForExit();
}
if (File.Exists(thumbnailFilePath))
{
Console.WriteLine("Hurray, it worked!!!");
}
else
{
Console.WriteLine("File was not created.");
}
}
}
}
Perhaps it is impossible to use System.Diagnostics.Process to create files? I have tried to make this as easy as possible to reproduce with the following:
clone the code at:
https://github.com/Dapp3rDanH/AzBatchFfmpegProcess.git
Using Visual Studio 2022, "Publish" BatchAppWithFfmpegProcess code to a Folder using
"Deployment mode" = Self-Contained
Target Framework of net5.0
Target runtime = win-x64.
Create a BatchAppWithFfmpegProcess.zip zip file of the publish folder. Make sure the BatchAppWithFfmpegProcess.exe is in the root of the zip.
Create a Batch account.
Add a Batch application using the BatchAppWithFfmpegProcess.zip file with appId of BatchAppWithFfmpegProcess and a version of 1.
Add a Pool called "Pool1" with 1 dedicated node using microsoftwindowsserver windowsserver 2022-datacenter-core (latest). Add BatchAppWithFfmpegProcess version 1 to Pool1.
Create a new Job with a Task with the command of:
cmd /c %AZ_BATCH_APP_PACKAGE_BatchAppWithFfmpegProcess#1%\BatchAppWithFfmpegProcess.exe
If you check out the stdout.txt file of the task, you will see "File was not created". Any way to get this to work?
Thanks!
Dan
The issue was related to my pool being based upon "microsoftwindowsserver windowsserver 2022-datacenter-core (latest)".
Switching to microsoftwindowsserver windowsserver 2016-datacenter (latest) fixed the issue.
I am developing a C# application that should run in Azure. I want to use the Azurite emulator to test it locally. What I want to achieve is: Have my tests detect whether Azurite is running and abort quickly with a nice error message if it is not running.
Apparently Azurite runs on Node.js.
With the old Microsoft Azure Storage Emulator, I can check it like this:
public static class AzureStorageEmulatorDetector
{
public static bool IsRunning()
{
const string exePath = #"C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator\AzureStorageEmulator.exe";
if (!File.Exists(exePath))
return false;
var processStartInfo = new ProcessStartInfo {FileName = exePath, Arguments = "status", RedirectStandardOutput = true};
var process = new Process {StartInfo = processStartInfo};
process.Start();
process.WaitForExit();
var processOutput = process.StandardOutput.ReadToEnd();
return processOutput.Contains("IsRunning: True");
}
}
I want to accomplish something similar with Azurite.
I have installed Azurite like this:
npm install -g azurite
I run it like this:
azurite --silent --location C:\temp\Azurite --debug c:\temp\Azurite\debug.log
I notice that the Azurite command-line application has no parameter that tells me whether it is already running. And when I start Azurite from the console I don't see any process or service in Task Explorer called anything like "azurite". So I don't know what process I'm supposed to check for.
EDIT: Apparently Azurite runs on Node.js. There is indeed a process called node.exe running, but that's not a sufficient condition. Can I query my running Node.js instance and get it to tell me what it is doing?
I am on Windows.
Does anyone know?
Inspired by the comment by Ivan Yang and this answer I did this:
private static bool IsAzuriteRunning()
{
// If Azurite is running, it will run on localhost and listen on port 10000 and/or 10001.
IPAddress expectedIp = new(new byte[] {127, 0, 0, 1});
var expectedPorts = new[] {10000, 10001};
var activeTcpListeners = IPGlobalProperties.GetIPGlobalProperties().GetActiveTcpListeners();
var relevantListeners = activeTcpListeners.Where(t =>
expectedPorts.Contains(t.Port) &&
t.Address.Equals(expectedIp))
.ToList();
return relevantListeners.Any();
}
EDIT: Alternatively, check out this thread on their GitHub for other possibilities: https://github.com/Azure/Azurite/issues/734
I am hosting a .NET Core Application on MS Azure (on a Linux Service Plan) and I want to run some NodeJS code in the .NET Core Application. I did this a while ago on a Windows Service Plan, there it was working. Now I am trying with a Linux Plan and it is not working.
First I was trying to use "Jering.Javascript.NodeJS" and then also "INodeServices" from Microsoft (which is obsolete). But "node" was not found. I also tried to start directly a Process (Code below), but also not working. "node" is not found.
var proc = new System.Diagnostics.Process
{
StartInfo = new System.Diagnostics.ProcessStartInfo
{
FileName = "node",
Arguments = " -v",
RedirectStandardOutput = true
}
};
result += "RUN: " + proc.StartInfo.FileName;
proc.Start();
var reader = proc.StandardOutput;
NodeJS is installed on the server and also the command works there but it seems that the .NET Core app is hosted as docker and does not have any access outside to run NodeJS. Image
I found a useful information here.
The problem is that Node is not present in the container so it is
necessary to have a script to install and start it before starting the
app itself.
Reproduceļ¼
Here is my script:
//using System.Diagnostics;
ProcessStartInfo startinfo = new ProcessStartInfo();
startinfo.FileName = "bash";
//startinfo.FileName = "/etc/opt/nodejs/14.15.0/bin/node"; //it's no use even node package located here.
Process process = new Process();
process.StartInfo = startinfo;
process.StartInfo.UseShellExecute = false;
process.StartInfo.RedirectStandardInput = true;
process.StartInfo.RedirectStandardOutput = true;
process.Start();
//install and start nodejs
process.StandardInput.WriteLine("apt-get install curl");
process.StandardInput.WriteLine("curl -sL https://deb.nodesource.com/setup_12.x | bash");
process.StandardInput.WriteLine("apt-get install -y nodejs");
//Run "node -v"
process.StandardInput.WriteLine("node -v");
string line = string.Empty;
while (!process.StandardOutput.EndOfStream)
{
line = process.StandardOutput.ReadLine();
_logger.LogInformation(line);
}
process.WaitForExit();
return string.Empty;
It works on my .net Core app based on Linux.
I think I found a better solution ;)
In an app service you can mount a storage. In my case I mounted a storage, which contains the nodeJS lib.
Azure Portal Screenshot
Now i can execute the following code:
string result = "";
var proc = new System.Diagnostics.Process
{
StartInfo = new System.Diagnostics.ProcessStartInfo
{
FileName = "/externallibs/node/bin/node",
Arguments = " -v",
RedirectStandardOutput = true
}
};
result += "RUN: " + proc.StartInfo.FileName;
proc.Start();
var reader = proc.StandardOutput;
return result + reader.ReadToEnd();
You can create on azure portal an environment var named POST_BUILD_COMMAND with a command to fix your environment path.
Linux Service Plans runs on Oryx which is documented here
POST_BUILD_COMMAND=PATH=/usr/bin/node:$PATH
Following the answer suggested in the question -
Is it possible to permanently set environment variables?
I was able to set new environment variables permanently with the command -
spawnSync('setx', ['-m', 'MyDownloads', 'H:\\temp\\downloads'])
But now my goal is to append new values to the PATH environment variable.
Is it possible?
Why don't you just get the environment variable and then append to it?
I.e.
const {spawnSync} = require("child_process");
const current_value = process.env.PATH;
const new_path_value = current_value.concat(";", "/some/new/path");
var result = spawnSync('setx', ['-m', 'PATH', new_path_value])
// STDOUT
var stdOut = result.stdout.toString();
console.log(stdOut)
// STDERR
var stdErr = result.stderr.toString();
if(stdErr === '') {
console.log('Successfully set environment variable')
} else {
console.log(`ERROR: ${stderr}`)
}
Update "/some/new/path" and run this as admin as the link you provided suggests and it should work.
Run your script with the admin permission:
Open cmd or PowerShell with admin
Run node your_script.js
To append PATH variable, you can set value is : %PATH%;your_new_value here (%PATH% get old value)
If you run with electron app, you should require admin permission.
Don't forget setx run on window
I don't have rights to modify my registry, and I also would rather not call an OS command such as setx.
The following adds an additional component to the Windows PATH. I then ran Selenium, which uses the new setting.
// Display current value of PATH
const current_value = process.env.PATH;
console.log("PREV VALUE:")
console.log(current_value)
// Add the additional entry
const addl_entry = String.raw`\my\new\path\component`
process.env["PATH"] = addl_entry + ";" + current_value
// Display the new value
console.log("NEW VALUE:")
console.log(process.env.PATH)
I want to upload automatically the APK file to the server when building the release version.
To do so, I'm going to use FTP protocol.
I'm new regarding Gradle scripting. I used those 2 questions (this and this) as a base but something is not working out.
Could anyone point out what it is?
This is the code (on build.gradle):
gradle.buildFinished {
println("---- Build finished. This message appears ----")
task ftp << {
project.logger.lifecycle('-- This message does not appear --')
ant {
taskdef(name: 'ftp',
classname: 'org.apache.tools.ant.taskdefs.optional.net.FTP',
classpath: configurations.ftpAntTask.asPath)
def destination = "ftp://xxxxxxxxxx#xxx.surftown.com/xxxxx/"
def source = null
android.applicationVariants.all { variant ->
if ( (variant.name).equals("release") ) {
variant.outputs.each { output ->
source = output.outputFile
}
}
}
def user = 'xxxxxxxxx'
def pass = 'xxxxxxxxx'
ftp(server: source, userid: user, password: pass, remoteDir: destination)
}
}
gradle.buildFinished registers a hook executed at the end of the build. In your case it just creates the ftp task.
Use this if the build task is involved :
build.finalizedBy(ftp)
Otherwise, to make sure it works whatever the invoked task :
tasks.all*.finalizedBy(ftp)
By the way, it was explained in the comment section of first answer of your first link.