I want to upload automatically the APK file to the server when building the release version.
To do so, I'm going to use FTP protocol.
I'm new regarding Gradle scripting. I used those 2 questions (this and this) as a base but something is not working out.
Could anyone point out what it is?
This is the code (on build.gradle):
gradle.buildFinished {
println("---- Build finished. This message appears ----")
task ftp << {
project.logger.lifecycle('-- This message does not appear --')
ant {
taskdef(name: 'ftp',
classname: 'org.apache.tools.ant.taskdefs.optional.net.FTP',
classpath: configurations.ftpAntTask.asPath)
def destination = "ftp://xxxxxxxxxx#xxx.surftown.com/xxxxx/"
def source = null
android.applicationVariants.all { variant ->
if ( (variant.name).equals("release") ) {
variant.outputs.each { output ->
source = output.outputFile
}
}
}
def user = 'xxxxxxxxx'
def pass = 'xxxxxxxxx'
ftp(server: source, userid: user, password: pass, remoteDir: destination)
}
}
gradle.buildFinished registers a hook executed at the end of the build. In your case it just creates the ftp task.
Use this if the build task is involved :
build.finalizedBy(ftp)
Otherwise, to make sure it works whatever the invoked task :
tasks.all*.finalizedBy(ftp)
By the way, it was explained in the comment section of first answer of your first link.
Related
I am using Azure Batch with a C# application. The overall C# application is responsible for doing many things. Unfortunately, 1 operation that works on my local machine does not work within an Azure Batch application.
The operation that works locally but not as an Azure C# application is programmatically starting a System.Diagnostics. A process that executes a FFmpeg argument to take a screenshot of a video. The process seems to run, but, the jpg file is not created. Locally, the code will create a jpg but as an Azure Batch app, the file is not created.
I have set the image file to be created in the AZ_BATCH_TASK_WORKING_DIR folder. I have reduced the code to just do the 1 operation that does not work as an Azure application and here it is:
using System;
using System.Diagnostics;
using System.IO;
namespace BatchAppWithFfmpegProcess
{
internal class Program
{
static void Main(string[] args)
{
#if DEBUG
string workingDirectory = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
#else
/*
Within each Tasks directory, the Batch service creates a working directory (wd) whose unique path is
specified by the AZ_BATCH_TASK_WORKING_DIR environment variable. This directory provides read/write access
to the task. The task can create, read, update, and delete files under this directory.
This directory is retained based on the RetentionTime constraint that is specified for the task.
The stdout.txt and stderr.txt files are written to the Tasks folder during the execution of the task.
*/
string workingDirectory = Environment.GetEnvironmentVariable("AZ_BATCH_TASK_WORKING_DIR");
#endif
var applicationPath = Path.GetDirectoryName(Process.GetCurrentProcess().MainModule.FileName);
string ffmpegPath = Path.Combine(applicationPath, #"lib\");
string videoFilePath = Path.Combine(applicationPath, "MichaelJacksonSmoothCriminal_Trimmed.mp4");
string thumbnailFilePath = Path.Combine(workingDirectory, "MichaelJacksonSmoothCriminal_Trimmed.jpg");
if (File.Exists(thumbnailFilePath))
{
File.Delete(thumbnailFilePath);
}
string arguments = $"-i \"{videoFilePath}\" -ss 00:00:01.000 -frames:v 1 \"{thumbnailFilePath}\"";
var startInfo = new ProcessStartInfo
{
FileName = ffmpegPath + $"\\ffmpeg.exe",
Arguments = arguments,
RedirectStandardError = false,
RedirectStandardOutput = true,
CreateNoWindow = true,
UseShellExecute = false,
WorkingDirectory = ffmpegPath
};
using (var process = new Process { StartInfo = startInfo })
{
process.Start();
process.WaitForExit();
}
if (File.Exists(thumbnailFilePath))
{
Console.WriteLine("Hurray, it worked!!!");
}
else
{
Console.WriteLine("File was not created.");
}
}
}
}
Perhaps it is impossible to use System.Diagnostics.Process to create files? I have tried to make this as easy as possible to reproduce with the following:
clone the code at:
https://github.com/Dapp3rDanH/AzBatchFfmpegProcess.git
Using Visual Studio 2022, "Publish" BatchAppWithFfmpegProcess code to a Folder using
"Deployment mode" = Self-Contained
Target Framework of net5.0
Target runtime = win-x64.
Create a BatchAppWithFfmpegProcess.zip zip file of the publish folder. Make sure the BatchAppWithFfmpegProcess.exe is in the root of the zip.
Create a Batch account.
Add a Batch application using the BatchAppWithFfmpegProcess.zip file with appId of BatchAppWithFfmpegProcess and a version of 1.
Add a Pool called "Pool1" with 1 dedicated node using microsoftwindowsserver windowsserver 2022-datacenter-core (latest). Add BatchAppWithFfmpegProcess version 1 to Pool1.
Create a new Job with a Task with the command of:
cmd /c %AZ_BATCH_APP_PACKAGE_BatchAppWithFfmpegProcess#1%\BatchAppWithFfmpegProcess.exe
If you check out the stdout.txt file of the task, you will see "File was not created". Any way to get this to work?
Thanks!
Dan
The issue was related to my pool being based upon "microsoftwindowsserver windowsserver 2022-datacenter-core (latest)".
Switching to microsoftwindowsserver windowsserver 2016-datacenter (latest) fixed the issue.
I'm trying to solve this Windows filename issue
Basically our CI job fails, with the 'filename too long' error for Windows.
warning: Could not stat path 'node_modules/data-validator/tests/data/data-examples/ds000247/sub-emptyroom/ses-18910512/meg/sub-emptyroom_ses-18910512_task-noise_run-01_meg.ds/sub-emptyroom_ses-18910512_task-noise_run-01_meg.acq': Filename too long
I've read the docs for Node's path module, which seems like a possible solution. I also read about a Windows prefix (\\?\) to bypass the MAX_PATH...but have no idea how to implement these in a clean way.
This part of the codebase with the tests that are failing. The hardcoded path (testDatasetPath) is likely part of the problem.
function getDirectories(srcpath) {
return fs.readdirSync(srcpath).filter(function(file) {
return (
file !== '.git' && fs.statSync(path.join(srcpath, file)).isDirectory()
)
})
}
var missing_session_files = //array of strings here
const dataDirectory = 'data-validator/tests/data/'
function createDatasetFileList(path) {
const testDatasetPath = `${dataDirectory}${path}`
if (!isNode) {
return createFileList(testDatasetPath)
} else {
return testDatasetPath
}
}
createFileList function
function createFileList(dir) {
const str = dir.substr(dir.lastIndexOf('/') + 1) + '$'
const rootpath = dir.replace(new RegExp(str), '')
const paths = getFilepaths(dir, [], rootpath)
return paths.map(path => {
return createFile(path, path.replace(rootpath, ''))
})
}
tl;dr A GitLab CI Job fails on Windows because the node module filenames become too long. How can I make this nodejs code OS agnostic?
This is a known error in the Windows Environment, however, there is a fix..
If you're using an NTFS based filesystem, you should be able to enable long paths in
Local Computer Policy > Computer Configuration > Administrative Templates > System > Filesystem > NTFS
This is also specified in the document you just linked, and theoretically, should work. However, the path shouldn't really be longer than 32 bits,
This will come with some performance hits, but should get the job done.
Also, you should preferably switch to a better package or search for alternatives. If this is your own package, maybe restructure it?
Finally, if none of these work, move your project folder directly to your data drive, (C:\) this will cut down on the other nested and parent folders.
This is inherently bad, and you may run into issues during deployment, if you choose to do it.
I am trying to search a file recursively inside a directory hence cannot use findFiles.
I have seen the directories via manually login in to the slave but it cannot be recognized in the code below. When I use isDirectory() it says false hence later while using dir.listFiles() it return null.
Below is the code:
def recursiveFileSearch(File dir, filename, filesPath) {
File[] files = dir.listFiles() // It returns null here as it cannot recognize it as directory
echo "$files"
for (int i=0; i < files.size(); i++) {
if (files[i].isDirectory()) {
recursiveFileSearch(files[i], filename, filesPath)
} else {
if (files[i].getAbsolutePath().contains(filename)) {
filesPath.add(files[i].getAbsolutePath())
return filesPath
}
}
}
return filesPath
}
node('maven') {
git 'https://github.com/rupalibehera/t3d.git'
sh 'mvn clean install'
File currentDir = new File(pwd())
def isdir = currentDir.isDirectory()
println "isdir:${isdir}" // The output here is False
def isexist = currentDir.exists()
println "isexist:${isexist}" // The output here is False
def canread = currentDir.canRead()
println "canread:${canread}" // The output here is False
def filesPath = []
def openshiftYaml = recursiveFileSearch(currentDir, "openshift.yml", filesPath)
}
I am not sure what is going wrong here.
But below are some observations:
When I do File currentDir = new File("."), it returns / and starts reading complete root directory which I don't want and in that also it does not recognize WORKSPACE as directory
It executes well if I run it on Master node, but in my use case it will be always a slave.
I have also checked the permissions of directory the user has read/write/execute permissions.
Any pointers/help is appreciated
Generally, run a sh step to do whatever work you need. You may not use java.io.File or the like from Pipeline script. It does not run on the agent, and is also insecure, which is why any such attempt will be rejected when the sandbox mode is left on (the default).
you are running into the Using File in Pipeline Description problem. I know it all too well. File objects and NIO work fine for breaking up paths, but their isDirectory, exists and other methods run on master as a part of the Jenkinsfile and not on the node. So, all use on master looks great, because the files are in the workspace. All use on a node, fails.
In short, don't do that. Use fileExists(), pwd(), findFiles etc
If you created a shareLibrary and want to use unit tests on the code outside of Jenkins, then you can create a fascade which relies on the script object ('this' from a pipeline)
Class for shared lib
class PipelineUtils implements Serializable {
static def pipelineScript = null;
/**
* Setup this fascade with access to pipeline script methods
* #param jenkinsPipelineScript
* #return
*/
static initialize(def jenkinsPipelineScript) {
pipelineScript = jenkinsPipelineScript
}
/**
* Use pipelineScript object ('this' from pipeline) to access fileExists
* We cannot use Java File objects for detection as the pipeline script runs on master and uses delegation/serialization to
* get to the node. So, File.exists() will be false if the file was generated on the node and that node isn't master.
* https://support.cloudbees.com/hc/en-us/articles/230922128-Pipeline-Using-java-io-File-in-a-Pipeline-description
* #param target
* #return true if path exists
*/
static boolean exists(Path target) {
if (!pipelineScript) {
throw new Exception("PipelineUtils.initialize with pipeline script not called - access to pipeline 'this' required for access to file detection routines")
}
if (! target.parent) {
throw new Exception('Please use absolutePaths with ${env.WORKSPACE}/path-to-file')
}
return pipelineScript.fileExists(target.toAbsolutePath().toString())
}
/**
* Convert workspace relative path to absolute path
* #param path relative path
* #return node specific absolute path
*/
static def relativeWorkspaceToAbsolutePath(String path) {
Path pwd = Paths.get(pipelineScript.pwd())
return pwd.resolve(path).toAbsolutePath().toString()
}
static void echo(def message) {
pipelineScript.echo(message)
}
}
class for tests
class JenkinsStep {
static boolean fileExists(def path) {
return new File(path).exists()
}
static def pwd() {
return System.getProperty("user.dir")
}
static def echo(def message) {
println "${message}"
}
}
usage in jenkins
PipelineUtils.initialize(this)
println PipelineUtils.exists(".")
// calls jenkins fileExists()
usage in unit tests
PipelineUtils.initialize(new JenkinsStep())
println PipelineUtils.exists(".")
// calls File.exists
I found the answer,
for searching any file in your workspace from Jenkinsfile you can use findFiles step,
I did try this but I was passing the incorrect glob for the same. Now I just do
def files = findFiles(glob: '**/openshift.yml') \\ it returns the path of file
I am working on a SoapUI project where I need to run my test suite using test runner. I am using external groovy scripting for environment variable. The problem I am facing here is whenever I am running test case from test runner its return Workspace as null, which is used in External groovy. So in external groovy I am getting workspace as null causing error [getProjectByname() cannot be invoked on null]. Below is the
constructor of global script where workspace is used
AvengerAPITestManager(String TestProject, String TestSuite, String TestCase,String TestStep)
{
TestName = "AvengerAPITests";
testProject = SoapUI.getWorkspace().getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep=testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName (TestStep);
}
Above we have user SoapUI.getWorkspace() which is working fine when trying to run from soapUI but whever I m trying to run from testrunner SoapUI.getWorkspace comes out to be null. I even tried passing workspace like I am passing testproject name still it didnt worked.
I tried something like this also
AvengerAPITestManager(Object workspace,String TestProject, String TestSuite, String TestCase, String TestStep)
{
TestName = "AvengerAPITests";
testProject = workspace.getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName(TestStep);
}
In the above code I tries passing Workspace object from the test case as I passed Testcase name and all but still I m getting null for workspace. Please tell me how do I deal with the problem.
Here is usefull working example https://github.com/stokito/soapui-junit
You should place your sample-soapui-project.xml to /src/test/resources folder that will expose it to classpath
If you want to use soap ui in external code, try to directly create new test runner with specific project file:
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile( "src/dist/sample-soapui-project.xml" );
runner.run();
Or if you want to define test execution more precisely, you can use something like this:
WsdlProject project = new WsdlProject( "src/dist/sample-soapui-project.xml" );
TestSuite testSuite = project.getTestSuiteByName( "Test Suite" );
TestCase testCase = testSuite.getTestCaseByName( "Test Conversions" );
// create empty properties and run synchronously
TestRunner runner = testCase.run( new PropertiesMap(), false );
PS: don't forget to import soap ui classes, that you use in your code and put them to classpath.
PPS: If you need just run test cases outside the soap ui and/or automate this process, why not just use testrunner.sh/.bat for the same thing? (here is description of this way: http://www.soapui.org/Test-Automation/functional-tests.html)
I am not sure if this is going to help anyone out there but here is what I did to fix the problem I was having with workspace as null causing error[getProjectByname() cannot be invoked on null] When i run from cmd
try this:
import com.eviware.soapui.model.project.ProjectFactoryRegistry
import com.eviware.soapui.impl.wsdl.WsdlProjectFactory
import com.eviware.soapui.impl.wsdl.WsdlProject
//get the Util project
def project = null
def workspace = testRunner.testCase.testSuite.project.getWorkspace();
//if running Soapui
if (workspace != null) {
project = workspace.getProjectByName("Your Project")
}
//if running in Jenkins/Hudson
else{
project = new WsdlProject("C:\\...\\....\\....\\-soapui-project.xml");
}
if (project.open && project.name == "Your Project") {
def properties = new com.eviware.soapui.support.types.StringToObjectMap()
def testCase = project.getTestSuiteByName("TestSuite 1").getTestCaseByName("TestCase");
if(testCase == null)
{
throw new RuntimeException("Could not locate testcase 'TestCase'! ");
} else {
// This will run everything in the selected project
runner = testCase.run(new com.eviware.soapui.support.types.StringToObjectMap(), false)
}
}
else {
throw new RuntimeException("Could not find project ' Order Id....' !")
}
The above code will run everything in the selected project.
I'd like to do something very simple: Create/write to a file located in the remote workspace of a slave via the jenkins groovy post-build script plug-in
def props_file = new File(manager.build.workspace.getRemote() + "/temp/module.properties")
def build_num = manager.build.buildVariables.get("MODULE_BUILD_NUMBER").toInteger()
def build_props = new Properties()
build_props["build.number"] = build_num
props_file.withOutputStream { p ->
build_props.store(p, null)
}
The last line fails, as the file doesn't exist. I'm thinking it has something to do with the output stream pointing to the master executor, rather than the remote workspace, but I'm not sure:
Groovy script failed:
java.io.FileNotFoundException: /views/build_view/temp/module.properties (No such file or directory)
Am I not writing to the file correctly?
While writing onto slave you need to check the channel first and then you can successfully create a file handle and start reading or writing to that file:
if(manager.build.workspace.isRemote())
{
channel = manager.build.workspace.channel;
}
fp = new hudson.FilePath(channel, manager.build.workspace.toString() + "\\test.properties")
if(fp != null)
{
String str = "test";
fp.write(str, null); //writing to file
versionString = fp.readToString(); //reading from file
}
hope this helps!
Search for words The post build plugin runs on the manager and doing it as you say will fail if you are working with slaves! on the plugin page (the link to which you've provided) and see if the workaround there helps.
Does the folder /views/build_view/temp exist?
If not, you will need to do new File( "${manager.build.workspace.remote}/temp" ).mkdirs()