I have been trying to start selenium tests with browsers, not headless. I have my code in SVN and this should be build by Jenkins. Jenkins is on Linux.
Has anyone ever tried that or do you know what steps I should take? I am going through tutorials available on the internet but none of them work for me.
My current error is:
java.io.IOException: Cannot run program "cmd" (in directory
"/var/lib/jenkins/jobs/Tests/workspace"): error=2, No such file or
directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at hudson.Proc$LocalProc.(Proc.java:240) at
hudson.Proc$LocalProc.(Proc.java:212) at
hudson.Launcher$LocalLauncher.launch(Launcher.java:815) at
hudson.Launcher$ProcStarter.start(Launcher.java:381) at
hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:95)
at
hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:64)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205) at
hudson.model.Build$BuildExecution.doRun(Build.java:162) at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1720) at
hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at
hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410) Caused by:
java.io.IOException: error=2, No such file or directory at
java.lang.UNIXProcess.forkAndExec(Native Method) at
java.lang.UNIXProcess.(UNIXProcess.java:187) at
java.lang.ProcessImpl.start(ProcessImpl.java:130) at
java.lang.ProcessBuilder.start(ProcessBuilder.java:1028) ... 15 more
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
When built without error, the workspace was only updated:
Building in workspace /var/lib/jenkins/jobs/Tests/workspace
Updating http://XX.XX.XXX.XX/resp/extend/Tests/EO at revision '2016-07-04T14:34:05.110 +0200'
At revision 5536
No changes for http://XX.XX.XXX.XX/resp/extend/Tests/EO since the previous build
Finished: SUCCESS
If you need any further details, let me know ...
UPDATE:
[workspace] $ /bin/sh -xe /tmp/hudson8771943326851387647.sh
+ ./script.sh
/tmp/hudson8771943326851387647.sh: line 2: ./script.sh: Permission denied
Build step 'Execute shell' marked build as failure
Finished: FAILURE
I have changed the windows command to shell command. The shell command is:
./script.sh
I think that the script inside the file i wrong, but does the error I have now mean that the script is wrong or does it reflect to something else? I do not know hwy we have "permission denied" - I have all the right in jenkins.
And this is my script.sh file content:
#!/bin/bash
function run_test {
echo "=== run Selenium tests in Jenkins ==="
ssh root#$1 "/src/test/java/mainTest/MainOrderTest start"
}
fi
I have no knowledge on shell command and I have not found any command that would run this program.
You are trying to run windows batch scripts on Linux hence the error.
Two options :
1. Either convert your build scripts to shell and use "execute shell" to call your shell scripts in jenkins
2. Add a windows slave which will build your windows batch script
Related
Hi people of the internet.
Basically I am unable to run even the simplest job and I keep getting the same error no matter what I put in the .gitlab-ci.yml file. See example below:
Here is the .gitlab-ci.yml file:
stages:
- test
job1:
stage: test
tags:
- testing
script:
- echo "Hello world!"
Here is the output ("?" corresponds to intentionally blacked out information):
Running with gitlab-runner 14.10.0 (c6bb62f6)
on runner_test ????????
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on LAPTOP-????????...
Getting source from Git repository
00:01
WriteError:
Line |
219 | $HOST="[MASKED]"
| ~~~~~~~~~~~~~~~~~~~~~~
| Cannot overwrite variable Host because it is read-only or constant.
ERROR: Job failed: exit status 1
I know that $HOST is a reserved variable in powershell but I don't see the link between the error and the code. It may have something to do with the configuration of the runner on Windows. Has anyone encountered this error on Gitlab before? Or any suggestions on how to debug?
Here are the steps that I took to install the runner on Gitlab for Windows (see https://docs.gitlab.com/runner/install/windows.html):
Create a folder somewhere in the system: C:\GitLab-Runner.
Download the binary for 64-bit and put it into the folder (see https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-amd64.exe).
Run prompt as an administrator
Run the following command:
cd C:\GitLab-Runner
gitlab-runner.exe register
Enter your GitLab instance URL (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter the token to register the runner (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter a description for the runner: runner_test for instance
Enter the tags associated with the runner, separated by commas: testing, windows for instance
Provide the runner executor: shell
Install GitLab Runner as a service and start it
cd C:\GitLab-Runner
gitlab-runner.exe install
gitlab-runner.exe start
I also had to install the latest version of pwsh in Windows (see gitlab-runner: prepare environment failed to start process pwsh in windows):
Run prompt as an administrator
Install the newer pwsh.exe:
winget install Microsoft.PowerShell
Restart the runner
cd C:\GitLab-Runner
gitlab-runner.exe restart
This issue was due to my choice of shell for some reason. A Gitlab runner can choose a shell among the following: bash, sh, powershell, pwsh, and cmd (the last one being deprecated now).
As I stated above I had been using pwsh. So, I went after the config.toml file inside of the C:\GitLab-Runner directory to manually make the change from pwsh to powershell.
...
[[runners]]
name = "runner_test"
executor = "shell"
shell = "powershell"
...
I then restarted the runner and got the job to complete properly:
cd C:\GitLab-Runner
gitlab-runner restart
I still get the error (more like a warning now) but it does not prevent the job from finishing anymore. If anyone has a better answer with a proper explanation I would gladly accept it as the answer to this question.
Note that pwsh to powershell are both powershell scripts (see https://docs.gitlab.com/runner/shells/index.html):
powershell Fully Supported PowerShell script. All commands are executed in PowerShell Desktop context. In GitLab Runner 12.0-13.12, this is the default when registering a new runner.
pwsh Fully Supported PowerShell script. All commands are executed in PowerShell Core context. In GitLab Runner 14.0 and later, this is the default when registering a new runner.
Ok, it is very strange. I have some init scripts that I would like to run when a cluster starts
cluster has the init script , which is in a file (in dbfs)
basically this
dbfs:/databricks/init-scripts/custom-cert.sh
Now , when I make the init script like this, it works (no ssl errors for my endpoints. Also, the event logs for the cluster shows the duration as 1 second for the init script
dbutils.fs.put("/databricks/init-scripts/custom-cert.sh", """#!/bin/bash
cp /dbfs/orgcertificates/orgcerts.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
echo "export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt" >> /databricks/spark/conf/spark-env.sh
""")
However, if I just put the init script in an bash script and upload it to DBFS through a pipeline, the init script does not do anything. It executes , as per the event log but the execution duration is 0 sec.
I have the sh script in a file named
custom-cert.sh
with the same contents as above, i.e.
#!/bin/bash
cp /dbfs/orgcertificates/orgcerts.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
echo "export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt"
but when I check /usr/local/share/ca-certificates/ , it does not contain /dbfs/orgcertificates/orgcerts.crt, even though the cluster init script has run.
Also, I have compared the contents of the init script in both cases and it least to the naked eye, I can't figure out any difference
i.e.
%sh
cat /dbfs/databricks/init-scripts/custom-cert.sh
shows the same contents in both the scenarios. What is the problem with the 2nd case?
EDIT: I read a bit more about init scripts and found that the logs of init scripts are written here
%sh
ls /databricks/init_scripts/
Looking at the err file in that location, it seems there is an error
sudo: update-ca-certificates
: command not found
Why is it that update-ca-certificates found in the first case but not when I put the same script in a sh script and upload it to dbfs (instead of executing the dbutils.fs.put within a notebook) ?
EDIT 2: In response to the first answer. After running the command
dbutils.fs.put("/databricks/init-scripts/custom-cert.sh", """#!/bin/bash
cp /dbfs/orgcertificates/orgcerts.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
echo "export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt" >> /databricks/spark/conf/spark-env.sh
""")
the output is the file custom-cert.sh and then I restart the cluster with the init script location as dbfs:/databricks/init-scripts/custom-cert.sh and then it works. So, it is essentially the same content that the init script is reading (which is the generated sh script). Why can't it read it if I do not use dbfs put but just put the contents in bash file and upload it during the CI/CD process?
As we aware, An init script is a shell script that runs during startup of each cluster node before the Apache Spark driver or worker JVM start. case-2 When you run bash
command by using of %sh magic command means you are trying to execute this command in Local driver node. So that workers nodes is not able to access . But based on
case-1 , By using of %fs magic command you are trying run copy command (dbutils.fs.put )from root . So that along with driver node , other workers node also can access path .
Ref : https://docs.databricks.com/data/databricks-file-system.html#summary-table-and-diagram
It seems that my observations I made in the comments section of my question is the way to go.
I now create the init script using a databricks job that I run during the CI/CD pipeline from Azure DevOps.
The notebook has the commands
dbutils.fs.rm("/databricks/init-scripts/custom-cert.sh")
dbutils.fs.put("/databricks/init-scripts/custom-cert.sh", """#!/bin/bash
cp /dbfs/internal-certificates/certs.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
echo "export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt" >> /databricks/spark/conf/spark-env.sh
""")
I then create a Databricks job (pointing to this notebook), the cluster is a job cluster which is just temporary . Of course , in my case , even this job creation is automated using a powershell script.
I then call this Databricks job in the release pipeline using again a Powershell script.
This creates the file
/databricks/init-scripts/custom-cert.sh
I then use this file in any other cluster that accesses my org's endpoints (without certificate errors).
I do not know (or still understand), why can't the same script file be just part of a repo and uploaded during the release process (instead of it being this Databricks job calling a notebook). I would love to know the reason . The other answer on this question does not hold true as you can see, that the cluster script is created by a job cluster and then accessed from another cluster as part of its init script.
It simply boils down to how the init script gets created.
But I get my job done. Just if it helps someone get their job done too.
I have raised a support case though to understand the reason.
I have created one .sh which contains java command required to run the tesng.XML file when I run this test.sh file on Jenkins it shows me permission denied error.
Jenkins console output:
Building in workspace /home/dev2/eclipse-workspace/weeklytask
[weeklytask] $ /bin/sh -xe /tmp/jenkins7439378074963422991.sh
+ ./test.sh
[TestNG] Running:
/home/dev2/eclipse-workspace/weeklytask/enquiryreminder.xml
Starting ChromeDriver 2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7) on port 8120
Only local connections are allowed.
FOLLOW UP REMINDER CRON WEEKLY CHECK TESTCASE FAILED
PLEASE CHECK THE TIME INTERVAL SETTING
SET THE TIME INTERVAL TO 600
FOLLOW UP REMINDER CRON WEEKLY CHECK TESTCASE FAILED
PLEASE CHECK THE TIME INTERVAL SETTING
SET THE TIME INTERVAL TO 600
[[Utils]] Error while writing to /home/dev2/eclipse-workspace/weeklytask/test-output/Suite/Test.html: /home/dev2/eclipse-workspace/weeklytask/test-output/Suite/Test.html (**Permission denied**)
[[Utils]] Error while writing to /home/dev2/eclipse-workspace/weeklytask/test-output/Suite/Test.xml: /home/dev2/eclipse-workspace/weeklytask/test-output/Suite/Test.xml (Permission denied)
===============================================
Suite
Total tests run: 1, Failures: 1, Skips: 0
===============================================
[[Utils]] Error while writing to test-output/junitreports/TEST-verify.enquiryremindertest.xml: test-output/junitreports/TEST-verify.enquiryremindertest.xml (Permission denied)
Jenkins use the user jenkins. If you have created your file with other user, there might be permission issue on the file. To resolve change the permission on the files and allow other users to execute the script. You can use
chmod +x test.sh
The directory eclipse-workspace belongs to dev2 user and here jenkins user is trying to write data in it. Give jenkins write permission on this folder. try
sudo chmod -R a+rw /home/dev2/eclipse-workspace
For more details about permission you can view This tutorial
I am trying to use sh or ssh to connect to a linux box via jenkins (I am a noob admittedly). Even trying a ls command I am getting error - I did have this working before however - any help greatly appreciated.
Building in workspace /var/lib/jenkins/jobs/Demo/workspace executing
script:
USER="jenkins" sh '''#!/bin/bash
HOST=10.59.151.121
USER=devuser
PASSWORD=TGMCfpfS
ls
bye
EOF
'''
: No such file or directory [SSH] exit-status: 127 Build step
'Execute shell script on remote host using ssh' marked build as
failure Finished: FAILURE
For some reason I found that adding commands after the ''' allows them to be executed - even although the same warning appears, it works fine!
I have used this Gradle Openshift quickstart to use gradle with openshift, everything worked even with Jenkins!
But, I wanna use Groovy, so I added the follow code in my build.gradle to compile groovy scripts to .class
See
sourceSets{
main{
java{
srcDir "src/main/java"
}
resources{
srcDir "src/main/java"
}
groovy{
srcDir "src/main/java"
}
}
test.java.srcDir "src/test/java"
}
It works locally, I'm using Tomcat7, but when I deploy the jenkins build fails with the message
:compileJava
:compileGroovy FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':compileGroovy'.
> java.net.SocketException: Permission denied
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or -- debug option to get more log output.
So, the problem is about Permission denied when gradle try to execute compileGroovy task, I guess is something in the OpenShift OS but I don't know what's wrong, I have inserted the follow code in the files .openshift/build and .openshift/pre-build, see below
pre_build file
if [ ! -d $OPENSHIFT_DATA_DIR/gradle-1.6 ]
then
cd $OPENSHIFT_DATA_DIR
mkdir gradle
wget http://services.gradle.org/distributions/gradle-1.6-bin.zip
unzip gradle-1.6-bin.zip
rm -f gradle-1.6-bin.zip
fi
if [ ! -d $OPENSHIFT_DATA_DIR/groovy ]
then
cd $OPENSHIFT_DATA_DIR
mkdir groovy
wget http://dl.bintray.com/groovy/maven/groovy-binary-2.4.1.zip
unzip groovy-binary-2.4.1.zip
rm -f groovy-binary-2.4.1.zip
fi
build file
cd $OPENSHIFT_REPO_DIR
echo SETTING GRADLE HOME
export GRADLE_USER_HOME=$OPENSHIFT_DATA_DIR/gradle
export GRADLE_HOME=$OPENSHIFT_DATA_DIR/gradle-1.6
export PATH=$GRADLE_HOME/bin:$PATH
gradle build
export GROOVY_USER_HOME=$OPENSHIFT_DATA_DIR/groovy
export GROOVY_HOME=$OPENSHIFT_DATA_DIR/groovy
export PATH=$GRADLE_HOME/bin:$PATH
Well, I can't compile groovy files inside openshift even I have installed the groovy, how to make groovy works on Openshift?
java.net.SocketException: Permission deniedusually means that you are trying to setup a server socket or comparable on a port you don't have permission for. This could be the firewall blocking, or on Linux systems a too low port number reserved for privileged users like root. I think OpenShift uses port in the 15000+ range, so the firewall is the most likely problem.