“No X11 DISPLAY variable” - what does it mean? - apache-spark

I installed Jmeter on one of my Mesos nodes, but I can not run it I have this error that appears
================================================== ==============================
Do not use GUI mode for load testing, only for Test creation and Test debugging.
For load testing, use CLI Mode (was NOT GUI):
jmeter -n -t [jmx file] -l [results file] -e -o [Path to web report folder]
& increase Java Heap to meet your test requirements:
Modify current env variable HEAP = "- Xms1g -Xmx1g -XX: MaxMetaspaceSize = 256m" in the jmeter batch file
Check: https://jmeter.apache.org/usermanual/best-practices.html
================================================== ==============================
An error occurred: Can not connect to the window server using ': 0' as the value of the DISPLAY variable.

A DISPLAY is only needed if you're running this in GUI mode. Since you're running this in Mesos, you cannot use GUI mode. Add the -n option to run in CLI mode.
JMeter: Getting Started: CLI Mode

Related

Docker - unable to run script

What I'm doing
I am using AWS batch to run a docker container for a large compute job. I have configured the ECR/ECS successfully to the best of my knowledge but am having issues running the required commands for reasons that are beyond my level of understanding with docker ( newbie )
What I need to do is pass the below commands into my application and start my application to perform some heavy computing tasks; all commands listed below must be present.
The Issue(s)
The issue arises when I send the submit job to AWS batch; this service pulls the image from the ACR ( amazon container repository ) and spins up a compute environment. The issue comes from when I try to run the command I pass in, below I will go throgh it.
"command": [
"mkdir -p logging",
"chmod 777 logging/",
"docker run -t -i -e my-application", # container name
"-e APIKEY",
"-e BASEURI",
"-e APIUSER",
"-v WORKSPACE /logging:/src/log",
"DOCKERIMAGE",
"python my_app.py",
"-t APP_USER",
"-e APP_ENVIRONMENT",
"-u APP_USERNAME",
"-p APP_PASSWORD",
"-i IN_PATH",
"-o OUT_PATH",
"-b tmp/"
]
The command above generates the following error(s)
container_linux.go:370: starting container process caused: exec: "mkdir -p log": executable file not found in $PATH
I tried to pass in the command to echo the env var $PATH but was unsuccesfull getting a response and resulted in a similar error.
I have ran successfully "ls" and was able to see the directory contents of my application inside.
I am not however able to run any of these commands that I have included in the command [] section. I have tried just running python and such in hopes of getting a more detailed error but was unsuccessful.
Logic in plain English
Create a path called logging if it doesnt exist
set the permissions for logging
run the docker container and pass in the environment variables while doing so
Tell docker to run the python file my_app.py and pass in the expected runtime args
Execute and perform the required logic deligated in the python3 application
Questions
Why can I not create a directory here called "logging" where am I ?
Am I running these properly as defined by AWS batch? or docker
What am I missing or where am I going wrong?
AWS Batch high level doc
AWS Batch link specific to what i'm doing
Assuming that you're following the syntax described in the Container
Properties
section of the AWS docs, you have several problems with the syntax of
your command directive.
First
The command directive can only run a single command. You can't mash together a bunch of commands as you're trying to do in your example. If you need to run multiple commands you would need to embed them as an argument to a shell. For example, something like:
command: ["/bin/sh", "-c", "mkdir -p logging; chmod 777 logging; ..."]
Second
You must properly tokenize your
command lines -- that is, when you type mkdir -p logging at the
command prompt, the shell splits this into three parts (or "tokens"): ['mkdir', '-p', 'logging']. You need to do the same thing when building up the
list of arguments to command.
This is invalid:
command: ["mkdir -p logging"]
That would looking for a command named mkdir -p logging, and of course no such command exists. That would properly be written as:
command: ["mkdir", "-p", "logging"]
Third
I'm not very familiar with the AWS batch environment, but it's unlikely you can run a docker command inside a docker` container as you're trying to do. It's unclear why you're doing this, though: why not just configure your AWS batch job with the appropriate image, environment variables, etc?
Take a look at some of these example job definitions.

Jmeter:Difference between jmeter.sh and jmeter without extension file in jmeter

I would need clarification on jmeter.sh and jmeter file without ext ,which are in bin folder.
With example:
1.If i setup different HEAP size in jmeter and jmeter.sh file,which one will be considered.?
2.Does the above depend on how i run the test?(for ex: jmeter -n -t or jmeter.sh -n -t)
3.If the test started with jmeter command instead of jmeter.sh ,will intern jmeter.sh be called and hence heap in jmeter.sh be used or vice-versa?
related question to the difference b/w jmeter.bat and jmeter-
difference between jmeter.bat/jmeter.sh And jmeter.file
jmeter.sh is a wrapper for jmeter script (without extension) which does some pre-requisite validations like getting current working dir, getting Java version, constructing arguments depending on Java version, etc. so you'd better use this file for running JMeter under Unix and derivatives
jmeter is a wrapper for ApacheJMeter.jar binary, it sets default JVM arguments and overrides and adds more Java arguments depending on your operating system
The sequence is the following:
jmeter.sh calls jmeter
jmeter calls ApacheJMeter.jar
If you want to change HEAP or whatever - set the appropriate environment variable like:
HEAP=4G && export HEAP && ./jmeter.sh -n -t /path/to/test.jmx ...
More information: How to Get Started With JMeter: Installation & Test Plans
jmeter.sh calls jmeter, both unix scripts , jmeter is the main/default script
jmeter
run JMeter (in GUI mode by default). Defines some JVM settings which may not work for all JVMs.
jmeter.sh
very basic JMeter script (You may need to adapt JVM options like memory settings).
You can set before running both the JVM_ARGS
It may be necessary to set a few environment variables to configure the JVM used by JMeter. Those variables can be either set directly in the shell starting the jmeter script. For example setting the variable JVM_ARGS will override most pre-defined settings, for example
JVM_ARGS="-Xms1024m -Xmx1024m" jmeter -t test.jmx [etc.]
will override the HEAP settings in the script.

error running run file in centOS - bad display

I have a task to install Oracle 11g on a centOS 8 using VM (i'm new to linux / oracle).
I downloaded the Oracle files and unzipped them, then I tried to ./runInstaller but I get an error and this is the full terminal with error:
login as: admin
admin#192.168.163.129's password:
Activate the web console with: systemctl enable --now cockpit.socket
Last login: Thu May 21 09:26:48 2020 from 192.168.163.1
[admin#oracledb ~]$ cd Downloads
[admin#oracledb Downloads]$ cd database
[admin#oracledb database]$ ls
doc linux.x64_11gR2_database_1of2 response runInstaller stage
install linux.x64_11gR2_database_2of2 rpm sshsetup welcome.html
[admin#oracledb database]$ ./runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 120 MB. Actual 2027 MB Passed
Checking swap space: must be greater than 150 MB. Actual 1759 MB Passed
Checking monitor: must be configured to display at least 256 colors
>>> Could not execute auto check for display colors using command /usr/bin/xdpyinfo. Check if the DISPLAY variable is set. Failed <<<<
Some requirement checks failed. You must fulfill these requirements before
continuing with the installation,
Continue? (y/n) [n] y
>>> Ignoring required pre-requisite failures. Continuing...
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2020-05-21_09-43-58AM. Please wait ...
DISPLAY not set. Please set the DISPLAY and try again.
Depending on the Unix Shell, you can use one of the following commands as examples to set the DISPLAY environment variable:
- For csh: % setenv DISPLAY 192.168.1.128:0.0
- For sh, ksh and bash: $ DISPLAY=192.168.1.128:0.0; export DISPLAY
Use the following command to see what shell is being used:
echo $SHELL
Use the following command to view the current DISPLAY environment variable setting:
echo $DISPLAY
- Make sure that client users are authorized to connect to the X Server.
To enable client users to access the X Server, open an xterm, dtterm or xconsole as the user that started the session and type the following command:
% xhost +
To test that the DISPLAY environment variable is set correctly, run a X11 based program that comes with the native operating system such as 'xclock':
% <full path to xclock.. see below>
If you are not able to run xclock successfully, please refer to your PC-X Server or OS vendor for further assistance.
Typical path for xclock: /usr/X11R6/bin/xclock
[admin#oracledb database]$
I am using putty and Xming but I still get this error.
Make sure your putty session is connecting with "x-11 port forwarding"
Be sure after you set this that you scroll back up to 'Session' and then 'save'

Specifying Parallel Environment on Google Compute Engine using Elasticluster

I recently created a Grid Engine cluster on Compute Engine using Elasticluster (http://googlegenomics.readthedocs.org/en/latest/use_cases/setup_gridengine_cluster_on_compute_engine/index.html).
I was wondering what is the appropriate command to run shared-memory multithreaded batch jobs on a cluster of Compute Engine virtual machine running Grid Engine.
In other words, what is the name (i.e. pe_name) of the Grid Engine parallel environment.
Let's say I want to run a job requesting 4 cpus on 1 node, what would be the right qsub command.
So far I tried the following command:
qsub -cwd -l h_vmem=800G -pe smp 6 run.sh
Unable to run job: job rejected: the requested parallel environment "smp" does not exist.
qsub -cwd -l h_vmem=800G -pe omp 6 run.sh
Unable to run job: job rejected: the requested parallel environment "omp" does not exist.
Thank you for your help!
I don't believe that Elasticluster's Ansible playbook includes a parallel environment. You can see the main configuration run on the master here:
https://github.com/gc3-uzh-ch/elasticluster/blob/master/elasticluster/providers/ansible-playbooks/roles/gridengine/tasks/master.yml
I believe you can simply connect to the master and issue the "add parallele environment" command:
$ qconf -ap smp
and write a configuration file like:
pe_name smp
slots 9999
user_lists NONE
xuser_lists NONE
start_proc_args /bin/true
stop_proc_args /bin/true
allocation_rule $fill_up
control_slaves FALSE
job_is_first_task FALSE
urgency_slots min
accounting_summary FALSE
and then modify the queue configuration for all.q:
$ qconf -mq all.q
...
pe_list make smp
...
I would also suggest filing an issue with Elasticluster here:
https://github.com/gc3-uzh-ch/elasticluster/issues
I would expect that someone has already done this in a fork of Elasticluster and may be able to provide a pull request to the master fork.
Hope that helps.
-Matt

Nodejs/Strongloop: working upstart config example

After update strongloop to v2.10 slc stops writing logs.
Also I couldn't make the app to start in production mode.
/etc/init/app.conf
#!upstart
description "StrongLoop app"
start on startup
stop on shutdown
env NODE_ENV=production
script
exec slc run /home/ubuntu/app/ \
-l /home/ubuntu/app/app.log \
-p /var/run/app.pid
end script
Can anybody check my upstart config or provide another working copy?
Are you were writing the pid to a file so that you can use it to send SIGUSR2 to the process to trigger log re-opening from logrotate?
Assuming you are using Upstart 1.4+ (Ubuntu 12.04 or newer), then you would be better off letting slc run log to its stdout and let Upstart take care of writing it to a file so that log rotation is done for you:
#!upstart
description "StrongLoop app"
start on startup
stop on shutdown
# assuming this is /etc/init/app.conf,
# stdout+stderr logged to: /var/log/upstart/app.log
console log
env NODE_ENV=production
exec /usr/local/bin/slc run --cluster=CPUs /home/ubuntu/app
The log rotation for "free" is nice, but the biggest benefit to this approach is Upstart can log errors that slc run reports even if they are a crash while trying to set up its internal logging, which makes debugging a lot easier.
Aside from what it means to your actual application, the only effect NODE_ENV has on slc run is to set the default number of cluster workers to the number of detected CPU cores, which literally translates to --cluster=CPUs.
Another problem I find is the node/npm path prefix not being in the $PATH as used by Upstart, so I normally put the full paths for executables in my Upstart jobs.
Service Installer
You could also try using strong-service-install, which is a module used by slc pm-install to install strong-pm as an OS service:
$ npm install -g strong-service-install
$ sudo sl-svc-install --name app --user ubuntu --cwd /home/ubuntu/app -- slc run --cluster=CPUs .
Note the spaces around the -- before slc run

Resources