My command in python is
feature_file_folder -f allure_behave.formatter:AllureFormatter -o target/allure-results --no-capture --no-capture-stderr
This runs all the feature files but if i want to run only 2 feature files out of 10 i am having problems i can run 1 or all feature file.
You can mention feature file names while executing.
behave -f allure_behave.formatter:AllureFormatter \
-o target/allure-results \
--no-capture \
--no-capture-stderr \
example1.feature example2.feature
This is what tags are used for. Lets say, hypothetically, the two you want to run are related to logging into a system:
login-from-main-page.feature:
#login-tests
Feature: Test logging in from the main page
Scenario: ...
login-from-mobile.feature
#login-tests
Feature: Test logging in from iOS App
Scenario: ...
Then you would run behave, specifying those tags:
feature_file_folder -f allure_behave.formatter:AllureFormatter \
-o target/allure-results \
--no-capture \
--no-capture-stderr \
--tags login-tests
Related
I am training a Tensorflow 2 model on Google Cloud Platform using the Vertex AI service. I build a docker image, upload it to Container Registry, then run it on Vertex.
Vertex automatically submits the output to Cloud Logging. Unfortunately it frequently loses some of the output. It seems to most often lose the output if the total output is short, or at the end of the output stream. It also seems that different log levels are treated differently, so for example level INFO might be collected but level WARNING lost. When I run the image locally I get everything.
It seems I can make the logs come through by dumping out a lot of extra output. I don't really want to do this for every log level. I tried various things to flush the logs which are shown in my script.
Is anyone aware of this or know why it is happening?
Some recreation test scripts are below for completeness.
Update: I tested the image on Google Cloud Run and the logs are collected correctly there (I still have to use Vertex though).
script.py
import logging
import time
logging.basicConfig()
logging.getLogger().setLevel(logging.INFO)
print('Test message')
logging.info("Info message")
logging.warning("Warning message")
print('Flushing print buffer.', flush=True)
logging.shutdown()
time.sleep(3.)
Dockerfile
FROM tensorflow/tensorflow:latest-gpu
WORKDIR /root
RUN apt-get install -y wget
RUN wget -nv \
https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz && \
mkdir /root/tools && \
tar xvzf google-cloud-sdk.tar.gz -C /root/tools && \
rm google-cloud-sdk.tar.gz && \
/root/tools/google-cloud-sdk/install.sh --usage-reporting=false \
--path-update=false --bash-completion=false \
--disable-installation-options && \
rm -rf /root/.config/* && \
ln -s /root/.config /config && \
# Remove the backup directory that gcloud creates
rm -rf /root/tools/google-cloud-sdk/.install/.backup \
ENV PATH $PATH:/root/tools/google-cloud-sdk/bin
RUN echo '[GoogleCompute]\nservice_account = default' > /etc/boto.cfg
COPY script.py /root/script.py
ENTRYPOINT ["python3", "script.py"]
I am trying to run a script inside a docker that was published by google.
The command I use mounts some datafiles onto the docker in a file called '/input' (inside the docker).
When I run the script, it says that it does not find the input file.
However, I do use the -v flag, and I ran a script that makes sure the input file is there (inside the docker).
So in summary - when I run
find /input -name "*.fasta"
It outputs:
/input/ucsc.hg19.chr20.unittest.fasta
As needed, but when I run the script, it says
./dv-quick-start: 19: ./dv-quick-start: --ref=/input/ucsc.hg19.chr20.unittest.fasta: not found
Full Script:
#!/bin/sh
BIN_VERSION="1.0.0"
INPUT_DIR="${PWD}/quickstart-testdata"
DATA_HTTP_DIR="https://storage.googleapis.com/deepvariant/quickstart-testdata"
OUTPUT_DIR="${PWD}/quickstart-output"
sudo docker run \
-v "${INPUT_DIR}":"/input" \
-v "${OUTPUT_DIR}":"/output" \
google/deepvariant:"${BIN_VERSION}" \
find /input -name "*.fasta"
sudo docker run \
-v "${INPUT_DIR}":"/input" \
-v "${OUTPUT_DIR}":"/output" \
google/deepvariant:"${BIN_VERSION}" \
/opt/deepvariant/bin/run_deepvariant \
--model_type=WGS \ **Replace this string with exactly one of the following [WGS,WES,PACBIO,HYBRID_PACBIO_ILLUMINA]**
--ref=/input/ucsc.hg19.chr20.unittest.fasta \
--reads=/input/NA12878_S1.chr20.10_10p1mb.bam \
--regions "chr20:10,000,000-10,010,000" \
--output_vcf=/output/output.vcf.gz \
--output_gvcf=/output/output.g.vcf.gz \
--intermediate_results_dir /output/intermediate_results_dir \ **This flag is optional. Set to keep the intermediate results.
Full output:
/input/ucsc.hg19.chr20.unittest.fasta
--ref is required.
Pass --helpshort or --helpfull to see help on flags.
./dv-quick-start: 19: ./dv-quick-start: --ref=/input/ucsc.hg19.chr20.unittest.fasta: not found
I feel there is some misunderstanding on my behalf, and I would appreciate any help.
Should more information be needed to answer the question, let me know.
You have some extraneous text in your shell script that's causing a problem. Delete the "replace this string" and "this flag is optional" text and all of the whitespace before them, making the \ the very last character on those lines.
In a shell script you can break commands across multiple lines using a \. But, the \ must be the absolute very last character in the line; if it's not, it escapes the character that comes after it.
# one line: ls -al $HOME
ls -al \
$HOME
# two lines: ls -al " " more text here; $HOME
ls -al \ more text here
$HOME
In your example you've left some explanatory text in
sudo docker run \
...\
--model_type=WGS \ **Replace this string with exactly one of the following [WGS,WES,PACBIO,HYBRID_PACBIO_ILLUMINA]**
# This is seen as a separate command
--ref=/input/ucsc.hg19.chr20.unittest.fasta \
...
Since the "Replace this string..." text makes the \ not be the absolute last character in the line, it causes the shell to break the command. You then get two commands, a docker run command without the --ref option and what looks like trying to run --ref=... as a separate command; that corresponds to the two errors you get.
I've built a script to automate a CMake build of OpenCV4. The relevant part of the script is written as:
install.sh
#!/bin/bash
#...
cd /home/pi/opencv
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
-D ENABLE_NEON=ON \
-D ENABLE_VFPV3=ON \
-D BUILD_TESTS=OFF \
-D OPENCV_ENABLE_NONFREE=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D BUILD_EXAMPLES=OFF ..
This part of the code is first executed from /home/pi/ directory. If I execute these lines within the cli they work and the cmake file is built without error. If I run this same code form a bash script it results in the cmake command resulting in -- Configuring incomplete, errors occurred!.
I believe this is similar to these two SO threads (here and here) in as much as they both describe situations where calling a secondary script from the first script creates a problem (or at least that's what I think they are saying). If that is the case, how can you start a script from /parent/, change to /child/ within the script, execute secondary script (CMake) as though executed from the /child/ directory?
If I've missed my actual problem - highlighting taht would be even more helpful.
Update with Full Logs
Output log for CMakeOutput.log and CMakeError.log as unsuccessfully run from the bash script.
When executed from the cli the successful logs are success_CMakeOutput.log and success_CMakeError.log
Update on StdOut
I looked through the files above and they look the same... Here is the failed screen output (noting the bottom lines) and the successful screen output.
You are running your script as the root user with the /root home directory, while the opencv_contrib directory is in /home/pi directory. The /home/pi is most probably the home directory of the user pi.
Update the:
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
With proper path to opencv_contrib. Either provide opencv_contrib in the home directory of the root user, if you aim to run the script as root, or provide full, non-dependent on HOME, path to opencv_contrib directory.
-D OPENCV_EXTRA_MODULES_PATH=/home/pi/opencv_contrib/modules \
I'm creating a FreeBSD port for an application (Cypress) that doesn't use Make; instead, it uses NPM:
npm run binary-build-linux
cd cli
npm run build
There are two options I can see:
Add a Makefile to the work directory as a patch.
Convince the upstream maintainers to take a Makefile that would be solely used by this port.
I'm wondering if there's a third option I've missed: modify my port's Makefile to run a series of shell commands in lieu of a Makefile? Having read the porter's handbook I can't see any way of doing that though.
You don't necessarily need to use make for example, this is port is using go in the do-build target: (check the Additional Build Targets, target-OPT-on and target-OPT-off:
do-build:
#cd ${WRKSRC}/src/github.com/${GH_ACCOUNT}/${GH_PROJECT}; \
${SETENV} ${MAKE_ENV} ${BUILD_ENV} GOPATH=${WRKSRC} go build -ldflags \
"-s -w -X main.version=${PORTVERSION}" -o immortal cmd/immortal/main.go;
#cd ${WRKSRC}/src/github.com/${GH_ACCOUNT}/${GH_PROJECT}; \
${SETENV} ${MAKE_ENV} ${BUILD_ENV} GOPATH=${WRKSRC} go build -ldflags \
"-s -w -X main.version=${PORTVERSION}" -o immortalctl cmd/immortalctl/main.go;
#cd ${WRKSRC}/src/github.com/${GH_ACCOUNT}/${GH_PROJECT}; \
${SETENV} ${MAKE_ENV} ${BUILD_ENV} GOPATH=${WRKSRC} go build -ldflags \
"-s -w -X main.version=${PORTVERSION}" -o immortaldir cmd/immortaldir/main.go;
This other port using node:
do-build:
#(cd ${WRKSRC}/public ; node ./bundler.js )
#(cd ${WRKSRC} ; go-bindata -o util/bindata.go -pkg util config.json db/migrations/ public/css public/html public/html/projects public/html/projects/repositories public/html/projects/inventory public/html/projects/templates public/html/projects/users public/html/projects/environment public/html/projects/keys public/html/users public/html/auth public/img public/js public/js/services public/js/controllers public/js/controllers/projects public/js/routes public/js/factories public/node_modules public/node_modules/lodash public/node_modules/lodash/fp public/node_modules/async public/node_modules/async/dist public/node_modules/async/internal public/vendor public/vendor/fontawesome public/vendor/fontawesome/less public/vendor/fontawesome/fonts public/vendor/sweetalert public/vendor/moment public/vendor/bootstrap public/vendor/bootstrap/fonts public/vendor/bootstrap/dist public/vendor/bootstrap/dist/css public/vendor/bootstrap/dist/fonts public/vendor/bootstrap/dist/js public/vendor/bootstrap/less public/vendor/bootstrap/less/mixins public/vendor/angular-loading-bar )
#(cd ${WRKSRC}/cli ; ${SETENV} ${MAKE_ENV} GOPATH=${WRKSRC} go build -o semaphore ./... )
Both ports use the BUILD_DEPENDS (check the Dependencies in the porter handbook)
BUILD_DEPENDS= ${LOCALBASE}/bin/go:lang/go \
${LOCALBASE}/bin/go-bindata:devel/go-bindata \
npm>=0:www/npm
I have installed gitlab community edition on my raspberry pi 3. Everything is working fine. But when the application is up there are 25 sidekiq threads. It's eating up my memory and I don't want so many threads.
I tried controlling by adding the file /opt/gitlab/embedded/service/gitlab-rails/config/sidekiq.yml.
# Sample configuration file for Sidekiq.
# Options here can still be overridden by cmd line args.
# Place this file at config/sidekiq.yml and Sidekiq will
# pick it up automatically.
---
:verbose: false
:concurrency: 5
# Set timeout to 8 on Heroku, longer if you manage your own systems.
:timeout: 30
# Sidekiq will run this file through ERB when reading it so you can
# even put in dynamic logic, like a host-specific queue.
# http://www.mikeperham.com/2013/11/13/advanced-sidekiq-host-specific-queues/
:queues:
- critical
- default
- <%= `hostname`.strip %>
- low
# you can override concurrency based on environment
production:
:concurrency: 5
staging:
:concurrency: 5
I have restarted the application many times and even ran "reconfigure". It's not helping. It's not considering the sidekiq.yml file at all.
Can anybody please let me know where I am going wrong?
i found your question by searching for a solution for the same problem. All i found doesn't work. So i tried bye myself and found the right place for reducing sidekiq from 25 to 5. I use the gitlab omnibus version. I think the path is idetical to yours:
/opt/gitlab/sv/sidekiq/run
In this file you find the following code:
#!/bin/sh
cd /var/opt/gitlab/gitlab-rails/working
exec 2>&1
exec chpst -e /opt/gitlab/etc/gitlab-rails/env -P \
-U git -u git \
/opt/gitlab/embedded/bin/bundle exec sidekiq \
-C /opt/gitlab/embedded/service/gitlab-rails/config/sidekiq_queues.yml \
-e production \
-r /opt/gitlab/embedded/service/gitlab-rails \
-t 4 \
-c 25
Change the last line to "-c 5". The result should look like this:
#!/bin/sh
cd /var/opt/gitlab/gitlab-rails/working
exec 2>&1
exec chpst -e /opt/gitlab/etc/gitlab-rails/env -P \
-U git -u git \
/opt/gitlab/embedded/bin/bundle exec sidekiq \
-C /opt/gitlab/embedded/service/gitlab-rails/config/sidekiq_queues.yml \
-e production \
-r /opt/gitlab/embedded/service/gitlab-rails \
-t 4 \
-c 5
Last but no least yout have to resart gitlab service
sudo gitlab-ctl restart
No idea, what happening on the gitlab update. I think i have to change this value again. It would be nice, if the gitlab developers add this option to gitlab.rb in /etc/gitlab directory.