I have some strange Pytest behaviour I can't explain and was wondering if someone had already seen this before.
So we're running Pytest with 4 treads - pytest -n 4 tests/e2e.
Now, sometimes, it runs the same test on 2 different workers, like here:
[gw1] [ 40%] PASSED tests/e2e/test_get_connector.py::test_invoke_api_with_permission_not_found
..... more tests....
[gw3] [100%] PASSED tests/e2e/test_get_connector.py::test_invoke_api_with_permission_not_found Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/home/jenkins/.virtualenvs/lib/python3.8/site-packages/jsii/_kernel/providers/process.py", line 284, in stop
self._process.stdin.close()
BrokenPipeError: [Errno 32] Broken pipe
You can see that the same test is run 2 times on 2 different worker threads.
The second run of that test fails with a Broken Pipe error, but the whole test run still succeeds (the second test is also marked as PASSED, as you can see) and the Jenkins job also succeeds.
Anyone has any idea why this is happening?
This behaviour makes it hard to trust the pipeline..
Thanks!
Related
The command gitlab-runner lets you "test" a gitlab job locally. However, the local run of a job seems to have the same problem as a gitlab job run in gitlab CI: The output is not immediate!
What I mean: Even if your code/test/whatever produces printed output, it is not shown immediately in your log or console.
Here is how you can reproduce this behavior (on Linux):
Create a new git repository
mkdir testrepo
cd testrepo
git init
Create file .gitlab-ci.yml with the following content
job_test:
image: python:3.8-buster
script:
- python tester.py
Create a file tester.py with the following content:
import time
for index in range(10):
print(f"{time.time()} test output")
time.sleep(1)
Run this code locally
python tester.py
which produces the output
1648130393.143866 test output
1648130394.1441162 test output
1648130395.14529 test output
1648130396.1466148 test output
1648130397.147796 test output
1648130398.148115 test output
1648130399.148294 test output
1648130400.1494567 test output
1648130401.1506176 test output
1648130402.1508648 test output
with each line appearing on the console every second.
You commit the changes
git add tester.py
git add .gitlab-ci.yml
git commit -m "just a test"
You start the job within a gitlab runner
gitlab-runner exec docker job_test
....
1648130501.9057398 test output
1648130502.9068272 test output
1648130503.9079702 test output
1648130504.9090931 test output
1648130505.910158 test output
1648130506.9112566 test output
1648130507.9120533 test output
1648130508.9131665 test output
1648130509.9142723 test output
1648130510.9154003 test output
Job succeeded
Here you get essentially the same output, but you have to wait for 10 seconds and then you get the complete output at once!
What I want is to see the output as it happens. So like one line every second.
How can I do that for both, the local gitlab-runner and the gitlab CI?
In the source code, this is controlled mostly by the clientJobTrace's updateInterval and forceSendInterval properties.
These properties are not user-configurable. In order to change this functionality, you would have to patch the source code for the GitLab Runner and compile it yourself.
The parameters for the job trace are passed from the newJobTrace function and their defaults (where you would need to alter the source) are defined here.
Also note that the UI for GitLab may not necessarily get the trace in realtime, either. So, even if the runner has sent the trace to GitLab, the javascript responsible for updating the UI only polls for trace data every ~4 or 5 seconds.
You can poll gitlab web for new log lines as fast as you can:
For running job, use url like: https://gitlab.example.sk/grpup/project/-/jobs/42006/trace It will send you a json structure with lines of log file, offset, size and so on. You can have a look at documentation here: https://docs.gitlab.com/ee/api/jobs.html#get-a-log-file
Sidenote: you can use undocumented “state” parameter from response in subsequent request to get only new lines (if any). This is handy.
Through, this does not affect latency of arrival newlines from actual job from runner to gitlab web/backend. See sytech answer for this question.
This answer should help, when there is configured redis cache, incremental logging architecture, and someone wants to get logs from currently running job in "realtime". Polling is still needed through.
Some notes can be found also on forum: https://forum.gitlab.com/t/is-there-an-api-for-getting-live-log-from-running-job/73072
Error while running selenium python code on linux VM '[Errno 26] Text file busy: '/home/*/.wdm/drivers/chromedriver/83.0.4103.39/linux64/chromedriver''
I have running Selenium automated test of UI on every 1 hour through script on Linux VM but getting this error recently(from last 3-4 days),
'[Errno 26] Text file busy: '/home/*****/.wdm/drivers/chromedriver/83.0.4103.39/linux64/chromedriver''
Can someone faced this error ever before, seems it said test file busy but before 3-4 days it was working properly with same script since beginning.(nothing has been change in script).
Is it error with OS-IO, if yes/no how to fix this issue?
Try killing first the process and then run your selenium code:
kill -9 `pgrep -f home/xxxxx/.wdm/drivers/chromedriver`
we are sitting behind a firewall and try to run a docker image (cBioportal). The docker itself could be installed with a proxy but now we encounter the following issue:
Starting validation...
INFO: -: Unable to read xml containing cBioPortal version.
DEBUG: -: Requesting cancertypes from portal at 'http://cbioportal-container:8081'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Error occurred during validation step:
Traceback (most recent call last):
File "/cbioportal/core/src/main/scripts/importer/validateData.py", line 4491, in request_from_portal_api
response.raise_for_status()
File "/usr/local/lib/python3.5/dist-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: http://cbioportal-container:8081/api-legacy/cancertypes
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/bin/metaImport.py", line 127, in <module>
exitcode = validateData.main_validate(args)
File "/cbioportal/core/src/main/scripts/importer/validateData.py", line 4969, in main_validate
portal_instance = load_portal_info(server_url, logger)
File "/cbioportal/core/src/main/scripts/importer/validateData.py", line 4622, in load_portal_info
parsed_json = request_from_portal_api(path, api_name, logger)
File "/cbioportal/core/src/main/scripts/importer/validateData.py", line 4495, in request_from_portal_api
) from e
ConnectionError: Failed to fetch metadata from the portal at [http://cbioportal-container:8081/api-legacy/cancertypes]
Now we know that it is a firewall issue, because it works when we install it outside the firewall. But we do not know how to change the firewall yet. Our idea was to look up the files and lines which throw the errors. But we do not know how to look into the files since they are within the docker.
So we can not just do something like
vim /cbioportal/core/src/main/scripts/importer/validateData.py
...because ... there is nothing. Of course we know this file is within the docker image, but like i said we dont know how to look into it. At the moment we do not know how to solve this riddle - any help appreciated.
maybe you still might need this.
You can access this python file within the container by usingdocker-compose exec cbioportal sh or docker-compose exec cbioportal bash
Then you can us cd, cat, vi, vim or else to access the given path in your post.
I'm not sure which command you're actually running but when I did the import call like
docker-compose run --rm cbioportal metaImport.py -u http://cbioportal:8080 -s study/lgg_ucsf_2014/lgg_ucsf_2014/ -o
I had to replace the http://cbioportal:8080 with the servers ip address.
Also notice that the studies path is one level deeper than in the official documentation.
In cbioportal behind proxy the study import is only available in offline mode via:
First you need to get inside the container
docker exec -it cbioportal-container bash
Then generate portal info folder
cd $PORTAL_HOME/core/src/main/scripts ./dumpPortalInfo.pl $PORTAL_HOME/my_portal_info_folder
Then import the study offline. -o is important to overwrite despite of warnings.
cd $PORTAL_HOME/core/src/main/scripts
./importer/metaImport.py -p $PORTAL_HOME/my_portal_info_folder -s /study/lgg_ucsf_2014 -v -o
Hope this helps.
I recently ported my scripts from 2.x to 3.x. During production runs through automation (rundeck) we are seeing errors caused by the logger not handling blocking I/O. Any ideas how to resolve would be great.
Ubuntu 18.04.1 LTS
Python 3.6.7
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.6/logging/__init__.py", line 998, in emit
self.flush()
File "/usr/lib/python3.6/logging/__init__.py", line 978, in flush
self.stream.flush()
BlockingIOError: [Errno 11] write could not complete without blocking
I was getting the same error on CI builds. It looks like it was a capacity issue with the output stream. After reducing the log output, the errors went away.
I recently faced the error while building my Docker image using docker-compose in CI and I found one Workaround maybe that will help someone:
the Error:
BlockingIOError: [Errno 11] write could not complete without blocking
if you do not want to lose any logs , you can send all the logs to file and save it as an artifact , tested on Bamboo and Jenkins:
docker-compose build --no-cache my_image > myfile.txt
if you do not want the logs:
docker-compose build --no-cache my_image > /dev/null
I'm trying to run the evaluation code in
http://lenskit.org/documentation/evaluator/quickstart/
but, after one minute or so, it finishes with an exception:
Exception in thread "main" Target "eval" does not exist in the project "null".
at org.apache.tools.ant.Project.tsort(Project.java:1912)
at org.apache.tools.ant.Project.topoSort(Project.java:1820)
at org.grouplens.lenskit.eval.EvalProject.executeTargets(EvalProject.java:168)
at org.grouplens.lenskit.eval.cli.EvalCLI.run(EvalCLI.java:91)
at org.grouplens.lenskit.eval.cli.EvalCLI.main(EvalCLI.java:127)
I just downloaded and unzipped ml-100k.zip, put the eval.groovy script in the same directory and run
lenskit-eval eval
I'm using lenskit 2.2 on Java 7.
What am I missing?
Cheers!!
The issue is the second eval - it tells the LensKit evaluator to try to run the eval target eval, which doesn't exist.
Either run:
lenskit eval
which is recommended, or the deprecated
lenskit-eval