The output limit of a job log is truncated at 500KiB. How can I increase that?
output_limit in the config.toml did not seem to change that. I think that is something different.
The problem as shown in an image:
Related
Is that possible to save/redirect Gitlab job traces(like for example this)? My Gitlab runners work in docker containers and I would like to save logs from executed jobs to syslog on the machine, then forward them to Graylog server. I was thinking about some python script catching and redirecting logs, but I don't know how to start and I am not sure if it's a good lead.
I know that job traces are stored on Gitlab Server, but I don't have access to it and need to do this a different way.
I hope my explanation is understandable, I am new to Gitlab CI.
Job traces aren't ever stored in files on the gitlab runner. They are only transmitted to the GitLab server and there is no option for local log propagation/redirection when using any of the official executors.
If you don't have direct access to the GitLab server storage, you can use the jobs API to retrieve trace logs from the GitLab server. You could then use any normal mechanisms for forwarding those logs to graylog, like a forwarder agent or whatever.
I have deployed a pod in kubernetes cluster that run a python script.
The problem is i want to force the k8s to stop the container after the script complete his job and not to re-create another pod.
To be aware that i have tried to use kind:job but it doesn't fulfill my need.
I tried two types of kind, job and deployments.
With the deployment the pod always show status first completed after that crush with crashloopbackof error.
With the job the pod always show the status completed but i don't have the possibility to re-excute it with an automated way
Do you have any suggestions about that?
I have posted community wiki answer to summarise the topic.
User Jonas has posted great suggestions:
A kind Job does exactly this. Use Job and your problem is solved.
If you deploy with kubectl create -f job.yaml and your job has a generateName: instead of name:, a new Job will be created each time.
For more information look at the documentation about Jobs. See also information about Gnerated values.
I am planning to store some of the key/values inside GitLab CI/CD variables? I found in the document that each value is limited to 1000 characters. I am wondering if there is any limitation for the number of keys/values as well?
For the default plan the limit of variables in self hosted instance would be 25 according to https://docs.gitlab.com/ee/administration/instance_limits.html#number-of-instance-level-variables
But it states that you can update this number
You can now see all limits for CI/CD with See GitLab 15.0 (May 2022):
Show instance CI/CD limits in /help
Instance Administrators can set a number of CI/CD limits in the Admin Area of their instance. It is hard for users without administrator access to know what the limits are, especially when pipelines are failing because they bump into limits they didn’t know existed.
Now the CI/CD related limits for the instance can be seen on the instance configuration page found at /help/instance_configuration.
Thanks to #wwwjon for this contribution!
See Documentation and Issue.
GitLab 15.7 (Dec. 2022) brings a precise answer:
Limit the number of project or group CI/CD variables to 200
If your instance has project or group maintainers adding too many CI/CD variables, it could use a lot of resources to process all the variables for each pipeline.
To ensure your pipelines continue to run efficiently, we have added a limit of 200 variables per group or project.
Projects and groups that already have over 200 CI/CD variables can continue to run pipelines as before, but will not be able to add any more variables.
We recommend reducing the number of CI/CD variables down to less than 200 per group or project for optimal performance.
See Documentation and Issue.
It would be 25 according to the default plan but you can change it. Hope that answers your question!
https://docs.gitlab.com/ee/administration/instance_limits.html#number-of-instance-level-variables
Well, I see now that Tolis already clarified it but I will still post this :)
I use CruiseControl.Net for continuous integration and I would like to read the log output of the current project in real time. For example, if it is running a compile command, I want to be able to see all the compile output so far. I can see where the log files are stored but it looks like they are only created once the project finishes. Is there any way to get the output in real time?
The CCTray app will allow you to see a snapshot of the last 5 or so lines of output of any command on a regular interval.
It's not a live update as that would be too resource intensive, as would be a full output of the log to-date.
Unless you write something to capture and store the snapshots you're out of luck. Doing this also presents to possibility of missing messages that appear between snapshots, so it would not be entirely reliable. It would however give you a slightly better idea of what is going on.
You can run ccnet.exe as a command line application instead of running ccservice as a Windows service. It will output to the terminal as it runs. It's useful for debugging.
In high performance computing is crucial to have a code tested against many different architectures/compilers: from a laptop to a supercomputer.
Assuming that we have
N testing machines/workers (each one running gitlab-ci-runner);
M tests,
what shall be the correct layout of .gitlab-ci.yml to ensure that each of the N machines runs all the tests?
Looks to me that adding just more workers ends up in a round-robin like assignment of the jobs.
Thanks for your help.
You could use tags in your .gitlab-ci.yml and on the runners to distribute your tests to all mashines you want. Gitlab CI is very open to those use cases. I assume that you don't use docker for testing.
To accomplish your goal, do following steps:
Install your gitlab-ci-multi-runners on your N mashines BUT if the runner ask you for tags, tag the mashine with a specifc name e.g. "MashineWithManyCPUsWhatever". Please use the gitlab-ci-multi-runner register command to do so. You can alternatively change the tags in Gitlab on the administration view after registration.
The type of the runners should be "shared" not "specific".
When you installed and tagged every runner on your mashines, tag you jobs.
Example:
Every job will now be excecuted on the specific mashine with the specific tag.
For more Information see Gitlab CI Documentation