How I can config the fluentd Proxy for install plugins? - linux

I can't install a plugin at fluentd. i've set the proxy-variable at the System (rhel) but it have no influence.
$ td-agent --gemfile grok-plugin/Gemfile
fatal: Not a git repository (or any of the parent directories): .git
Fetching source index from https://rubygems.org/
Retrying source fetch due to error (2/3): Bundler::HTTPError Could not fetch specs from https://rubygems.org/
Retrying source fetch due to error (3/3): Bundler::HTTPError Could not fetch specs from https://rubygems.org/
Could not fetch specs from https://rubygems.org/
$ echo $HTTP_PROXY
http://proxy.de:port
$ echo $HTTPS_PROXY
http://proxy.de:port
$ echo $https_proxy
http://proxy.de:port
$ echo $http_proxy
http://proxy.de:port
Please can you help me to set the proxy for the gem-install?

I find the solution. I have an other Problem and it gave me an approach. In rhel6 you must write the proxy without http://, then it works. (Write the proxy without protocol, is only the solution for this problem)

Related

keytar.node causing issues in Buildkite pipeline

I'm trying to create a CI/CD pipeline using buildkite where the build steps runs on a Virtual Machine (Ubuntu). However, an npm package called keytar which is a dependency of tedious library is causing issues in the pipeline.
When running one of the steps from using the buildkite-agent, it tries to clean up an existing repository. And during the command git clean -fxdq it fails to remove keytar.node file due to a Permission denied Error.
Here's a full log of the step:
> Running global environment hook
> Setting up plugins
> Preparing working directory
$ cd /var/lib/buildkite-agent/builds/build-server-1/someorg/some-repo
# Host "bitbucket.org" already in list of known hosts at "/var/lib/buildkite-agent/.ssh/known_hosts"
$ git remote set-url origin git#bitbucket.org:someorg/some-repo.git
$ git clean -fxdq
warning: failed to remove node_modules/keytar/build/Release/keytar.node: Permission denied
# Removing /var/lib/buildkite-agent/builds/build-server-1/someorg/some-repo
⚠️ Warning: Checkout failed! Error running `/usr/bin/git clean -fxdq`: exit status 1 (Attempt 1/3 Retrying in 2s)
# Host "bitbucket.org" already in list of known hosts at "/var/lib/buildkite-agent/.ssh/known_hosts"
$ git clone -v -- git#bitbucket.org:someorg/some-repo.git .
fatal: destination path '.' already exists and is not an empty directory.
# Removing /var/lib/buildkite-agent/builds/build-server-1/someorg/some-repo
⚠️ Warning: Checkout failed! Error running `/usr/bin/git clone -v -- git#bitbucket.org:someorg/some-repo.git .`: exit status 128 (Attempt 2/3 Retrying in 2s)
# Host "bitbucket.org" already in list of known hosts at "/var/lib/buildkite-agent/.ssh/known_hosts"
$ git clone -v -- git#bitbucket.org:someorg/some-repo.git .
fatal: destination path '.' already exists and is not an empty directory.
# Removing /var/lib/buildkite-agent/builds/build-server-1/someorg/some-repo
⚠️ Warning: Checkout failed! Error running `/usr/bin/git clone -v -- git#bitbucket.org:someorg/some-repo.git .`: exit status 128 (Attempt 3/3 Retrying in 2s)
# Skipping artifact upload, no checkout
🚨 Error: Error running `/usr/bin/git clone -v -- git#bitbucket.org:someorg/some-repo.git .`: exit status 128
Upon examining the permissions on the folder /some-repo using the ls -l command the permissions seem to be drwxr-xr-x and the owner is buildkite-agent. I'm stumped as how this file does not get deleted while all other files get deleted without any issue.

Jenkins git fetch returned status code 143

I'm facing the problem with Jenkins with fetching git repo on Linux. I attach the stack trace. I have SSH gitlab connection configured with a private key and a passphrase. The configuration is made in Credentials/System/Global credentials (unrestricted). The path to the private key is /home/jenkins/.jenkins/secrets/my_private_key.pub. Owner and a group: jenkins:jenkins, where jenkins is the user making the installation. Permissions: 600.
The private key has been checked with the passphrase using ssh from the command line. The git fetch command used by jenkins has also been checked with command line. Both works.
Repository URL format is like:
git#github.com:org-name/project.git (short notation for ssh protocol)
Environment:
* CentOS 7 64 bit
* openjdk version "1.8.0_131" 64 bit
* Jenkins ver. 2.67
* Git client plugin 2.4.6Git plugin 3.3.0 (3.3.1 is not helping too)
* GIT server Plugin 1.7
* GitLab Plugin 1.4.5 (1.4.6 is not helping too)
* SCM API Plugin 2.1.1
* Credentials Plugin 2.1.14
* Jenkins is run directly
* Jenkins was installed with yum installer
* Browser: Google Chrome 59
* GitLab Community Edition 9.3.5
* git version 1.8.3.1
Stack trace:
started by user Jenkins Admin
Building in workspace /home/jenkins/.jenkins/workspace/my_project
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url git#gitlab.repo:XYZ/docs/project.git #
timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
No valid HEAD. Skipping the resetting
> git clean -fdx # timeout=10
Fetching upstream changes from git#gitlab.repo:XYZ/docs/project.git
> git --version # timeout=10
using GIT_SSH to set credentials
> git fetch --no-tags --progress git#gitlab.repo:XYZ/docs/project.git
+refs/heads/*:refs/remotes/origin/* --depth=1
ERROR: Timeout after 10 minutes
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git#gitlab.repo:XYZ/docs/project.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:809)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1076)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1107)
at hudson.scm.SCM.checkout(SCM.java:495)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1212)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:560)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:485)
at hudson.model.Run.execute(Run.java:1735)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:415)
Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --progress git#gitlab.repo:XYZ/docs/project.git +refs/heads/*:refs/remotes/origin/* --depth=1" returned status code 143:
stdout:
stderr:
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1903)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1622)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:71)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:348)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:807)
... 11 more
ERROR: null
Finished: FAILURE
I found a solution. Running Jenkins as a service with systemctl resolved the problem, instead of running Jenkins with java -jar jenkins.war.
To give more details:
According to the documentation, Jenkins is started by
java -jar jenkins.war
And it was visible as running in the system when I checked it with:
ps aux | grep jenkins
However, on CentOS 7 the suggested way of running Jenkins is to run it as a daemon with systemctl command.
When I checked it with
sudo systemctl status jenkins.service
it was inactive.
So I started Jenkins with
sudo systemctl start jenkins.service
and it worked.

How do I point a BitBake recipe to a local file / Yocto build fails to fetch sources for libtalloc

I'm trying to build Yocto for Raspberry Pi3, with console-image, and it gives me some build errors, most I have been able to resolve with
bitbake -c cleansstate libname
bitbake libname
However, now it got to libtalloc and it can't do_fetch the source files.
I went to the URL of the sources, and I was able to download the exact tar.gz archive it was trying to fetch. i.e. http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz
I even put it into /build/downloads folder.
But when I try to bitbake, it keeps giving me the same errors
Is there a way I can configure the build process to always fetch with http or wget, it seems that the these scripts are all broken, because it cant fetch a file that exists.
Thanks,
Here is the full printout:
WARNING: libtalloc-2.1.8-r0 do_fetch: Failed to fetch URL http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz, attempting MIRRORS if available
ERROR: libtalloc-2.1.8-r0 do_fetch: Fetcher failure: Fetch command export DBUS_SESSION_BUS_ADDRESS="unix:abstract=/tmp/dbus-ATqIt180d4"; export SSH_AUTH_SOCK="/run/user/1000/keyring-Ubo22d/ssh"; export PATH="/home/dmitry/rpi/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/bin/python-native:/home/dmitry/poky-morty/scripts:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/bin/arm-poky-linux-gnueabi:/home/dmitry/rpi/build/tmp/sysroots/raspberrypi2/usr/bin/crossscripts:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/sbin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/usr/bin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/sbin:/home/dmitry/rpi/build/tmp/sysroots/x86_64-linux/bin:/home/dmitry/poky-morty/scripts:/home/dmitry/poky-morty/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"; export HOME="/home/dmitry"; /usr/bin/env wget -t 2 -T 30 -nv --passive-ftp --no-check-certificate -P /home/dmitry/rpi/build/downloads 'http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz' --progress=dot -v failed with exit code 4, output:
--2017-01-24 12:35:19-- http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz
Resolving samba.org (samba.org)... 144.76.82.156, 2a01:4f8:192:486::443:2
Connecting to samba.org (samba.org)|144.76.82.156|:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.
--2017-01-24 12:35:20-- (try: 2) http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz
Connecting to samba.org (samba.org)|144.76.82.156|:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Giving up.
ERROR: libtalloc-2.1.8-r0 do_fetch: Fetcher failure for URL: 'http://samba.org/ftp/talloc/talloc-2.1.8.tar.gz'. Unable to fetch URL from any source.
ERROR: libtalloc-2.1.8-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/dmitry/rpi/build/tmp/work/cortexa7hf-neon-vfpv4-poky-linux-gnueabi/libtalloc/2.1.8-r0/temp/log.do_fetch.80102
ERROR: Task (/home/dmitry/poky-morty/meta-openembedded/meta-networking/recipes-support/libtalloc/libtalloc_2.1.8.bb:do_fetch) failed with exit code '1'
Is there a way I can configure the build process to always fetch with http or wget, it seems that the these scripts are all broken, because it cant fetch a file that exists.
The scripts already use both wget and http. They're also not really broken, the people maintaining the samba download servers just changed several things in the past week: I believe the libtalloc recipes main SRC_URI just needs to be changed to https://download.samba.org/pub/talloc/talloc-${PV}.tar.gz (the current canonical samba download server).
I'm sure meta-oe maintainer would appreciate a patch if this is indeed the case.
I applied the following patch to meta-openembedded and got it built. There are several samba links already broken.
http://pastebin.com/0uTnAY4g
Regards,
M.

gitlab 8.2.0 git not working

I installed gitlab 8.2.0 from source and followed the instructions on https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.
When I run the checks with
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
I get the following error:
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: FAILED. code: 301
gitlab-shell self-check failed
Try fixing it:
Make sure GitLab is running;
Check the gitlab-shell configuration file:
sudo -u git -H editor /home/git/gitlab-shell/config.yml
Please fix the error above and rerun the checks.
Checking GitLab Shell ... Finished
Here is my config.yml file from /home/git/gitlab-shell:
user: git
gitlab_url: "http://www.example.com/gitlab"
http_settings:
#ca_path: /etc/ssl/certs
self_signed_cert: true
repos_path: "/home/git/repositories/"
auth_file: "/home/git/.ssh/authorized_keys"
redis:
bin: "/usr/bin/redis-cli"
namespace: resque:gitlab
socket: "/var/run/redis/redis.sock"
log_level: INFO
audit_usernames: false
Note that I am using the relative URL feature to run gitlab on www.example.com/gitlab. Also the server is behind a reverse proxy that terminates https to http if this should be of any relevance for this problem.
I also cant' push or pull any repositories. When I try to I get the following error:
git pull https://www.example.com/gitlab/test/testproject.git
Username for 'https://www.example.com': test
Password for 'https://test#www.example.com':
fatal: Couldn't find remote ref HEAD
Unexpected end of command stream
I also can't add any files to any project via the webUI. If I try to do so I get the following error
Commit was rejected by pre-receive hook
The log for this from production.log looks as follows:
Sarted POST "/gitlab/test/testprojekt/create/master" for 147.86.8.115 at 2015-11-26 15:24:43 +0100
Processing by Projects::BlobController#create as JSON
Parameters: {"new_branch"=>"master", "create_merge_request"=>"1", "commit_message"=>"test", "file"=># <ActionDispatch::Http::UploadedFile:0x007f776230faf8 #tempfile=# <Tempfile:/tmp/RackMultipart20151126-7584-308za1>, #original_filename="test2.txt", #content_type="text/plain", #headers="Content- Disposition: form-data; name=\"file\"; filename=\"test2.txt\"\r\nContent-Type: text/plain\r\n">, "namespace_id"=>"test", "project_id"=>"testprojekt", "id"=>"master"}
Completed 200 OK in 628ms (Views: 0.3ms | ActiveRecord: 3.3ms)
Started GET "/gitlab/test/testprojekt/new/master" for 147.86.8.115 at 2015-11-26 15:24:44 +0100
Processing by Projects::BlobController#new as HTML
Parameters: {"namespace_id"=>"test", "project_id"=>"testprojekt", "id"=>"master"}
Completed 200 OK in 85ms (Views: 49.6ms | ActiveRecord: 6.0ms)
I think you could try switching gitlab_url: "http://www.example.com/gitlab" to https -> gitlab_url: "https://www.example.com/gitlab"
Not sure if its a complete fix but its a recomendations.
EDIT:
config.yml line 4 should look like
ca_path: "/etc/ssl/certs"

GIT: Can't Push (Strange Config Issue)

I'm on a fresh install of Linux Mint.
I'm getting the following error when trying to push from any repository:
error: Malformed value for push.default: simple
error: Must be one of nothing, matching, tracking or current.
fatal: bad config file line 8 in /home/leng/.gitconfig
fatal: Could not read from remote repository.
This is very odd, because I definitely have a version that supports the simple push behavior.
The output of git --version is git version 1.8.3.2.
The contents of ~/.gitconfig:
[user]
name = My Name
email = MyEmail#website.com
[color]
ui = true
[push]
default = simple
Here's where it gets creepy.
If I change the behavior to matching (or to nothing, tracking, or current, for that matter), then attempt to push, I get the same exact error message. How is that possible? Is it caching the config somehow? I've even tried rebooting. I've even tried purging GIT completely from the system (and deleting ~/.gitconfig) then reinstalling it.
If I delete the [push] section completely from the .gitconfig file (or if I delete the file entirely), then try to push, then I get this:
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:
git config --global push.default matching
To squelch this message and adopt the new behavior now, use:
git config --global push.default simple
See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)
error: Malformed value for push.default: simple
error: Must be one of nothing, matching, tracking or current.
fatal: bad config file line 8 in /home/leng/.gitconfig
fatal: Could not read from remote repository.
...so it appears to be both acknowledging that I haven't chosen a pushing behavior, but then also saying that I've chosen an unsupported behavior. What on earth is going on here?
I even get the error if I delete ~/.gitconfig completely.
Can anyone help me out with this witchcraft?
Thanks!
EDIT:
Here is a .git/config file requested:
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
url = ssh://{my remote repo}
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
Okay, so I fixed it, but the method is absolute witchcraft.
I tried to isolate the problem by purging GIT, deleting the config file, reinstalling GIT, then creating a local bare repository, then cloning it, then attempting to push from there. Pretty much like this:
sudo apt-get purge git-core
rm -f ~/.gitconfig
sudo apt-get install git-core
cd /git
mkdir foo
cd foo
git init --bare
cd /var/www
git clone /git/foo
cd foo
touch blah.txt
git add -A
git config --global user.name "Name"
git config --global user.email "user#email.com"
git commit -m "Blah"
git push
...same exact error message, no change there. (Still some serious witchcraft.)
Then, I deleted one of my repositories that doesn't have a local origin (it connects to its origin via SSH) and cloned the repository anew after deleting it (with a fresh git clone ssh://... command).
I got an error from the clone command:
remote: Malformed value for push.default: simple
remote: Must be one of nothing, matching, tracking or current.
Ah ha! Now it says remote instead of error. So the remote doesn't support this behavior. (That doesn't explain why the error persists on local-only repositories with local origins, then, though.)
So I then SSH'ed into the remote server and updated the git-core there to the latest version, re-attempted to clone the repository from my local machine, and it worked.
Now, I can finally git push. Insanely, this also fixed it so I can git push from the entirely local /var/www/foo to the also entirely local /git/foo (the local origin bare repository). SSH'ing into this remote server and updating it somehow - WITCHCRAFT - fixed my local machine's error.
Why on earth the entirely local repos care about an entirely different machine's GIT version is... beyond me. How utterly, utterly insane.
I had the same error message on git push.
For me it turned out that the remote user's git was an older version (1.7.2.5),
and I had recently updated the remote ~/.gitconfig to include:
[push]
default = simple
The solution was to remove this setting from the remote's configuration.
Since it seems other people are having this issue, and I found a solution HERE, I thought I'd post the solution that worked for me.
IN SHORT:
The solution I found was at this page. Evidently the best solution is to upgrade to a newer version of Git (if possible). That was not an option for me, however. From a local machine, I typed the following command:
git config -–global push.default upstream
This got rid of the Malformed value for push.default: simple error I had been getting. I'm not entirely sure what upstream does, however.
MY CONTEXT (for comparison): I had an empty (bare) repository on a remote computer, and I had a few repositories on a couple "local" workstations. I pull from the remote repository, do some work, and then push my work to the remote repository. Pushing/pulling was accomplished via SSH. Most of the time, while working on a local machine, pushing/pulling would result in the error described above.
In short, before the fix, I had the following ~/.gitconfig file on the remote machine:
[user]
name = Foo Bar
email = FooBarPerson#email.com
[diff]
external = /Users/foobar/bin/git-diff-cmd.sh
[color]
diff = auto
status = auto
branch = auto
[push]
default = simple
After entering in the above command, my ~/.gitconfig file on the remote machine changed to:
[user]
name = Foo Bar
email = FooBarPerson#email.com
[diff]
external = /Users/foobar/bin/git-diff-cmd.sh
[color]
diff = auto
status = auto
branch = auto
[push]
default = upstream
Version information:
Remote machine (repository location): 1.9.4
My laptop: 1.8.5.2 (Apple Git-48)
Other computer I work on: 1.7.7.4
Here's another site that may be useful to some people:
http://www.lorrin.org/blog/2011/10/03/argumentless-git-pull-and-git-push/comment-page-1/

Resources