I referred many solutions yet no luck. I have a linux automation which runs few gcloud commands with some conditions. I made this script with node js, but it is incredibly slow that I even finish it manually before the scrips completes the run.
Same with the gcloud commands when I connect to a cluster and kubectl commands when i query something.
Please help!!
It could be a DNS config error on WSL side. I hadthe same issue today, here's how I fixed it !
1. Checking the (deadly slow) response time
[tbg#~] time kubectl get deployments
No resources found in default namespace.
real 0m1.212s
user 0m0.151s
sys 0m0.050s
2. Checking the WSL/DNS configuration
[tbg#~] cat /etc/wsl.conf
[network]
generateResolvConf=false
[tbg#~] cat /etc/resolv.conf
nameserver XX.XXX.XXX.X
nameserver YYY.YY.YY.YY
nameserver 1.1.1.1
If you see that, remove these lines to get back to automatic resolv.conf generation and restart WSL (wsl --shutdown)
3. Checking the (fixed !) response time
[tbg#~] time kubectl get deployments
No resources found in default namespace.
real 0m10.530s
user 0m0.087s
sys 0m0.043s
I found out my resolv.conf configuration was causing that latency, by trying to reinstall kubectl with apt, and finding apt really slow too
Right now access to /mnt folders in WSL2 is too slow and by default at launch the entire Windows PATH is added to the Linux $PATH so any Linux binary that scans $PATH will make things unbearably slow.
To disable this feature, edit the /etc/wsl.conf to add the following section:
[interop]
appendWindowsPath = false
Avoid adding Windows Path to Linux $PATH and best for now is adding folders to the $PATH manually.
Terminate the WSL distro (wsl.exe --terminate <distro_name>) to make it immediately effective or wsl.exe --shutdown and start the terminal again.
Refer to the stack link for more information.
Related
I have downloaded the sandbox from hortonworks (Centos OS), then tried to follow the tutorial. It seems like the ambari-admin-password-reset command is not there and missing. I tried also to login with putty, the console asked me to change the password so I did.
now it seems like the command is there, but I have different passwords for the console and one for the putty for the same user.
I have tried to look for the reason why for the same user 'root' I have 2 different passwords (one for the virtual box console and one for the putty) that I can login with. I see different commands on each box. more than that when I share folder I can only see it on the virtual box console but not on the putty console) which is really frustrating.
How can I enforce that what I would see from putty would be the same as what I see from the virtual box console.
I think it somehow related to TTY but I am not sure.
EDIT:
running commands from the virtual box machine output:
grep "^passwd" /etc/nsswitch.conf
OUT: passwd: files sss
grep root /etc/passwd
OUT: rppt"x"0"0"root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
getent passwd root
OUT: root:x:0:0:root:/root:/bin/bash
EDIT:
I think this is all about docker containers. It seems like the machine 2222 port is the ssh port for the hdp 2.5 container and not for the hosting machine.
Now I get another problem. when running
docker exec sandbox ls
it is getting stuck. any help ?
Thanks for helpers
So now I had the time to analyze the sandbox vm, and write it up for other users.
As you stated correctly in your edit of the question, its the docker container setup of the sandbox, which confuses with two separate root users:
via ssh root#127.0.0.1 -p 2222 you get into the docker container called "sandbox". This is a CentOS release 6.8 (Final), containing all the HDP services, especially the ambari service. The configuration enforces a password change at first login for the root user. Inside this VM you can also execute the ambari-admin-password-reset and set there a password for the ambari admin.
via console access you reach the docker host running a Centos 7.2, here you can login with the default root password for the VM as found in the HDP docs.
Coming to your sub-question with the hanging docker exec, it seems to be a bug in that specific docker version. If you google that, you will find issues discussing this or similar problems with docker.
So I thought that it would be a good idea to just update the host via yum update. However this turned out to be a difficult path.
yum tried to update the kernel, but complained that there is not enough space on the boot partion.
So I moved the boot partion to the root partition:
edit /etc/fsab and comment out the boot entry
unmount /boot
mv /boot
cp -a /boot.org /boot
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-install /dev/sda
reboot
After that I have found out that the docker configuration is broken and docker does not start anymore. In the logs it complained about
"Error starting daemon: error initializing graphdriver:
\"/var/lib/docker\" contains other graphdrivers: devicemapper; Please
cleanup or explicitly choose storage driver (-s )"
So I edited /etc/systemd/system/multi-user.target.wants/docker.service and changed the ExecStart setting to:
ExecStart=/usr/bin/dockerd --storage-driver=overlay
After a service docker start and a docker start sandbox. The container worked again and I could could login to the container and after a ambari-server restart everything worked again.
And now - with the new docker version 1.12.2, docker exec sandbox ls works again.
So to sum up the docker exec command has a bug in that specific version of the sandbox, but you should think twice if you want to upgrade your sandbox.
I ran into the same issue.
The HDP 2.5 sandbox runs all of its components in a docker container, but commands like docker exec -it sandbox /bin/bash or docker attach sandbox got stuck.
When I ran a simple ps aux, I found several /usr/bin/docker-proxy commands which looked like :
/usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 60000 -container-ip 172.17.0.2 -container-port 60000
They probably forward the HTTP ports of the various UIs of HDP components.
I could ssh into the container ip (here 172.17.0.2) using root/hadoop to authenticate. From there, I could use all "missing" commands like ambari-admin-password-reset.
$ ssh root#172.17.0.2
... # change password
$ ambari-admin-password-reset
NB: I am new to docker, so there's probably a better way to deal with this.
I'd like to post here the instructions for 3.0.1 here.
I followed the instructions of installing hortonworks version 3.0.1 here: https://youtu.be/5TJMudSNn9c
After running the docker container, go to your browser and enter "localhost:4200", that will take you to the in browser terminal of the container, that hosts ambari. Enter "root" for login and "hadoop" for password, change the root password, and then enter "ambari-admin-password-reset" in order to reset ambari password.
In order to be able to use sandbox-hdp.hortonworks.com, you need to add the line "127.0.0.1 sandbox-hdp.hortonworks.com" at the end of the /private/etc/hosts file on your mac.
Incorrect Pass
Then right corner click on power button >> power off drop down >> Restart >> when it boots up then press Esc key to get into recovery menu
Restart
select advance option and hit enter
Advance Option
Select Recovery mode hit enter
Select Root
Root enter
Command
mount -rw -o remount/
ls /home
change pass command
passwd username
user as yours
last step
enter pass two times by pressing enter
enter image description here
Hopefully you changed password (:
I have an Ubuntu 14.04 host running xen, with a couple of VM's (win7 and another 14.04). I created these remotely using virt-manager. They've been running nicely for quite some time.
But when I try "virsh list --all" on the host, I get nothing in the list.
I attempted to import one of the VM's, like this:
virt-install -n my_name -r 512 --os-type=linux --os-variant=ubuntutrusty --disk /var/lib/libvirt/images/my_name.img --import
As far as virsh was concerned, this looked like it worked. It showed up as "running" on virsh list. However, I couldn't connect to it, not even ping it. Looking at virt-manager, it still thought the VM was shut off.
I ran "virsh destroy my_name" to halt it, then started it in virt-manager. It's running as normal there, but virsh list shows as shut off.
The two apps seem to be pulling from different sources to get their info, but I don't know where those are, and don't know how to get them to get along.
How do I get virsh and virt-manager to be friends?
I posted an answer at Server Fault about this. Reproducing it here:
Check the output of virsh uri. If it returns qemu:///session, but
you're using a qemu:///system connection in Virt-Manager, you found
the cause.
In order to fix it, you should either create a "QEMU/KVM user session"
connection in virt-manager, or run virsh define ~/.config/libvirt/qemu/<filename>.xml as root. This will create
the xml definition under /etc/libvirt/qemu which will then be picked
up by virt-manager.
Another way to change your connection URI and use qemu:///system as default is to edit your .bahsrc and add:
export LIBVIRT_DEFAULT_URI="qemu:///system"
logout and login again and virsh and virt-manager will be friends!
You can connect to qemu:///session with the cmd:
$> virt-manager -c qemu:///session
or
$> virt-manager --connect=qemu:///session
A few questions are already asking about how to fix the mongodb warning:
** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always.'
** We suggest setting it to 'never'
But I'm wondering if it should be fixed. I get this warning from MongoDB 3.0.1 on a Ubuntu VM running on Google's Cloud. Should I trust MongoDB that 'never' is better? Or should I trust Google/Ubuntu that they set it to 'always' for a good reason? I imagine there are tradeoffs to be considered and don't know what I'd be trading to keep it or fix it.
Asking how to fix it is fine, but asking whether to fix it seems wiser.
Edit: Mongodb have addressed this issue since I wrote this answer. Their recommendation is at https://docs.mongodb.com/master/tutorial/transparent-huge-pages/ and probably ought to be your go-to solution. My original answer will still work, but I'd consider it a hack now that an official solution is available.
Original answer: According to the MongoDB documentation, http://docs.mongodb.org/manual/reference/transparent-huge-pages/, and support, https://jira.mongodb.org/browse/DOCS-2131, transparent_hugepage (THP) is designed to create fewer large memory blocks rather than many small memory blocks in systems with a lot of memory. This is great if your software needs large contiguous memory accesses. For MongoDB, however, regardless of memory available, it requires numerous smaller memory accesses and therefore performs better with THP disabled.
That makes me think either way will work, but you'll get better mongo (or any database) performance with THP off, giving you smaller bites of memory. If you don't have much memory anyway, THP probably ought to be off no matter what you run.
Several ways to do that are outlined in the link above. The most universally applicable appears to be editing rc.local.
$ sudo nano /etc/rc.local
Insert the following lines before the "exit 0" line.
...
if test -f /sys/kernel/mm/transparent_hugepage/khugepaged/defrag; then
echo 0 > /sys/kernel/mm/transparent_hugepage/khugepaged/defrag
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
exit 0
Note: redhat-based systems may use "redhat_transparent_hugepage" rather than "transparent_hugepage" and can be checked by:
ls /sys/kernel/mm/*transparent_hugepage*/enabled
cat /sys/kernel/mm/*transparent_hugepage*/enabled
To apply the changes, reboot (which will run rc.local) or:
$ sudo su
# source /etc/rc.local
# service mongod restart
# exit
to properly apply the changes made above
For Ubuntu using upstart scripts:
Since we are deploying machines with Ansible I don't like modifying rc files or GRUB configs.
I tried using sysfsutils / sysfs.conf but ran into timing issues when starting the services on fast (or slow machines). It looked like sometimes mongod was started before sysfsutils. Sometimes it worked, sometimes it did not.
Since mongod is an upstart process I found that the cleanest solution was to add the file /etc/init/mongod_vm_settings.conf with the following content:
# Ubuntu upstart file at /etc/init/mongod_vm_settings.conf
#
# This file will set the correct kernel VM settings for MongoDB
# This file is maintained in Ansible
start on (starting mongod)
script
echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
echo "never" > /sys/kernel/mm/transparent_hugepage/defrag
end script
This will run the script just before mongod will be started.
Restart mongod (sudo service mongod restart) and done.
In Ubuntu I used the option 'Init Script' of this document: http://docs.mongodb.org/manual/tutorial/transparent-huge-pages/
None of these worked for me on Amazon ec2 instance running Ubuntu 14.04, not even the init.d script recommended by MongoDB. I had to use the hugeadm tool by first installing it via apt-get and then running sudo hugeadm --thp-never, this post pointed me to hugeadm. I'm still trying to figure out how to disable the transparent_hugepage defrag. hugeadm doesn't seem to have an easy way to do that.
I don't know if this was an effect of the shellshock attack which my server was victim to (or another attack that worked) but it basically enabled the hacker to overwrite my SSH config file when the server rebooted.
This new file used wget to load in a file from a website, then another library of hack functions which I guessed he then used to run hacks/DOS from my server. I caught it pretty fast and ideally want to upgrade but because I have cancer and just had a big operation it is too much effort at the moment.
Therefore I did a lot of house keeping, changing passwords, removing shell access, reverting back to DASH, replacing the default shell for root and any other users to another folder with symbolic links, restoring the config file for SSH, removing CGI functionality from config files e.g
ScriptAlias /cgi-bin/ /home/searchmysite/cgi-bin/
#
allow from all
#
Removed AW stats and Webalizer for all virtual min sites.
I already had DenyHosts and Fail2Ban installed.
I also blocked in/outbound traffic to the IPs of the sites he was getting the files from.
However it seems since this change I have lost the visual cron manager from webmin.
When I go to the menu item "Scheduled Cron Jobs", it says, "The command crontab for managing user Cron configurations was not found. Maybe Cron is not installed on this system?"
However I can see in the file system it exists.
When I run crontab -l or crontab -e I get "Permission Denied"
whoami shows "root"
I did think at the time of the hack this was all related and he had used SSH and a Cron job to get his hack running.
What I want to know is how I can get the CronTab manager back.
All the cron jobs are still running such as importing feeds into my websites, running scheduled emails and so on, what I don't know is how to resolve this without a full rebuild.
If I had the time and energy I would do that but I am totally drained and before this hack everything was just running smoothly and my websites which bring me in money were working fine.
They currently are still working fine and I regularly check my logs for IPs that look odd, have strong htacess rules for xss/sql/path travesal/file hacks and ban whole countries from Cloudflare which the site sits behind. So I don't "think" the machine is compromised at the moment even if it is old - could be wrong though!
details of box
Operating system Debian Linux 5.0 Virtualmin version 3.98.gpl GPL WebMin Version: 1.610 Kernel and CPU Linux 2.6.32.9-rscloud on x86_64
So if anyone can help me get my crontab manager back that would be great.
Thanks
1) check if chattr exists, if not, download a new one.
2) type whereis crontab, then chattr -isa /path/to/crontab.(usually /usr/bin/cron) then chmod crontab back to it original settings.
3) navigate to /var/spool/ and
chattr -isa cron
cd cron
chattr -isa crontabs
4) remove cron entry in /etc/cron.weekly
Look in /etc/cron.weekly for any new
I set a proxy under Plugins in Jenkins like suggested online.
I also edited the /etc/environment
bash-3.2$ cat /etc/environment
http_proxy=proxy.company.net:8080
https_proxy=proxy.company.net:8080
HTTP_PROXY=proxy.company.net:8080
HTTPS_PROXY=proxy.company.net:8080
HTTPS_PROXY_REQUEST_FULLURI=false
HTTP_PROXY_REQUEST_FULLURI=false
I verfied the variables and they are available on logon.
When I start ant manualy as root via ssh, my "composer.phar" script is able to connect and download files. As soon as Jenkins starts the job (I think its the "jenkins" linux user), he waits until timeout and aborts the build. I used "su jenkins -s /bin/bash" to get a shell as "jenkins" and the env-vars are set correctly...
What can I do? Why does Jenkins ignore these ENV-Vars?
Thanks.
The http_proxy variables (as seen e.g. on the wget man page) require a "http://" prefix to work properly for many programs.
Jenkins on the other hand has a proxy configuration at Manage Jenkins > Plugin Manager > Advanced. This configuration overrides the environment variables.
Check Alex' answer to another question for getting around this behavior for individual nodes/builds.
I did not get it solved. After a restart the server fails all Jenkins Jobs for some minutes... suddenly the connection to the proxy succeeds and everything works well.