I am trying to check my installation of hadoop. I did create the environment variables and when I call printenv, I do see my HADOOP_HOME and PATH variables printed and correct (home/hadoop and HADOOP_HOME/bin respectively).
If I go to home/hadoop in the terminal and call ls, I see the hadoop file there. If I try to run it by calling hadoop, it still tells me command not found.
First day on Linux, so there may be a stupid answer to this problem.
Your current working directory is probably not part of your path.
That is default on linux systems.
If you are in the same directory, where your hadoop file is, run that command with an relative path, like: ./hadoop
HOME DIRECTORY:
/home/hadoop is a home directory created by linux similar to Document and settings in windows.
Open your terminal and type:
ls -l /home/hadoop
Post your result for this command: ls -l /home/hadoop
SETTING GLOBAL PATH:
Go to /home/hadoop and open .bashrc in text editor.
Add these lines at the end:
export HADOOP_HOME=/path/to/your/hadoop/installation/folder
export PATH=$PATH:$HADOOP_HOME/bin
Save and exit. Now type, this in your teminal:
echo $PATH
echo $HADOOP_HOME
If these commands shows correct directories, try hadoop command. It should work.
Post your result for these command: echo $PATH and echo $HADOOP_HOME
Go to Hadoop-x.x.x/bin folder
check for hadoop folder there
run ./hadoop version
You must run “hadoop version” command.
If the hadoop setup is fine, then you should see the following result:
Hadoop 2.4.1
Subversion https://svn.apache.org/repos/asf/hadoop/common -r 1529768
Compiled by hortonmu on 2013-10-07T06:28Z
Compiled with protoc 2.5.0
From source with checksum 79e53ce7994d1628b240f09af91e1af4
For installation related guide you can refer here:
Hadoop Environment Setup
Link to my quora answer https://qr.ae/TWngHN
Hope this helps.
Thanks
Enter which hadoop in your terminal. If you see a path as an output, hadoop is set in PATH of your system. If you get something similar to this,
usr/bin/which: no hadoop in (/usr/local/hadoop.... you might not have setup everything properly. Modify the /etc/bash.bashrc with
export HADOOP_HOME = /path/to/hadoop/folder and add it to PATH using export PATH=$PATH:HADOOP_HOME/bin
You may be editing the wrong ~/.bashrc file.
Open terminal and run sudo gedit ~/.bashrc and edit these command
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
Note: You must not use sudo gedit ~/.bashrc.sh these both work differently on newer OS
Related
I am using Ubuntu 16.04 LTS and installed hadoop 2.7.2. The Output of
hadoop version
is
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using /usr/local/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar
and when i run
whereis hadoop
it gives output as
hadoop: /usr/local/hadoop /usr/local/hadoop-2.7.2/bin/hadoop.cmd /usr/local/hadoop-2.7.2/bin/hadoop
But when i run command
start-all.sh
it says command not found.
also when i run
start-dfs.sh
it gives output as command not found.
I am able to run these command when i navigate to hadoop directory but i want to run these command without navigating into hadoop directory.
Your problem is that bash doesn't know where to look for ./start-all.sh.
You can fix this by opening $HOME/.bashrc and adding a line that looks like this:
PATH=$PATH:/usr/local/hadoop/sbin
This tells bash that it should look in '/usr/local/hadoop/sbin' for start-all.sh.
Note:
Changes to $HOME/.bashrc will not take affect in any terminals that are currently open.
If you need the changes to take affect in a terminal that is currently open, run
source $HOME/.bashrc
I had to search for it with find.
find / -iname start-all.sh 2> /dev/null
It found:
/usr/local/sbin/start-all.sh
/usr/local/Cellar/hadoop/3.3.4/libexec/sbin/start-all.sh
/usr/local/Cellar/hadoop/3.3.4/sbin/start-all.sh
So in addition to the previous answer, the variable in $HOME/.bashrc looks like this:
PATH=$PATH:/usr/local/sbin/
Note: I'm not sure which one should be set as PATH
I'm new to spark, I downloaded precompiled spark.
When I try to run spark-shell from bin folder on command line, it returns
:cd /users/denver/spark-1.6/bin
:spark-shell
command not found
But if I run it like
:cd /users/denver/spark-1.6
:./bin/spark-shell
it launches spark ..
can you please let me know why it is throwing error in the 1st case
The reason why you can not run the spark-shell command in first case is because of environment variable
The terminal searches for executables in $PATH. This is a Unix environment variable that lists directories containing system binaries (such as ls, echo, or gcc). If you call an executable that's not in a $PATH directory (such as spark-shell), you need to indicate its absolute path in the file system.
In the terminal . is a synonym for the current working directory, thus ./bin/spark-shell can work properly. You could equally well call ./some/path/bin/spark-shell.
Hope it helps.
In linux:
Add to your ~/.bashrc
export SPARK_HOME=/home/das/spark-1.6.2-bin-hadoop2.6
export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin
Source ~/.bashrc.
Relaunch terminal
then spark-shell works from anywhere
If it doesn't work add to ~/.profile
This happened to me too. Using ./bin/spark-shell after changing to the spark directory should solve the issue.
I have zero experience in hadoop and trying to set up hadoop in ec2 environment. After formatted the filesystem, I tried to start hadoop and it keeps saying command not found.
I think I have tried every advice I found on stackoverflow previous questions/answers.
Here is the line I am having trouble with:
[root#ip-172-31-22-92 ~]# start-hadoop.sh
-bash: start-hadoop.sh: command not found
I have tried all the following commands (which I found on previous answers)
[root#ip-172-31-22-92 ~]# hadoop-daemon.sh start namenode
-bash: hadoop-daemon.sh: command not found
[root#ip-172-31-22-92 ~]# ./start-all.sh
-bash: ./start-all.sh: No such file or directory
[root#ip-172-31-22-92 ~]# cd /usr/local/hadoop/
-bash: cd: /usr/local/hadoop/: No such file or directory
Honestly, I don't know what I am doing wrong. Plus, I am doing this as root...is this right? it seems like I should be in user...?! (discard this question if i just sounded dumber)
I am not sure whether you have downloaded/installed the hadoop package or not, so let me walk you through the process of it briefly:
Download the latest package using wget:
wget http://apache.cs.utah.edu/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz
Extract the package relative to where you have downloaded it:
tar xzf hadoop-2.7.1.tar.gz
change the dir into the extracted directory
cd hadoop-2.7.1
Now you would be able to find or start the hadoop daemons using:
sbin/start-all.sh
You can find the script's you are trying to use in the extracted dir's (hadoop-2.7.1) sbin folder.
Make sure you follow the proper documentation to get it completed properly, because I haven't really covered installing Java or configuring hadoop which are extensively covered in the following documentation link:
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html
The scripts in this repository could help you to understand the steps to install hadoop. https://github.com/lalosam/EasyHadoop (hadoop.sh). You could try to download it and execute it. The script should download the hadoop library and confgure it as pseudo cluster. start-hadoop and stop-hadoop scripts start and stop all the services required for hadoop.
First You may have to add your HADOOP_HOME variable in .bashrc file .
Ex:
export HADOOP_HOME=/usr/local/bigdata/hadoop/hadoop-1.2.1
export CLASSPATH=$JAVA_HOME:/usr/local/bigdata/hadoop/hadoop-1.2.1/hadoop-core-1.2.1.jar
export PATH=$PATH:$HADOOP_HOME/bin
Then open a new session and execute ./start-all.sh
In Cassandra the official documentation (https://wiki.apache.org/cassandra/GettingStarted) it states, to start the service use
'bin/cassandra -f'
Then use
'bin/cqlsh'
to access. But to use cqlsh in this way I always have to go to the bin folder. What is the procedure to make it work such that I can type 'cqlsh' from anywhere in the console to access (not have to be in the bin folder of Cassandra setup) ?
(just like we access python directly from anywhere by just typing python3 in console )
To get this work work, you have to add your Cassandra bin directory to your $PATH.
From a terminal prompt, check the contents of your $PATH.
$ echo $PATH
On my Ubuntu VM, this is what I see:
/usr/local/apache-maven/apache-maven-3.1.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/jvm/jdk1.7.0_45/bin
Since you mention Python3, I'll check the location of that on my system as well:
$ which python3
/usr/bin/python3
As you can see, Python3 is in my /usr/bin directory, and /usr/bin is in my $PATH, which is why simply typing python3 works for me (and you as well).
There are a few ways to get your Cassandra bin directory into your $PATH. There is some debate about which is the "correct" way to do accomplish this. So in lieu of telling you how I would do it, I will provide a link to a question on AskUbuntu that details something like 3 ways to add a directory into your $PATH: How to add a directory to my path?
Use cassandra -f in your root folder and then you should be able to use cqlsh anywhere you have cassandra installed
I'm trying to run jedit from the terminal on fedora. its installed to the path /usr/local/share....
How do I get it to run "jedit" from the terminal rather than switching to the directory and typing java -jar jedit.jar to execute the program.
I created a sh file in that same direectory with "java -jar jedit.jar" and added that directory to ~/.bashrc. I'm aware that I need to type "sh file.sh" to run that file but I know there is a gap in my knowledge somewhere.
i want to use jedit as a commit editor for git and be able to change core.editor='jedit -w' in git config
someone please help
I assume you used the Java installer?
If this is the case, the installer should also have placed a jedit launch script in /usr/local/bin which should be in your PATH by default, so calling jedit from a terminal should instantly work after installation.
If it does not work, then please check whether the file jedit is available in /usr/local/bin and whether /usr/local/bin is in your PATH environment variable.