I installed PIG (0.10.0) on windows xp with Cygwin. I also set JAVA_HOME variable. But now when I run pig -help, I get "Cannot locate pig.jar. do 'ant jar', and try again". I did not install hadoop (using the embedded version).
Also when I ran the command for the first time after setting JAVA_HOME, I got the following warning:
cygwin warning:
MS-DOS style path detected: C:\Pig\PIG-01~1.0/pig.jar
Preferred POSIX equivalent is: /cygdrive/c/Pig/PIG-01~1.0/pig.jar
CYGWIN environment variable option "nodosfilewarning" turns off this warning.
Consult the user's guide for more details about POSIX paths:
...
Cannot locate pig.jar. do 'ant jar', and try again
I could not locate from where it is reading this path. Any help will be really appreciated.
I can answer your specific questions, but must warn you that I haven't got pig working yet on the cygwin UNIX emulator on my PC. I'll tell you what I know.
The message: 'Cannot locate pig.jar. do 'ant jar' and try again.' comes from a block of code near the end of the pig shell script. You are using pig-0.10.0. I tried to get pig-0.11.1 working but received the same error messages as you. If Hadoop is not installed, there's no directory to point the environment variable HADOOP_BIN to in the shell script, since the script uses - HADOOP_BIN=which hadoop - to set it. So near the end of the script, with no HADOP_BIN set, the code branches to require either pig.jar or pig-?.!(*withouthadoop).jar in the location given by $PIG_HOME, to be put into the variable PIG_JAR. Your shell script finds neither of these, so PIG_JAR is empty, hence the error message.
if [ -n "$PIG_JAR" ]; then
CLASSPATH="${CLASSPATH}:$PIG_JAR"
else
echo "Cannot locate pig.jar. do 'ant jar, and try again"
exit 1
fi
The java container pig.jar doesn't exist in your directory because pig hasn't been built using ant. But in fact, the script should find pig.?.!(*withouthadoop).jar. You will have pig-0.10.0.jar in your directory and the pattern matching means pig- followed by a single character followed by . followed by anything at all except something ending in 'withouthadoop', followed by .jar . The 'withouthadoop' means that the jar doesn't contain an embedded hadoop, so hadoop must already be installed. If hadoop isn't installed, pig-0.10.0.jar , it seems, should be fine.
So why isn't it finding it? In the shell script is a little branch of code for folks running the script in cygwin UNIX:
if $cygwin; then
CLASSPATH=cygpath -w "$CLASSPATH"
PIG_HOME=cygpath -d "$PIG_HOME"
PIG_LOG_DIR=cygpath -d "$PIG_LOG_DIR"
fi
This converts paths passed into java.exe into a form that java.exe will understand, since it is a Windows executable. I've found that using -m rather than -w or -d in these expressions - getting cygpath to convert e.g. /cygdrive/c/Program Files/Java .. to c:/Program Files/Java .. using forward slashes - which -m stipulates - works.
After a lot more pain with 'cannot find org.apache.pig.Main ' in the pig.jar (yes, I 'anted' it before figuring out the above) I've finally got a 'grunt>' prompt. The alterations I have made to the pig shell script in order to achieve this are:
Remove the entire if $cygwin; ... fi block described above. I assume that converting $PIG_HOME to Windows file path format is causing the code block: if [-f $PIG_HOME/pig.jar]; then; PIG_JAR=$PIG_HOME/pig.jar; else; PIG_JAR=echo $PIG_HOME/pig-?.!(*withouthadoop).jar; fi to throw the errors you see: cygwin warning, MS-DOS style path detected: c:\pig\pig-01~1/pig.jar, etc.
Following the place where you have deleted the cygwin path translation block, rewrite the PIG_OPTS variable settings as:
PIG_OPTS="$PIG_OPTS -Dpig.log.dir=cygpath -m $PIG_LOG_DIR"
PIG_OPTS="$PIG_OPTS -DPIG.log.file=pig.log"
PIG_OPTS="$PIG_OPTS -Dpig.home.dir=cygpath -m $PIG_HOME"
Rewrite the line of code at the end of the shell script that invokes java.exe - exec "$JAVA" .. as:
exec "$JAVA" $JAVA_HEAP_MAX $PIG_OPTS -classpath "cygpath -p -m $CLASSPATH" $CLASS "${remaining[#]}"
create a 'logs' directory in your PIG_HOME
Put the following export entries in the .bashrc file in your home directory to initialize environment variables when the bash shell starts:
export PATH="$PATH:/cygdrive/c/Program Files/Java/jdk-your_version/bin:/cygdrive/..your-pig-home/bin"
export JAVA_HOME="/cygdrive/c/Program Files/Java/jdk-your_version"
export CLASSPATH=""
All this lets me type 'pig -x local' and I get a 'grunt>' prompt. Interestingly, by downloading pig-0.7.0, unpacking the pig-0.7.0.tar.gz file and running pig -x local, it works out of the box, straight away. The same 'grunt>' prompt.
But, unfortunately, it's a sham. In both cases. A false grunt - a ventriloquist's grunt. The arrow keys move the cursor all over the prompt - in fact anywhere you like on the screen - the return key enters nothing, whatever you may have typed in, and only control+backslash works, to return the dollar prompt. If you get to this point and understand what's happening, please let me know.
Related
I've been trying to do some scripting, but I can't seem to understand this one error I have.
bash: /test.sh: No such file or directory
Basically, I want to run a script, test.sh using bash with the -c and -l options:
bash -l -c ~/opt/ct/src/test.sh
but this whole line is called from an Windows batch file (.cmd) located in the same directory as test.sh. I'm using 64-bit Cygwin (but I also have MinGW installed). I have ensured that the filepaths point to bash.exe (did a set PATH) and that it is the correct one, as is the shebang. I also did dos2unix on the script.
Any ideas on why? FYI the line I wrote worked perfectly fine in Cygwin.
I found the problem.
I knew I had added the path to MinGW a long time ago in the Windows Path variable. It seems that when I did bash, it was actually running the one from MinGW and not Cygwin (verified using which bash.exe).
I removed the path to MinGW and voila, it worked.
Moral of the story: Never put two paths that point to two identically named program in a Path variable!
I'm trying to get exiftool to work on my dedicated server. The issue is that PHP exec seems to run different than when a command is run as a user. Oddly enough, PHP shows up as the same user I log in with, but it does not behave the same with system commands.
Oddly enough everything works great on my localhost, but not on my server.
So as mentioned, running exiftool commands logged in via ssh is fine.
But running in a php testing script (note I've installed exiftool on each tested directory, and it runs through ssh), nothing is accessible, though it runs as user orangeman...
And it fails
Here is an update - having been on this all day:
On the shell:
-bash-4.1$ which exiftool -a
~/perl5/bin/exiftool
/usr/bin/exiftool
~/perl5/bin/exiftool
In PHP shell_exec('exiftool -a');
/usr/bin/exiftool
And here is what that file links to:
lrwxrwxrwx 1 root root 33 May 15 02:10 exiftool -> /home/orangeman/perl5/bin/exiftool
I've also tried creating symlinks of various sorts, tampering with the main $PATH variable via putenv(); in php ... I'm truly in the dark here. Works on localhost, not on dedicated server.
I've updated this with a bounty - its a serious issue in development.
I'm on a dedicated server, and the problem is as outlined above.
UPDATE
Per #gcb suggestion, I was able to print out the error that is occurring when php's exec() function runs the system command with no effect.
PHP
<?php
exec('exiftool 2>&1', $output, $r);
var_dump($output, $r);
?>
Output:
array(2) {
[0]=>
string(230) "Can't locate Image/ExifTool.pm in #INC (#INC contains: /bin/lib /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /bin/exiftool line 33."
[1]=>
string(59) "BEGIN failed--compilation aborted at /bin/exiftool line 33."
}
UPDATE
#gcb's solution worked. Thank you very much.
So you do not have a php problem now, but a perl one. your include path is bad.
Answer here.
You either have to install the ExifTool libraries in the
standard location (ie. somewhere in the #INC directories
listed in your post), or add the location to the include path,
something like this:
Code:
#!/usr/bin/perl
BEGIN { unshift #INC, "PATH_TO_DIRECTORY_CONTAINING_LIBRARIES" }
use Image::ExifTool;
You should be able to add "Image/ExifTool.pm" to the path you add
to find the ExifTool module.
- Phil
I still think using my suggestion #3 from the previous answer will fix it. If not and you really want to know the reason, create a new perl script that just outputs the contents of #INC and run it via the shell and via php. you will see the difference, then you need to find which login script is not being honored in php and open a bug against php for shell_exec not respecting it...
though the easier solution for your problem (as it does not look like you are too interested in explanations) is to just set the PERLLIB var before calling the script.
So, just do:
find / -name ExifTool.pm This will tell you where the lib is installed. let's say this returns /example/perl/Image/ExifTool.pm
append PERL5LIB=/example/perl/ to your exec() call.
exec("PERL5LIB=/example/perl/ /var/www/myscript/execscript.sh {$param}"); #and
0) you should look on your error log. usually under /var/log/apache/error
it will have messages such as "access denied" or something else.
1) you are clearly not getting enough output of that command to see any error. so try to run it with exiftool 2>&1. this will redirect stderr to stdout, so errors will appear on the output. not sure if that is relevant or php already does that. you may also want to use passthru instead of exec
2) safe-mode exec dir
your file may be out of your safe mode exec dir. read this up:
http://www.php.net/manual/en/ini.sect.safe-mode.php#ini.safe-mode-exec-dir
3) all else fails, run the command as a login shell, which should have the same scripts loaded as when you login in via ssh. just replace exec with shell_exec
...i'm pretty sure looking at the error log will solve your mistery.
One possibility is that on the command line, your $PATH had been set/modified by both your $HOME/.bashrc and your $HOME/.bash_profile because your command line is a log-in shell. When PHP is invoked by the Web server, it runs as "orangeman" BUT only as a shell, not a log-in shell, so its $PATH may not be the same, which is what you're seeing here.
Have you tried putting export PATH="what:you:want:your:PHP:path:to:be" in your $HOME/.bashrc?
I believe this is happening because you have a private installation of perl for you user.
Basically #INC is an array which perl uses for locate its library, and it does not contain the path for your installation library.
There are a couple of ways to change #INC which you can find on the below link:
http://perlmaven.com/how-to-change-inc-to-find-perl-modules-in-non-standard-locations
I hope this helps.
I recently installed a /usr/local copy of python3.3.2 and when convinced it was solid, re-installed under /usr and removed the /usr/local version. When I run the executable as /usr/bin/python3.3, everything is fine but when I run it as 'python3.3' I get the message:
> python3.3
bash: /usr/local/bin/python3.3: No such file or directory
'which' finds /usr/bin/python3.3. I did a 'set -u' and 'set echo' trying to figure out what is going on without success. How is bash getting in here?
Thank you.
Steve S.
Your executable file is still remembered by the shell as if it's done with hash:
hash: hash [-lr] [-p pathname] [-dt] [name ...]
Remember or display program locations.
Determine and remember the full pathname of each command NAME. If
no arguments are given, information about remembered commands is displayed.
-r forget all remembered locations
Running hash -r without needing to restart your shell would fix that.
Update: Actually the shell also remembers it not just by running through hash. Perhaps when you try to execute it or do things like type -P prog, the shell would remember it already. This is the error I had on my test and I didn't run w:
bash: /usr/local/bin/w: No such file or directory
And hash -r fixed it.
It seems like bash is caching the previous location of python3.3 somewhere. Try closing your shell and logging in once again - that should wipe the cache and allow bash to pick up the proper location of python3.3.
I have created a simple script:
echo "the path of the current directory is `pwd`"
and saved it by the name pathinfo
then i have created a bin directory at my home page with path as
/home/vpnsadmin/bin
and copied my script(pathinfo) to that bin directory.
Now i want run this script as a command but it is showing error
-bash: /usr/bin/test2: No such file or directory
but if copy my script(pathinfo) to "/usr/bin/" then it runs as a command.
the PATH environment variable is set as-
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/home/vpnsadmin/bin
My question is why does the shell not run it as a command when it is present in /home/vpnsadmin/bin.
or else
why does it only check for the binary at /usr/bin and not at /home/vpnsadmin/bin or at /bin
The shell that is to execute your command needs to have the correct PATH variable set at the time of execution and, depending on shell, might need to have created its own internal (hash)map of the available commands.
Assuming you are using bash, try the following with your script saved in /usr/bin:
$ PATH=/ test2
$ PATH=/usr/bin test2
In the first case you should get an expected "not found" error, in the second it should work. The third test to perform is left as an exercise...
And I have to say that the supplied error message looks a bit odd if you actually tried to do
$ test2
and not
$ /usr/bin/test2
before copying the command to /usr/bin.
Edit:
Also, avoid naming your scripts test, in any way shape or form. This causes so much confusion for beginners.
Hint:
man test
Did you have the path to bash at the top of your script and did you use backticks around pwd?
#!/bin/bash
echo "the path of the current directory is `pwd`"
Did you make the file executable?
chmod +x pathinfo
There is another script pathinfo somewhere in your path which contains a call to /usr/bin/test2
Try whereis pathinfo to see how many there are and which pathinfo to see which one your shell currently prefers.
While writing BASH scripts, I generally use the which command of a Linux machine (where Linux Machine refers to Desktop based Linux OS like Ubuntu, Fedora, OpenSUSE) for finding path or availability of other binaries. I understand that which can search for binaries (commands) which are present in the PATH variable set.
Now, I am unable to understand how to proceed in case the which command itself is not present on that machine.
My intention is to create a shell script (BASH) which can be run on a machine and in case the environment is not adequate (like some command being used in script is missing), it should be able to exit gracefully.
Does any one has any suggestions in this regard. I understand there can be ways like using locate or find etc - but again, what if even they are not available. Another option which I already know is that I look for existence of a which binary on standard path like /usr/bin/ or /bin/ or /usr/local/bin/. Is there any other possibility as well?
Thanks in advance.
type which
type is a bash built-in command, so it's always available in bash. See man bash for details on it.
Note, that this will also recognize aliases:
$ alias la='ls -l -a'
$ type la
la is aliased to 'ls -l -a'
(More of a comment because Boldewyn answered perfectly, but it is another take on the question that may be of interest to some.)
If you are worried that someone may have messed with your bash installation and somehow removed which, then I suppose in theory, when you actually invoked the command you would get an exit code of 127.
Consider
$ sdgsdg
-bash: sdgsdg: command not found
$ echo $?
127
Exit codes in bash: http://tldp.org/LDP/abs/html/exitcodes.html
Of course, if someone removed which, then I wouldn't trust the exit codes, either.