Unable to install and run my own npm module in Linux - node.js

I've created a npm module that I intend to publish, but not without testing that it works first. So I install the module I'm working with, npm install -g . and it works well on my Windows computer, but it won't run on my Linux (Debian) computer. Instead I get the following error:
15:52 $ transval
: No such file or directory
The only thing I've found so far when I compare the generated cmd and bash file on my windows computer is that whilest (when comparing to, say, 'gulp') the cmd-files are identical in structre the bash files are not. The second line, where the basedir is set differs. This the full output of the published bash file for my module:
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
case `uname` in
*CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac
if [ -x "$basedir/node" ]; then
"$basedir/node" "$basedir/node_modules/transval/bin/transval.bin.js" "$#"
ret=$?
else
node "$basedir/node_modules/transval/bin/transval.bin.js" "$#"
ret=$?
fi
exit $ret
But if I compare the top two lines with any other file there is a significant difference! This is the top two lines from any other module, such as gulp:
#!/bin/sh
basedir=`dirname "$0"`
All other bash files get that dirname. If I change my bash file to that basedir it all of a sudden works. It is driving me mad!
EDIT:
These two files are created when I run the command npm install -g . (thus installing my package globally for testing) or when I have published (i.e. npm publish), so I'm not generating these files my self.
My package.json has a bin entry which points at a file that looks like this:
#!/usr/bin/env node
var app = require('../bundle.js');
app.init(process.argv);
Anyone have any idea why it would work on Windows and not in Linux?

Ok, found the problem. It seems to have been a problem with publishing from Windows. Once I had published from Linux (Ubuntu in this case) I could install it on both Linux and Windows computers.
I'm not sure what the reason for this is, be it some npm bug or an issue with doze line breaks, but now it's working :)
I did try to publish previously from Linux, and failed, but with an old version of Node (4.something) and that didn't work but now I've upgraded to the latest version and it works well, so that might've had something to do with it.
Edit:
I can now verify that publishing on a Debian machine running node 6.2.2 creates an unusable published version whereas publishing on a Ubuntu machine running node 7.4.0 works well and can be installed and run anywhere. Both machines are running npm version 4.0.5.

Edit Per additional information in the OP's answer, it is indeed a line-ending problem. The problem actually is not related to $() vs. ``.
When generating on Windows, the lines end with a carriage return and a linefeed, \r\n. However, when running the generated script on Debian, only the \n is taken as the end of line. As a result, the assignment to basedir is effectively:
basedir=$(dirname "...")$'\r'
# ^^^^^ Carriage return! Oops!
I think that is why the error message was ': No such file or directory': before the :, the contents of $basedir were actually printed, ending with the \r. The \r moved the cursor back to the beginning of the line, then the rest of the error message, beginning with :, overprinted the path. (That's a guess, though — I can't reproduce the exact error message on my system.)
A workaround is to put a # (space-hash) after the basedir assignment:
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')") #
# add these ^^
That way the \r will be part of a comment and not part of basedir.
Note: see this question and its answers for other ways of getting $basedir.
Per chat, the OP is going to add the $basedir values for both options tomorrow.
For reference, here's where we are at present:
Per this answer, npm generates the wrapper scripts based on bin entries in the package.json.
npm uses the cmd-shim module to make the scripts.
cmd-shim was updated 2013/10/28 to use the dirname ... echo ... sed sequence so that it would work on msysgit.
gulp and other scripts using dirname "$0" were presumably generated with a cmd-shim predating that update.
The OP's Debian /bin/sh is apparently dash (currently 0.5.7-4 in debian stable).
for the OP, on Debian, bash is version: 4.3.46(1)-release and sed is version 4.2.2
I tried both basedir types on my Cygwin dash 0.5.8-3 and they both worked.
On Ubuntu, the OP has a different problem: /usr/bin/env: 'node\r': No such file or directory. That looks like a line-ending issue to me, probably different from the Debian issue.

Related

A shell script to check the version of NodeJS doesn't work in Git Bash on Windows

I am trying to make a shell script to automatically download, compile and build some program I've made. You can see the full script on my blog. My program requires NodeJS 11 or newer, so I am trying to make my script output an appropriate error message in case it's not installed. Here is what I've tried:
node_version=$(node -v) # This does not seem to work in Git Bash on Windows.
# "node -v" outputs version in the format "v18.12.1"
node_version=${node_version:1} # Remove 'v' at the beginning
node_version=${node_version%\.*} # Remove trailing ".*".
node_version=${node_version%\.*} # Remove trailing ".*".
node_version=$(($node_version)) # Convert the NodeJS version number from a string to an integer.
if [ $node_version -lt 11 ]
then
echo "NodeJS version is lower than 11 (it is $node_version), you will probably run into trouble!"
fi
However, when I try to run it in Git Bash on Windows 10, with NodeJS 18 installed and in PATH, here is what I get:
stdout is not a tty
NodeJS version is lower than 11 (it is 0), you will probably run into trouble!
What is going on here?
The problem here is a mismatch in expectations of how a terminal should behave from a program with unix/posix expectations compared to how terminals behave on windows. This is a known problem, and there is a longer discussion of the issue on the nodejs repository (although this is not unique to just nodejs).
To compensate there is a tool winpty which provides the missing expectations (or alternatively node-pty), although you normally do not need to install explicitly, it should be included in the normal Git bash installation.
For this reason node is normally set up to be an alias for winpty node, although that causes problems for stout/stdin redirects, so a common practice is to then invoke node as node.exe (e.g. node.exe --help | wc -l) to sometimes avoid using winpty.
For your case you probably want to change your script to
if ...is-running-on-windows... # Check https://stackoverflow.com/q/38086185/23118
then
# I expect one of these to work
NODE_BINARY=node.exe
NODE_BINARY="winpty node"
else
NODE_BINARY=node
fi
node_version=$($NODE_BINARY -v)

(oh-my-)zsh behaviour: `$ command_name` at home directory results in `cd command_name` effect

I'm currently using zsh with oh-my-zsh, and I've run into an annoying shell behaviour.
I must have done a subtle yet breaking change in the $PATH export while editing my .zshrc, because the following thing happens:
At the shell prompt while in ~/, issuing, for example, the flutter command...
$ flutter
...results in:
$ ~/flutter/
(as if calling $ flutter had been interpreted as $ cd flutter)
However, issuing $ flutter in any other directory, including $ ~/flutter results in the correct execution of the flutter command. Same thing for go and others.
Line 2 on my .zshrc exports $PATH in the following fashion:
export PATH=$HOME/bin:$HOME/.emacs.d:$HOME/flutter/bin/cache/dart-sdk:$HOME/flutter/bin/cache/dart-sdk/bin:$HOME/.pub-cache/bin:$HOME/.composer/vendor/bin:$HOME/.cargo/env:$HOME/.platformio/penv/bin:$HOME/flutter/bin:$PATH
I've been comparing .zshrc versions with other backups, and I may be missing something, but no differences were found.
What seems off, in your opinion?
If you've encountered this annoyance before, how did you correct it?
Explanation
It'a feature called AUTO_CD.
AUTO_CD (-J)
If a command is issued that can’t be executed as a normal command, and the command is the name of a directory, perform the cd command to that directory.
http://zsh.sourceforge.net/Doc/Release/Options.html#Changing-Directories
AUTO_CD is enabled by oh-my-zsh in file oh-my-zsh/lib/theme-and-appearance.zsh.
...
setopt auto_cd
setopt multios
setopt prompt_subst
[[ -n "$WINDOW" ]] && SCREEN_NO="%B$WINDOW%b " || SCREEN_NO=""
...
Solution
Append the following command in ~/.zshrc after oh-my-zsh is loaded to disable this feature.
unsetopt AUTO_CD

Why does binary npm file have two checks

Every binary inside node_modules/.bin have the following code:
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
case `uname` in
*CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac
# check if there is node executable in the same directory as this binary
if [ -x "$basedir/node" ]; then
"$basedir/node" "$basedir/../angular-cli/bin/ng" "$#"
ret=$?
else
node "$basedir/../angular-cli/bin/ng" "$#"
ret=$?
fi
exit $ret
I sort of understand what this code is doing (comments are mine), but is there anywhere an explanation why they are here (some use cases)?
Checking for the node binary in the same directory is for the cases when this module was installed globally and has an executable file in the same place as Node. It makes sure that it uses the same node for which it was installed, even if you have some other Node in your PATH.
The else branch uses just node which means the same binary as you get by typing:
which node
which is the first node in one of the directories in the PATH environment variable.
The problem with installing Node scripts is that they have to have some shebang line. People usually use #!/usr/bin/env node to run whatever node you have in PATH. But it may be a different node (possibly incompatible) than the one that was actually used to install that script.
Incidentally, that is one of the reasons why I prefer installing Node from sources than from the binary packages, because otherwise my npm script can run the wrong node if I have multiple versions installed. See my tutorial on Node installation for details.
Thye Cygwin test checks if the output of the uname command contains the word CYGWIN and in that case uses a Cygwin-specific path resolution using cygpath -w "$basedir". Cygwin is a collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows - see: https://www.cygwin.com/ - and it does some path translation to make scripts written for Unix and Linux work on Windows. For Linux the output of uname is just Linux. On Cygwin it contains CYGWIN.

Bash: Unexpected parallel behavior when reading arguments from file using xargs

Previous
This is a follow-up to this question.
Specs
My system is a dedicated server running Ubuntu Desktop, Release 12.04 (precise) 64-bit, 3.14.32-xxxx-std-ipv6-64. Neither release or kernel can be upgraded, but I can install any package.
Problem
The problem discribed in the question above seems to be solved, however this doesn't work for me. I've installed the latest lftp and parallel packages and they seem to work fine for themselves.
Running lftp works fine.
Running ./job.sh ftp.microsoft.com works fine, but I needed to chmod -x the script
Running sed 's/|.*$//' end_unique.txt | xargs parallel -j20 ./job.sh ::: does not work and produces bash errors in the form of /bin/bash: <server>: command not found.
To simplify things, I cleaned the input file end_unique.txt, now it has the following format for each line:
<server>
Each line ends in a CRLF, because it is imported from a windows server.
Edit 1:
This is the job.sh script:
#/bin/sh
server="$1"
lftp -e "find .; exit" "$server" >"$server-files.txt"
Edit 2:
I took the file and ran it against fromdos. Now it should be standard unix format, one server per line. Keep in mind that the server in the file can vary in format:
ftp.server.com
www.server.com
server.com
123.456.789.190
etc. All of those servers are ftp servers, accessible by ftp://<serverfromfile>/.
With :::, parallel expects the list of arguments it needs to complete the commands it's going to run to appear on the command line, as in
parallel -j20 ./job.sh ::: server1 server2 server3
Without ::: it reads the arguments from stdin, which serves us better in this case. You can simply say
parallel -j20 ./job.sh < end_unique.txt
Addendum: Things that can go wrong
Make certain two things:
That you are using GNU parallel and not another version (such as the one from moreutils), because only (as far as I'm aware) the GNU version supports reading an argument list from stdin, and
That GNU parallel is not configured to disable the GNU extensions. It turned out, after a lengthy discussion in the comments, that they are disabled by default on Ubuntu 12.04, so it is not inconceivable that this sort of thing might be found elsewhere (particularly downstream from Ubuntu). Such a configuration can hide in
The environment variable $PARALLEL,
/etc/parallel/config, or
~/.parallel/config
If the GNU version of parallel is not available to you, and if your argument list is not too long for the shell and none of the arguments in it contain whitespaces, the same thing with the moreutils parallel is
parallel -j20 job.sh -- $(cat end_unique.txt)
This did not work for OP because the file contained more servers than the shell was willing to put into a command line, but it might work for others with similar problems.

Node script executable not working on Mac : env: node\r: No such file or directory

I have created my node script executable to execute some tasks grunt. On Windows, my node script works fine. But on Mac OS X (Yosemite), it's not working.
My node script has been published on Windows.
My node script is installed via npm command :
npm install -g task-app
My node script have this first line :
#! /usr/bin/env node
I tried many some solutions to solve my problem but I'm still stuck.
Here's these solutions that I used :
uninstall and reinstall Node.js
execute this command to
create a link for node : sudo ln -s /usr/bin/nodejs
/usr/local/bin/node
set my path with this command : export PATH=$PATH:/usr/local/bin/node
Do you have other solutions to propose ?
EDIT :
the beginning of my script :
#! /usr/bin/env node
var grunt = require('grunt');
//Get parameters from command line
var args = process.argv.splice(2);
[...]
After all, I found the solution to my problem.
As my node script file has been created on Windows, the file is DOS format (line endings in DOS format I think). So, I used a module which allow to converting a file to a unix format :
brew install dos2unix
sudo dos2unix /usr/local/lib/node_modules/task-app/src/task-app.js
You could also use vim:
vim script
:se ff=unix
:wq
That will confirm DOS-style newlines to Unix-style newlines.
There is a problem with newlines in your script. Make sure that #!/usr/bin/env node is followed by \n (unix style) instead of \r\n (windows/dos style).
To fix that, use the tr command to remove \r's from your file:
cat your_script.js | tr -d '\r' > fixed_script.js
As PauloDev says above, this is a Mac/Windows line ending issue. To elaborate, if you are using nvm you'll need to locate your script first (in my case I'm using express-mvc-generator):
# install dos2unix
brew install dos2unix
# output the full path of your node version
which node
>> /Users/<username>/.nvm/versions/node/v8.0.0/bin/node
# confirm the file path
cat /Users/<username>/.nvm/versions/node/v8.0.0/lib/node_modules/express-mvc-generator/bin/express
# convert the line endings
sudo dos2unix /Users/<username>/.nvm/versions/node/v8.0.0/lib/node_modules/express-mvc-generator/bin/express
# then run your script
Reason:
This is typically due to a difference in line endings, especially the difference in LF vs. CRLF . Unix systems like Linux and macOS use LF , the line feed character, for line breaks by default. Windows, on the other hand, is special and uses CR/LF , carriage return AND line feed character, by default.
Ref: https://qvault.io/clean-code/line-breaks-vs-code-lf-vs-crlf/
Solution:
For mac users, change CRLF to LF in that file in which the error occurred.
This should no longer be a problem since npm#^5.4.0. npm will now auto-convert to the correct line endings. See https://github.com/npm/npm/issues/12371.
This is, however, still an issue in yarn: https://github.com/yarnpkg/yarn/issues/5480.
If you've come to this page because you've encountered this error when using yarn instead of npm, like I did, you might want to consider using npm instead of yarn. npm has most of yarn's best features these days, anyway (arguably).
The carriage return inserted by MS-DOS is interpreted as part of the script interpreter name, which is the correct behavior for Un*x systems by the way. Hence, the system looks for a file /usr/bin/node\r instead of /usr/bin/node. As others have pointed out, npm now "fixes" the problem by stripping off the newline character which is a somewhat dubious behavior.
Executable files with a shebang line that has a DOS line ending are corrupt and must be fixed by the author and not by users, npm, or yarn. At the time of this writing, there is little reason to still use DOS line endings, even if you develop on Windows systems. But you should at least fix the files you produce before distributing them to the general public. See https://help.github.com/en/github/using-git/configuring-git-to-handle-line-endings for how to configure git to handle line endings correctly.
The first command tells Git to never change line endings (in the future). Next, we refresh each repository by removing every file from Git's index and, finally, rewriting the Git index to pick up all the new line endings. This fixes the CRLFs that were introduced to your local file system when you cloned each repository.
Run this command:
git config core.autocrlf false
git rm --cached -r .
git reset --hard

Resources