My package.json has a script in it like this:
"buildTslint": "node_modules/typescript/bin/tsc node_modules/awesomeLibrary_node_tslint/{,helpers/}*.ts",
Note the {,helpers/}*.ts part, this is called Brace Expansion and is only possible in bash, not sh.
When running yarn buildTslint I get the following output:
# yarn buildTslint
yarn buildTslint v0.22.0
$ node_modules/typescript/bin/tsc node_modules/awesomeLibrary_node_tslint/{,helpers/}*.ts
error TS6053: File 'node_modules/awesomeLibrary_node_tslint/{,helpers/}*.ts' not found.
error Command failed with exit code 2.
It seems that Yarn uses sh to execute these scripts, but I'd like to use bash for this, to be able to use brace expansion.
yarn version 1.19 added support for a new config parameter script-shell. You can now do the following:
yarn config set script-shell /bin/bash
It may launch the command using system function see also man 3 system.
To see which system call is used :
strace yarn ...
system uses fork+exec+wait and the exec family functions uses shell /bin/sh
to use bash the command can be changed to bash -c 'command ..'
Yarn doesn't yet provide a way to configure the default shell used for running scripts, see: https://github.com/yarnpkg/yarn/issues/4248
However, since yarn uses Node for spawning its processes, you can work around that by changing the default shell that Node itself uses.
On Ubuntu, if you have root permissions, you can do this by changing the symlink /bin/sh to point to something other than dash:
sudo ln -sf /bin/bash /bin/sh
In Git-bash in Windows, you can change the COMSPEC environment variable to something other than C:\Windows\system32\cmd.exe, but I haven't gotten that to work for me.
See also:
Force node to use git bash on windows
Related
I'm having an issue with WSL2:
$ where b4a
/usr/local/bin/b4a
$ b4a new
/usr/local/bin/b4a: 1: Not: not found
Even though where finds commands, I can't run them. And it's not a PATH issue either:
echo $PATH
/usr/local/sbin:/usr/local/bin:[...]
And b4a isn't the only command with this problem. What could be the cause? My distribution is Debian 10 and host is Windows 10.
Not necessarily a full answer, but hopefully the troubleshooting methods you need to arrive at a solution ...
Note that it doesn't say that the command itself isn't found. For instance, if you run:
# lllllllllll
lllllllllll: command not found
That's truly a command not found. This is different. While I don't (yet) know the exact cause, this seems closer to the issues we might see with improperly quoted paths with spaces in a shell script.
You mention that other commands have this problem -- Is there something in common with the commands that don't work properly? Is it possible that they are all shell scripts?
Try several things to debug:
Start WSL without your startup profile. It's very likely that something (or you) added a line that is causing problems. From PowerShell or CMD:
wsl ~ -e bash --noprofile --norc
b4a
If that works, then there's a problem in one of your startup files that you'll need to debug. Look for anything modifying environment variables without proper quoting, especially the PATH. WSL automatically appends the Windows path to your Linux path to make it easy to run Windows commands, but the fact that almost every Windows path has spaces in it can cause problems for unsuspecting scripters that don't take this corner case into account.
Having a space in a path is fully allowed in Linux, but some scripts just don't handle it properly.
If the command that is failing is a shell script, trying starting it with:
bash -x /usr/local/bin/b4a
Or even start WSL with wsl ~ -e bash -x to see all trace output from the shell.
In this case, you'll be looking for some problem in the script right around where it actually fails.
If all else fails, you can disable WSL's PATH modification via its config file:
sudo -e /etc/wsl.conf
Add the following:
[interop]
appendWindowsPath = false
Then exit Debian, run wsl --shutdown and restart Debian. Try the b4a command again.
If this works, then the problem is almost certainly due to some problem in the PATH quoting in these commands. I don't recommend it as a permanent solution since you will have to type out the full path of Windows applications each time you want to run them.
I'd like to run a command line script using the coffee executable, but I'd like to call that executable through npx.
Something like #!/usr/bin/env npx coffee does not work, because only one argument is supported via env.
So, is there a way to run an npx executable via env?
Here is a solution using ts-node.
Any single OS, ts-node installed globally
#!/usr/bin/env ts-node
// TypeScript code
You probably also need to install #swc/core and #swc/cli globally unless you do further configuration or tweaking (see the notes at the end). If you have any issues with those, be sure to install the latest versions.
macOS, ts-node not installed globally
#!/usr/bin/env npx ts-node
// TypeScript code
Whether this always works in macOS is unknown. There could be some magic with node installing a shell command shim (thanks to #DaMaxContext for commenting about this).
This doesn't work in Linux because Linux distros treat all the characters after env as the command, instead of considering spaces as delimiting separate arguments. Or it doesn't work in Linux if the node command shim isn't present (not confirmed that's how it works, but in any case, in my testing, it doesn't work in Linux Docker containers).
This means that npx ts-node will be treated as a single executable name that has a space in it, which obviously won't work, as that's not an executable.
See the notes at the bottom about npx slowness.
Cross-platform with ts-node not installed globally in macOS, and some setup in linux
Creating a shebang that will work in both macOS and Linux (or macOS using Docker running a Linux image), without having to globally install ts-node and other dependencies in macOS, can be accomplished if one is willing to do a little bit of setup on the Linux/Docker side. Obviously, Linux must have node installed.
Use the #!/usr/bin/env npx ts-node shebang. We just have to fool Linux into thinking that npx ts-node with the space is actually a valid executable name.
Build a named Docker image that has the required dependencies globally installed and a symbolic link making npx ts-node resolve to just ts-node.
Here is an example all-in-one command line on macOS that will both build this image and run it:
docker buildx build -t node-ts - << EOF
FROM node:16-alpine
RUN \
npm install -g #swc/cli #swc/core ts-node \
&& ln -s /usr/local/bin/ts-node '/usr/local/bin/npx ts-node'
ENV SWC_BINARY_PATH=/usr/local/lib/node_modules/#swc/core/binding
WORKDIR /app
EOF
docker run -it --rm \
-v "$(pwd):/app" \
node-ts \
sh
Note that for this example script to work, the above line containing EOF must not have any other characters on the line, before or after it, including spaces.
Inside of the running container, all .ts scripts that have been made executable chmod +x script.ts will be executable simply by running them from the command line, e.g., ./test-script.ts. You can replace the above sh with the name of the script, as well (but be sure to precede it with ./ so Docker knows to run it as an executable instead of pass it as an argument to node).
Additional Thoughts & Considerations
There are other ways to achieve the desired functionality.
The docker run command can mount files into the image, including mounting executables in various directories. Some creative use of this could avoid needing to install anything or build a docker image first.
The install commands could be part of the docker run instead of pre-building an image, but then would be performed on each execution, taking much longer.
The PATH could be modified in macOS, linux, and in the docker build to add the folder containing ts-node's bin.js from any ts-node dist directory, then a shebang of #!/usr/bin/env bin.js should theoretically work (and can try bin-esm.js to avoid needing SWC, though this enters experimental node territory and may not be suitable for production scripts). This works in macOS, and in Docker outside of an npm project, and in Docker inside of an npm project configured to use TS & swc by passing the --skipProject flag to ts-node or setting environment variable TS_NODE_SKIP_PROJECT=true. A working test command line example: docker run -it --rm -v "$(pwd):/app" -e TS_NODE_SKIP_PROJECT=true -w /app --entrypoint sh node:16-alpine -c 'PATH="$PATH:/app/node_modules/ts-node/dist" ./test.ts'.
Any named executable that can be found in the PATH and run via direct command can be a shebang (using #!/usr/bin/env executable). It can be a shell script, a binary file, anything. A shell script can easily be put at a known location, added to the PATH, and then call whatever you like. It could be multi-statement, compiling the file to .js, then running that. Whatever your needs are.
In some special cases you might want to simply use node as your shebang executable, setting node options through environment variables to force ts-node as your loader. See ts-node Recipes:Other for more info on this.
Notes:
The SWC_BINARY_PATH environment variable ensures that ts-node can find the architecture-specific swc compiler (to avoid error "Bindings not found"). If you're running on only one architecture, you won't need it. Or, if you are mounting node_modules that have these #swc packages already installed for the correct architecture, you won't need it.
It is possible to install node_modules binaries for multiple architectures. The way to do this varies between the different package managers. For example, yarn 3 lets you define which binaries to install all at once in .yarnrc.yml. There are other options for npm and possible yarn 1 (and 2?) using environment variables.
ts-node does offer options for running without swc (though this is slower). You could try shebangs with ts-node-esm instead of ts-node. Look at all the symlinks in the /usr/local/bin folder or consult ts-node documentation for more information. If ts-node
It is possible to run .ts files directly using node and setting node options in environment variables. node --loader=ts-node does work, in recent versions (16+?). The experimental mode warnings can be suppressed.
There are some crazy ways to trick the shell to run JavaScript instead of a unix shell. Check out this answer that uses a normal sh shebang, but a clever shell statement to transfer execution over to node, that is basically ignored by JavaScript. This isn't great as it requires extra lines of trickery, but could help some people. Other answers on the page are also instructive and it's worth reviewing to get the full picture.
Some of the complexity here might go away if running .ts files outside of an npm project. In my own testing in Docker, the context was always in a project having its own tsconfig.json and swc installed, so with a different setup you might have different results. It proved to be difficult to get ts-node to ignore npm project context found with the executed .ts file.
The difference between ESM and CommonJS module handling has not been explained here. This is a complicated topic and beyond the scope of this answer.
Suffice it to say that if you can figure out how to run your scripts from the command line in the form executable [options] [file], then you should be able to figure out how to run ./[file] with an appropriate shebang, by mixing and matching all the ideas presented here. You don't have to use ts-node. You can directly use node, swc, tsc itself (by first compiling and then running any .js file or set of .js files in the found context), or any utility or tool that is able to compile or run TypeScript.
Note that using npx is significantly slower than running ts-node directly, because it may need to download the ts-node package, and dependencies, every time it runs.
Some various random tips on possible strategies for SWC architecture support:
https://socket.dev/npm/package/#rnw-community/nestjs-webpack-swc
https://github.com/yarnpkg/yarn/issues/2221
I am not a Linux user so this might be a easy fix but I have tried the following:
first I install it using the command curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin for which I get the following result:
dotnet-install: .NET Core SDK version 2.1.403 is already installed.
dotnet-install: Adding to current process PATH:
/home/<!username!>/.dotnet. Note: This change will be visible only when sourcing script.
dotnet-install: Installation finished successfully.
I do . ~/.profile to reload the profile,
but even after this when I run dotnet I get the following error:
Command 'dotnet' not found, but can be installed with:
sudo snap install dotnet-sdk`
I was expecting the script to do everything and make dotnet available.
TLDR: curl | bash can not modify PATH so it will not add dotnet to your PATH. You need to add dotnet to your path manually. Add export PATH="$PATH:/home/<!username!>/.dotnet" to your ~/.profile (or ~/.bashrc or equivalent) and log out and log back in.
Long version:
When you run a command in the shell (for example, bash), the shell tries to find an executible with the name in the all the paths listed in the environment variable PATH. PATH is generally set to something like /bin:/usr/bin. So when you type a command like curl, your shell looks in both /bin and /usr/bin for an executible file named curl.
You can see what your PATH is by doing env | grep PATH or echo $PATH.
The other important piece of information is how environment variables propagate. It's quite simple, actually:
A program (or process) can only modify its own set of environment variables.
Any child processes that a process creates inherit its environment variables.
What this means is that a program that you execute can not modify the environment variables of another random program. The shell actually provides a special command, export to set its own environment variables (and any child processes it later creates will inherit those).
Note the output at the end of step 1.
Note: This change will be visible only when sourcing script.
If you run curl | bash, it runs bash as a child process. That child process can not modify the environment variables of the program that started it (the shell that invoked curl | bash). So it can not modify PATH to add the location of dotnet to it. It even (helpfully) tells you that it can't.
In step 2, you are reloading ~/.profile. But does it contain any commands to add dotnet to PATH? I dont think so. I know the dotnet-install.sh script has not historically added it. You need to add a line like
export PATH="$PATH:/home/<!username!>/.dotnet"
To your ~/.profile (or ~/.bashrc, or equivalent) manually.
Actually, I would write it as follows to make the change more portable to other users:
export PATH="$PATH:$HOME/.dotnet"
Try running this again:
sudo add-apt-repository universe
sudo apt-get install apt-transport-https
sudo apt-get update
sudo apt-get install dotnet-sdk-2.2
I am using Ubuntu 16.04 LTS and installed hadoop 2.7.2. The Output of
hadoop version
is
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using /usr/local/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar
and when i run
whereis hadoop
it gives output as
hadoop: /usr/local/hadoop /usr/local/hadoop-2.7.2/bin/hadoop.cmd /usr/local/hadoop-2.7.2/bin/hadoop
But when i run command
start-all.sh
it says command not found.
also when i run
start-dfs.sh
it gives output as command not found.
I am able to run these command when i navigate to hadoop directory but i want to run these command without navigating into hadoop directory.
Your problem is that bash doesn't know where to look for ./start-all.sh.
You can fix this by opening $HOME/.bashrc and adding a line that looks like this:
PATH=$PATH:/usr/local/hadoop/sbin
This tells bash that it should look in '/usr/local/hadoop/sbin' for start-all.sh.
Note:
Changes to $HOME/.bashrc will not take affect in any terminals that are currently open.
If you need the changes to take affect in a terminal that is currently open, run
source $HOME/.bashrc
I had to search for it with find.
find / -iname start-all.sh 2> /dev/null
It found:
/usr/local/sbin/start-all.sh
/usr/local/Cellar/hadoop/3.3.4/libexec/sbin/start-all.sh
/usr/local/Cellar/hadoop/3.3.4/sbin/start-all.sh
So in addition to the previous answer, the variable in $HOME/.bashrc looks like this:
PATH=$PATH:/usr/local/sbin/
Note: I'm not sure which one should be set as PATH
When I run node from the command line with no arguments, I enter an interactive shell. If I execute some commands, exit node, and restart node, the up arrow doesn't do anything (I'd like it scroll through my previous commands).
Is there a way I can invoke node interactively such that it will remember my old commands?
You could use rlwrap to store node.js REPL commands in a history file.
First, install rlwrap (done easily with a package manager like apt-get or brew etc).
Then add an alias for node:
alias node='env NODE_NO_READLINE=1 rlwrap node'
I'm on OSX so I add that alias to my ~/.bash_profile file, and would reload my bash_profile file via source ~/.bash_profile.. and I'm good to go!
Hope this helps!
I found a nice little project, which solves the problem:
https://www.npmjs.org/package/repl.history
install using npm (npm install -g repl.history)
and run repl.history on the command line.
io.js 2.0 includes support for persistent REPL history.
env NODE_REPL_HISTORY_FILE=$HOME/.node_history iojs
The maximum length of the history can be set with NODE_REPL_HISTORY_SIZE, which defaults to 1000.
In io.js 3.0+, REPL history is enabled by default and NODE_REPL_HISTORY_FILE is deprecated in favor of NODE_REPL_HISTORY (default: ~/.node_repl_history).
Node supports this natively.
When in node REPL just issue the following command to save the history:
$> .save ./file/to/save.js
Reference: Commands and Special Keys
As well check file .node_repl_history in user home directory.
I like the combination of both dreampulse's and badsyntax's answers. With repl.history, plus an addition to my .bash_profile I get the behavior I expect, which is command history and syntax highlighting in the node interactive shell, but bypassing repl when called with arguments (to run a script).
npm install -g repl.history
Then edit your ~/.bash_profile, adding:
function node(){
if test "$#" -lt 1; then repl.history
else env node $#; fi; }
Now restart your shell or run . ~/.bash_profile and you're good to go.
Now running
$ node
will open the repl.history nodejs interactive shell, and
$ node program.js [...]
will run program.js with node as expected.