Enable native NTFS symbolic links for Cygwin - cygwin

Recent NTFS and Windows implement symlinks:
NTFS junction point can be used as directory symlink since NTFS 3.0 (Windows 2000) using linkd or junction tools.
NTFS symbolic link can also be used as symlink (for both file and directory) since Windows Vista using mklink tool.
But on Cygwin 1.7 (installed on Windows 7), ln -s creates a text file.
on Cygwin:
$ ln -s -v target mylink
`mylink' -> `target'
on MinGW (or your favorite editor):
$ cat mylink
!<symlink>ÿþt a r g e t
Is it possible to tell Cygwing to use NTFS junction point or NTFS symbolic link?
other question: Is this available on MinGW?

⸻⸻  Short answer  ⸻⸻
Define environment variable:
CYGWIN=winsymlinks:nativestrict
As pointed out by mwm you may also have to go to the settings or to run bash as Administrator. See the Notes section.
⸻⸻  Long answer  ⸻⸻
Default Cygwin symlinks are just regular files
By default Cygwin creates text files as workaround for Windows symlink flaw.
These files are not really symlinks.
Almost all Windows programs do not considers these files as symlinks.
Native symlinks are available on recent Windows versions
Recent NTFS and Windows implement symlinks:
NTFS junction point can be used as directory symlink
since NTFS 3.0 (Windows 2000) using linkd or junction tools.
NTFS symbolic link can also be used as symlink
(for both file and directory) since Windows Vista using mklink tool.
Cygwin can create native NTFS symlinks
Simplified extract of the Cygwin documentation:
Symbolic links
[...]
Cygwin creates symbolic links potentially in multiple different ways:
The default symlinks are plain files containing a magic cookie
followed by the path to which the link points. [...]
The shortcut style symlinks are Windows .lnk [...] created
if the environment variable CYGWIN [...] is set to contain
the string winsymlinks or winsymlinks:lnk. [...]
Native Windows symlinks are only created on Windows Vista/2008 and later,
and only on filesystems supporting reparse points.
Due to to their weird restrictions and behaviour, they are only created
if the user explicitely requests creating them.
This is done by setting the environment variable CYGWIN
to contain the string winsymlinks:native or winsymlinks:nativestrict.
[...]
On the NFS filesystem, Cygwin always creates real NFS symlinks.
Configuring Cygwin
Cygwin User's Guide presents variable CYGWIN and option winsymlinks:
The CYGWIN environment variable is used to configure many global settings [...].
It contains the options listed below, separated by blank characters. [...]
[...]
[...]
[...]
[...]
winsymlinks:{lnk,native,nativestrict} -
if set to just winsymlinks or winsymlinks:lnk, Cygwin creates symlinks
as Windows shortcuts with a special headerand the R/O attribute set.
If set to winsymlinks:native or winsymlinks:nativestrict,
Cygwin creates symlinks as native Windows symlinks on filesystems
and OS versions supporting them. If the OS is known not to support
native symlinks (Windows XP, Windows Server 2003), a warning message
is produced once per session.
The difference between winsymlinks:native and
winsymlinks:nativestrict is this: If the filesystem supports native
symlinks and Cygwin fails to create a native symlink for some reason,
it will fall back to creating Cygwin default symlinks with
winsymlinks:native, while with winsymlinks:nativestrict
the symlink(2) system call will immediately fail.
CYGWIN=winsymlinks:native always creates a link but uses a Cygwin fall-back when target does not exists
on Cygwin:
$ export CYGWIN="winsymlinks:native"
$ ln -s -v target mylink
`mylink' -> `target'
$ echo content > target
on MinGW:
$ cat mylink
content
People using both Windows and Cygwin programs may have issues when a symlink is created as a dummy file (Cygwin fallback when target is missing)...
CYGWIN=winsymlinks:nativestrict always uses native-Windows symlink but fails when target does not exist
on Cygwin:
$ export CYGWIN="winsymlinks:nativestrict"
$ rm -f a b
$ ln -sv a b
ln: failed to create symbolic link `b': No such file or directory
$ touch b
$ ln -sv a b
ln: failed to create symbolic link `b': File exists
$ rm b
$ touch a
$ ln -sv a b
`b' -> `a'
Because nativestrict requires the target exists before the symlink creation, some commands/scripts may fail when creating a link.
Notes
Since Windows 10 build 14972, native NTFS symlinks are available in a non-elevated shell by enabling the Developer Mode in the Developer Settings.
Reference: https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/
In older versions, only administrators have the ability to create native NT symlinks
so under Windows UAC, the Cygwin terminal emulator (mintty)
should be run with elevated privileges
(right-click the shortcut and choose Run as Administrator
or set the mintty shortcut property, Advanced → Run as Administrator).
Special thanks to Guria, Spooky and Gene Pavlovsky for their contributions.

The accepted answer is right, two little side notes.
If you only care about the symlinks you create yourself on the command line, install cygutils-extra package, it includes a winln command, which has the same syntax as ln, but creates native Windows links. Create an alias: alias ln=winln (only works in interactive shell), or even replace the ln file with winln (works in shell scripts as well) - but it might get overwritten the next time coreutils package is updated.
I've only found out it's possible to use native symlinks when I already had Cygwin installed, and added some symlinks by myself as well. So after I set CYGWIN=winsymlinks:native as my system environment variable, I wanted to convert all the existing non-native links to native. Here's what I did.
Just in case, back up your entire Cygwin directory first.
Find all symlinks and save the list to /links file:
cd /; find . -regextype egrep -regex './(dev|proc|mnt|cygdrive)' -prune -o -type l -print >links
Review links.
Create a tar archive with all the links: tar c --files-from=links >links.tar
Extract the tar archive: tar x --files-from=links <links.tar
Since native symlinks are now enabled, tar will overwrite the old Cygwin's symlinks with native symlinks.
Clean up: rm -f links links.tar
P.S. At first I used CYGWIN=winsymlinks:nativestrict, but then I found out that in this mode, ln -s target link fails if target doesn't exist. By contrast, native will create a Cygwin (non-native) symlink link pointing to the nonexistent target - this matches the behavior of ln on UNIX systems. In rare cases, nativestrict can break some programs or scripts, for example Gentoo run-crons script uses a lockfile which is a symlink pointing to the PID of the running process. In nativestrict mode the script stopped working, because it could no longer create the lockfile. Note: run-crons is a crontab helper script on Gentoo Linux, adding support for cron.{hourly,daily,weekly,monthly}/ dirs, it works very well with Cygwin.

Since #olibre answer didn't work for me. I just created a shell function.
: '
mklink - Create NTFS (Windows) links that is usable by Windows and Cygwin
Usage: mklink [/D | /H | /J] <link-path> <target-path>
Options:
/D Directory Symbolic Link
/H Hardlink
/J Directory Junction (you should prefer /D)
With no options, it creates a NTFS file symlink.
'
mklink () {
if [ "$#" -ge "3" ]; then
cmd /c mklink "$1" "$(cygpath --windows --absolute "$2")" "$(cygpath --windows --absolute "$3")"
else
cmd /c mklink "$(cygpath --windows --absolute "$1")" "$(cygpath --windows --absolute "$2")"
fi
}
Do note you need administrator permissions (for Cygwin) to run the above without problems.
Note that I am unaware whether there's any difference between symlinking to an absolute path versus symlinking to a relative path using CMD's mklink. On Linux, those 2 have different behaviours if you ever decide to move the symlink or move the target file, or move both.

I guess the easiest way is to
grant SeCreateSymbolicLinkPrivilege from Local Group Policy editor (gpedit.msc, on path by default, non-home versions)
create script named ln on path (batch or bash), implementation
similar to above described shell function
profit

You were probably looking for a way to get to another destination in catalogue tree using MSYS. There is a way. You should create a shell script ("*.sh" file) which contains line:
cd "/drive_letter/SubCatalogue/SubFolder/..."

Related

How do I move files out of a broken directory in linux?

I know the premise of the question may be confusing, but I want to understand what happened.
Recently I have been experimenting with the rockchip OK3399 single-chip computer(see here) and have installed a linux system on it with TF card installation. Using Putty and connecting with serial protocol, I was able to establish a connection with the OK3399 computer and control it through my laptop.
I am trying to self-learn some linux with the OK3399 system. I created a bash code by the name of displayvids.sh inside the directory /usr/bin, which is meant to take a variable number of pictures with a mipi camera and then save in a directory for work.
I finished writing the code, but for some reason I cannot run the .sh file when my working directory is not the /usr/bin directory, despite /usr/bin being in the %PATH% environment variable. So, I executed the following command:
mv /usr/bin/display* /usr/local/bin
... attempting to move the file to /usr/local/bin instead. The command ran successfully, but when I tried to run the command:
cd /usr/local/bin
It tells me that I cannot cd to bin
As seen from the above image, the /usr/local/bin is not even a directory. Why would mv succeed if the target was not a directory? How can I retrieve my bash file?
Why would mv succeed if the target was not a directory?
mv can also rename files:
mv foo.txt bar.txt
You renamed your script to bin and moved it under /usr/local.
You may want to remember to add a trailing slash next time, to have mv barf if the target isn't a directory:
mv /usr/bin/display* /usr/local/bin/
How can I retrieve my bash file?
Rename it back.
mv bin displayvids.sh
For future reference, you can use the file command to (try to) identify the contents of a file, if it's installed:
file bin
would have probably said bin: Bash script or similar.

How do I redirect from a built-in bin to execute an AppImage instead?

Scenario: the current version of Kate in Ubuntu 18LTS points at their customized version (which doesn't appear to support regex search capability). The bin is: /usr/bin/kate.
Desired solution: run the Kate AppImage (which has the regex search/replace functionality). The AppImage currently resides in ~/Downloads.
Question: how do I redirect the system to execute the AppImage version of Kate, instead of the built-in version?
Can I simply create a link to the AppImage in /usr/bin?
Yes, it appears you can... i.e. in my case I replaced the existing kate bin with a link that points to the appimage:
# 1st remove the existing kate binary
# (cp kate somewhere first if you want to keep a copy)
sudo rm /usr/bin/kate
# 2nd create a link in the system bin that points to the appimage
sudo link [directory where the appimage resides]/Kate.AppImage /usr/bin/kate
Done! The system will now execute the appimage when 'kate' is executed (e.g. via context menus).
=========================
UPDATE...
The above solution kinda works... it does run the appimage, however the parameters normally passed to kate (i.e. file to open) are lost in the hard link.
So... the better solution is to create a simple executable shell script (named 'kate' in the /usr/bin directory) to execute the appimage:
#!/bin/sh
exec [directory where the appimage resides]/Kate.AppImage "$#"
This passes any provided parms to the appimage.
You may want to keep (for whatever reasons) your system-installed Kate in /usr/bin/kate...
Then do not touch it. Instead create a directory in your $HOME named bin (it may already be present depending on the Linux distro you run).
Inside that directory, create a symlink:
ln -sf ~/Downloads/kate.AppImage ~/bin/kate
This may already work. If not, you have to move the ~/bin directory to the front of your path:
export PATH=${HOME}/bin:${PATH} # if you use Bash
To permanently modify this $PATH, add this same line into ${HOME}/.bashrc

How can I make Cygwin pass the home directory as a path that other applications will understand, as opposed to what amount to relational links?

I'm currently using Cygwin in my Z drive, but let's say I want to create and open a file in Sublime that is located in my Cygwin home directory (or somewhere in relation to it, such as through a symbolic link).
Manic#Babbage /cygdrive/z
$ ls ~
bin desktop programming z
Manic#Babbage /cygdrive/z
$ touch ~/test
Manic#Babbage /cygdrive/z
$ subl ~/test
However, this simply has Sublime create a new file: Z:\home\Manic\test
Is there any way to get Cygwin to pass a full Windows-style path? This is not an issue with Sublime--every other application I've tried to use this with has also failed.
cygpath converts paths. Try capturing its output by $():
subl $(cygpath -w ~/test)

How can I prevent finding executable on $PATH?

I am using a system with an incomplete installation of GNAT, the GNU Ada compiler. A script (in the gdb testsuite) is finding /usr/bin/gnatmake and assumes that it can run Ada compiles. These fail because a the linker can't find libgnat.so.
I don't have root access, so I can't install libgnat.so or remove /usr/bin/gnatmake.
Is there any way to prevent a script from finding gnatmake in /usr/bin? I clearly cannot remove /usr/bin from the path.
Can you install a private, working version of gnatmake?
If you can, then you can create a symlink to the working version of gnatmake in your $HOME/bin directory:
ln -s /path/to/real/gnatmake ~/bin/gnatmake
Then insert your own $HOME/bin directory into your $PATH:
export PATH="$HOME/bin:$PATH"
Now the shell will find your version of gnatmake before the one in /usr/bin.
Try sudoing the script as yourself (sudo -u you ./script). In case you're not allow to sudo, you can also try exec VAR=val ./script. A third way would be to add another directory to $PATH with 'fake' empty scripts to shadow the ADA files.

Safely change home directory

I'm trying to safely update the home directory as specified in /etc/passwd,
but the standard Linux utils - usermod and vipw - for doing so aren't provided
by Cygwin.
Could anyone tell me how they changed this in Cygwin?
EDIT: For recent versions of Cygwin (1.7.34 and beyond), see this newer question.
Like sblundy's answer, you can always edit by-hand.
But if you want to do it the "official" way, use the cygwin-specific mkpasswd command. Below is a snippet from the official docs on mkpasswd :
For example, this command:
Example 3.11. Using an alternate home root
$ mkpasswd -l -p "$(cygpath -H)" > /etc/passwd
would put local users' home directories in the Windows 'Profiles' directory.
There's a bunch of other really useful commands described on the Cygwin Utilities documentation page (which includes mkpasswd). The use of cygpath in the example above is another of these cygwin-specific tools.
While you're at it, you probably also want to read the Using Cygwin Effectively with Windows documentation. There's a bunch of really good advice.
I ended up exiting all my cygwin shells and editing it by hand in a text editor. So far, so good.
Note: don't escape the spaces in the "Documents and Settings" directory. The entry will look like
user:...:/cygdrive/c/Documents and Settings/user:/bin/bash
The line is tokenized on the : character.
The simplest answer I have found is to make /home to be a soft link to your Windows Home/UserProfile directory
cd /
mv home oldhome
ln -s "$(cygpath -H)" home
I used cygpath as it will get the proper location for the HOME directory on the current version of Windows. On my box cygpath -H returns /cygdrive/c/Users
For the current user the following worked for me:
Close Cygwin.
Set the HOME Windows user environment variable.
Start Cygwin.
run "mkpasswd -c -p "$(cygpath -H)" > /etc/passwd".
Restart Cygwin.
I confirmed it worked by running ssh-keygen without any arguments. After making this change the app now defaults to saving the key to /cygdrive/c/Users/user instead of /home/user.
I don't know if setting HOME is required, but I did it anyway per instructions for setting up TortoiseGit with Cygwin using Tortoise's official documentation for unofficial Cygwin support here. Setting HOME alone though was not enough for ssh-keygen to recognize the home directory change.
Also, note that Cygwin's official documentation on this issue can be found here.
Confirmed in Windows 7 using 64-bit Cygwin v1.7.35.
I always set HOME as a user-specific environment variable in Computer Properties.
To avoid problems caused by having spaces in the path to your home directory, use the short-form of the Windows 'Profiles' directory - i.e. /cygdrive/c/DOCUME~1/user.
You can do this by typing the command:
mkpasswd -l -p "$(cygpath $(cygpath -dH))" > /etc/passwd
Original answer by Christopher from elsewhere
Cygwin 1.7.34+
For those using Cygwin 1.7.34 or higher Cygwin supports configuring how to fetch home directory, login shell, and gecos information in /etc/nsswitch.conf. This is detailed in the Cygwin User Guide section:
Cygwin user names, home dirs, login shells
If you've previously created an /etc/passwd or /etc/group file you'll want to remove those and configure Cygwin using the new Windows Security model to POSIX mappings.
[[ -f /etc/passwd ]] && mv /etc/passwd /etc/passwd.bak
[[ -f /etc/group ]] && mv /etc/group /etc/group.bak
The /etc/nsswitch.conf file's db_home: setting defines how Cygwin fetches the user's home directory. The default setting for db_home: is
db_home: /home/%U
So by default, Cygwin just sets the home dir to /home/$USERNAME. You can change that though to point at any other custom path you want. The supported wildcard characters are:
%u The Cygwin username (that's lowercase u).
%U The Windows username (that's uppercase U).
%D Windows domain in NetBIOS style.
%H Windows home directory in POSIX style. Note that, for the db_home: setting, this only makes sense right after the preceeding slash, as in db_home: /%H/cygwin
%_ Since space and TAB characters are used to separate the schemata, a space in the filename has to be given as %_ (that's an underscore).
%% A per-cent character.
In place of a path, you can specify one of four named path schemata that are predefined.
windows The user's home directory is set to the same directory which is used as Windows home directory, typically something along the lines of %USERPROFILE% or C:\Users\$USERNAME. Of course, the Windows directory is converted to POSIX-style by Cygwin.
cygwin AD only: The user's home directory is set to the POSIX path given in the cygwinHome attribute from the cygwinUser auxiliary class. See also the section called “The cygwin schema”.
unix AD only: The user's home directory is set to the POSIX path given in the unixHomeDirectory attribute from the posixAccount auxiliary class. See also the section called “The unix schema”.
desc The user's home directory is set to the POSIX path given in the home="..." XML-alike setting in the user's description attribute in SAM or AD. See the section called “The desc schema” for a detailed description.
The following will make the user's home directory in Cygwin the same as is used for the Windows home directory.
db_home: windows
Cygwin 1.7.33 or earlier
For those using Cygwin 1.7.33 or earlier, update to the latest version Cygwin and remove previously used /etc/passwd and /etc/group files, then see the steps above.
Else, follow these older steps below.
Firstly, set a Windows environment variable for HOME that points to your user profile:
Open System on the Control Panel
On the Advanced tab click Environment Variables (toward the bottom)
In the User Variables area click "New…"
For Variable name enter HOME
For Variable value enter %USERPROFILE%
Click OK in all the open dialog boxes to apply this new setting
Now we are going to update the Cygwin /etc/passwd file with the Windows %HOME% variable we just created. Shell logins and remote logins via ssh will rely on /etc/passwd to tell them the location of the user's $HOME path.
At the Cygwin bash command prompt type the following:
cp /etc/passwd /etc/passwd.bak
mkpasswd -l -p $(cygpath -H) > /etc/passwd
mkpasswd -d -p $(cygpath -H) >> /etc/passwd
The -d switch tells mkpasswd to include DOMAIN users, while -l is to only output LOCAL machine users. This is important if you're using a PC at work where the user information is obtained from a Windows Domain Controller.
Now, you can also do the same for groups, though this is not necessary unless you will be using a computer that is part of a Windows Domain. Cygwin reads group information from the Windows account databases, but you can add an /etc/group file if your machine is often disconnected from its Domain Controller.
At the Cygwin bash prompt type the following:
cp /etc/group /etc/group.bak
mkgroup -l > /etc/group
mkgroup -d >> /etc/group
Now, exit Cygwin and start it up again. You should find that your HOME path points to the same location as your Windows User Profile -- i.e. /cygdrive/c/Users/username
I like to keep my cygwin installation sync'd to a pen drive and another computer, so I hate hard-coding the home directory. I use the following cygwin.bat:
echo off
SETLOCAL
set SHELL=\\bin\\bash
set HOME=%~dp0..\..\doc\unix
bin\bash --login -i
ENDLOCAL
SETLOCAL and ENDLOCAL make sure that SHELL and HOME don't clobber existing env variables for other programs. HOME=%~dp0..\..\doc\unix sets HOME to be two directories up, in the doc/unix subdirectory. Then in ....\doc\unix.bashrc, I include PATH="/bin:/usr/local/bin:/usr/X11R6/bin:/usr/bin".
I did not use start /wait %CD%\bin\bash to start bash, because I am using Console2, so I don't need an additional cmd window.
Using Windows Environment Variable: HOME
This works for me for a permanent, non-portable, non-network solution; i.e. setting the HOME Environment variable permanently in Windows.
Note that this doesn't affect ssh or telnet sessions which always refer to /etc/passwd
ref: Setting up Cygwin- My HOME environment variable is not what I want.
CMD
For current user (needs to run once per user)::
reg add HKCU\Environment /v HOME /t REG_EXPAND_SZ /d ^%USERPROFILE^%
For new Users:
reg add HKU\.DEFAULT\Environment /v HOME /t REG_EXPAND_SZ /d ^%USERPROFILE^%
Note: Carets ^ before percent-signs %
IMPORT REG FILE
Import this reg file (current user):
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Environment]
"HOME"=hex(2):25,00,55,00,53,00,45,00,52,00,50,00,52,00,4f,00,46,00,49,00,4c,\
00,45,00,25,00,00,00
For new users:
Windows Registry Editor Version 5.00
[HKU\.DEFAULT\Environment]
"HOME"=hex(2):25,00,55,00,53,00,45,00,52,00,50,00,52,00,4f,00,46,00,49,00,4c,\
00,45,00,25,00,00,00
REGEDIT
In Regedit, under:
For current user:
HKEY_CURRENT_USER\Environment
For new Users:
HKU\.DEFAULT\Environment
Create HOME as a new Expandable String Value (*REG_EXPAND_SZ*) and put in %USERPROFILE%
cd /home
rm -rf chris
ln -s /cygdrive/z chris
I am not really sure if it is the safest solution but it is a possible solution that works for me ;)
I edited my /etc/passwd file directly (making sure nothing else would be accessing it), and changed all references to /home to be /Users (on Windows 7). I found that, in order for everything to work correctly, I had to delete any directories in the /home directory (or move them to the appropriate other location). Otherwise, cygwin would develop a split personality where, for example, 'bash -l' would start in /home/Pablo but $HOME would be /Users/Pablo and emacs would appear to do the reverse. Once I deleted /home/Pablo, everything worked fine.
I only needed to be in C:\Users\username when I start cygwin. So, I just added to .bashrc and .profile
cd ${HOMEPATH}
If you prefer to use ~/. instead of $HOMEPATH, you can also add the following:
export HOME=${HOMEPATH}
This way I don't disturb the cygwin installation.

Resources