I would like to read the actual source code which the linux commands are written with. I've gained some experience using them and now I think it's time to interact with my machine at a deeper level.
I've found some commands here http://directory.fsf.org/wiki/GNU. Unfortunately I wasn't able to find basic commands such as 'ls' which seems to me easy enough to begin.
How exactly do I read the source code of the simple shell commands like 'ls'?
I'm running on Ubuntu 12.04
All these basic commands are part of the coreutils package.
You can find all information you need here:
http://www.gnu.org/software/coreutils/
If you want to download the latest source, you should use git:
git clone git://git.sv.gnu.org/coreutils
To install git on your Ubuntu machine, you should use apt-get (git is not included in the standard Ubuntu installation):
sudo apt-get install git
Truth to be told, here you can find specific source for the ls command:
http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/ls.c
Only 4984 code lines for a command 'easy enough' as ls... are you still interested in reading it?? Good luck! :D
Direct links to source for some popular programs in coreutils:
cat (767 lines)
chmod (570 lines)
cp (2912 lines)
cut (831 lines)
date (570 lines)
df (1718 lines)
du (1112 lines)
echo (272 lines)
head (1070 lines)
hostname (116 lines)
kill (312 lines)
ln (651 lines)
ls (4954 lines)
md5sum (878 lines)
mkdir (306 lines)
mv (512 lines)
nice (220 lines)
pwd (394 lines)
rm (356 lines)
rmdir (252 lines)
shred (1325 lines)
tail (2301 lines)
tee (220 lines)
touch (437 lines)
wc (801 lines)
whoami (91 lines)
Full list here.
ls is part of coreutils. You can get it with git :
git clone git://git.sv.gnu.org/coreutils
You'll find coreutils listed with other packages (scroll to bottom) on this page.
Actually more sane sources are provided by http://suckless.org look at their sbase repository:
git clone git://git.suckless.org/sbase
They are clearer, smarter, simpler and suckless, eg ls.c has just 369 LOC
After that it will be easier to understand more complicated GNU code.
CoreUtils referred to in other posts does NOT show the real implementation of most of the functionality which I think you seek. In most cases it provides front-ends for the actual functions that retrieve the data, which can be found here:
It is build upon Gnulib with the actual source code in the lib-subdirectory
You can have it on github using the command
git clone https://github.com/coreutils/coreutils.git
You can find all the source codes in the src folder.
You need to have git installed.
Things have changed since 2012, ls source code has now 5309 lines
BSD distributions are actually a nice way of reading the source code, by using their repositories, since it is all packed into one place, and you can view how historically the source code has evolved, or changed. So why not use BSD repos , such as NetBSD or OpenBSD for this task.
cd ~ && apt-get source coreutils && ls -d coreutils*
You should be able to use a command like this on ubuntu to gather the source for a package, you can omit sudo assuming your downloading to a location you own.
Related
I have Rails project. When I try to run any rake task or rails server it give me this error
env: ruby\r: No such file or directory
Could someone help me?
If you are working on a Unix / Mac, then this error is because you have incorrect line endings.
Here is a solution using dos2unix; you may need to install this program on your system. If apt is available, you can use sudo apt install dos2unix.
Set your line endings correctly, and have git manage how it handles them:
git config --global core.autocrlf input
In your directory you are going to convert all of the files by running:
find ./ -type f -exec dos2unix {} \;
This will cycle through all of your files, converting them. and solving the problem. Add your changes. Commit them, and you should be good to go.
You probably have edited ./bin/rake file and added \r at the end of first line:
#!/usr/bin/env ruby
begin
load File.expand_path("../spring", __FILE__)
rescue LoadError
end
require_relative '../config/boot'
require 'rake'
Rake.application.run
Make sure that after "ruby" you have only new line char.
Thanks to the comments above, I solved my server issue that was caused from cloning my group's github rails app and causing localhost:3000 to fail. I was just working on the backend from my fullstack app: ruby(-v 2.7.1)/rails(-v 6.0.3.4). And these 2 people's comments solved my error:
"For those of you who got "find: ‘dos2unix’: No such file or directory" error: sudo apt install dos2unix" – RealMan Jul 26 '17 at 14:59
"Note that that find command may be excessive... this point is arguable; it may well be fine, but it may be overkill in some situations. Another possible route (for step 2 in this answer) is git rm -r --cached . followed by git reset --hard HEAD... which is likely faster (if nothing else, it won't run dos2unix on files in the .git housekeeping directory!)... This has potential gotchas as well (probably quite fine if you're running from a "clean" checkout, though), but thought I'd at least mention it." – lindes Jul 13 '19 at 0:42
I kept getting this error and finally figured out how to fix it.
I made sure all the permissions on the files in my bin folder were
executable.
Run ls -lha in your current repository. You want each file to have an x at the end like this
-rwxr-xr-x.
To achieve this, you will want to run chmod +x <file_name_here> for each file in your bin folder, such as chmod +x rails, chmod +x bundle, etc.
Now when you run ls -lha you should see that they all have an x at the end.
Next, either in SublimeText, Atom or what ever text editor you have, you will want to check that you are not using Windows line endings. The \r character is something Windows uses. Unix just uses \n for a new line.
I use Atom so I went to the plugins section (Cmd + , on Mac) and then searched for line-ending-selector in the Packages section, and then went to the line-ending-selectors settings. Change your default to 'LF'.
You will find that at the bottom of files, Atom will tell you the type of line ending the file is using with a CRLF for Windows and LF for Unix/Mac. You want all your files to use 'LF'.
So in your terminal, open each file in your bin folder in Atom, by running atom ./bin/filename (such as atom ./bin/rake).
At the bottom you will see 'CRLF' or 'LF'. If you see 'CRLF', click on it and, at the top of Atom, you can choose 'LF'.
Cmd + s to save.
Do this for each. You are basically telling your file to strip all Windows line endings and use Unix line endings instead.
Once all files are edited, you should be able to run your rake or rails command.
Note: Sublime Text and Text Mate should have equivalents to Atom's line-ending-selector.
For macOS users
Step 1: HOMEBREW_NO_AUTO_UPDATE=1 brew install dos2unix
Step 2: git config --global core.autocrlf input
Step 3: find ./ -type f -exec dos2unix {} \; (in the repo you were trying to run your task on)
git add and git commit
You are good to go!
If none of the works, try this:
git config --global core.autocrlf true
rails app:update:bin
I had the same problem on Windows Terminal, using WSL 2! I followed a post that recommended to install the dos2unix dependencie: sudo apt install dos2unix (Using apt package manager) and run other two commands:
git config --global core.autocrlf input (Set your line endings correctly, and have git manage how it handles them)
find ./ -type f -exec dos2unix {} \; (In your directory you are going to convert all of the files)
The git will identify a couple of changes, but you don't need to commit it. I just made a git restore . , remove node dependencies rm -rf node_modules and download it again yarn install.
There are similar questions about how to add numbers with leading zero etc. but in my case my filename has two numbers which is the number of chapter and the number of page. Both lack the leading zero, so they aren't sorted alphabetically. Using rename or any other method I want to convert files like these:
file_1_1.mp3 to file_01_01.mp3
file_1_12.mp3 to file_01_12.mp3
file_12_1.mp3 to file_12_01.mp3
...
I tried this:
rename 's/\d+/sprintf("%02d",$&)/e' *.mp3
but it just add leading zero to the chapter number.
Like this:
rename -n 's/(\d+)_(\d+)\./sprintf("%02d_%02d.", $1, $2)/e' *.mp3
Remove -n switch when the output looks good for you
Output
rename(file_1_12.mp3, file_01_12.mp3)
rename(file_1_1.mp3, file_01_01.mp3)
rename(file_12_1.mp3, file_12_01.mp3)
man rename
There are other tools with the same name which may or may not be able to do this, so be careful.
The rename command that is part of the util-linux package, won't.
If you run the following command (GNU)
$ rename
and you see perlexpr, then this seems to be the right tool.
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo apt install rename
$ sudo update-alternatives --set rename /usr/bin/file-rename
For archlinux:
pacman -S perl-rename
For RedHat-family distros:
yum install prename
The 'prename' package is in the EPEL repository.
For Gentoo:
emerge dev-perl/rename
For *BSD:
pkg install gprename
or p5-File-Rename
For Mac users:
brew install rename
If you don't have this command with another distro, search your package manager to install it or do it manually (no deps...)
This tool was originally written by Larry Wall, the Perl's dad.
This shell script works:
for file in *mp3
do
new=$(echo "$file" | sed 's/_/_0/g; s/_0\([0-9][0-9]\)/_\1/g;');
mv "$file" "$new";
done;
Appends a 0 to each underscore found
Removes that 0 if it resulted in at least digits in a row
Edit: added global flag to the 2nd substitute command, per comment by #PaulHodges
I'm trying to run git diff --no-index --numstat <string> <string> on Linux (Debian Docker container), but I'm failing in finding a way to achieve this. In essence I want to pass the files' contents as strings instead of their file paths. The goal is to retrieve the stats from the --numstat flag.
This command should be executable outside of a git repository/directory and on Linux. So far, I've found two solutions which lack the former requirements:
git diff --no-index --numstat /dev/fd/3 /dev/fd/4 3<<<$(echo "<string>") 4<<<$(echo "<string>"): This works on MacOS, but fails to work on Linux.
git diff --numstat $(echo <string> | git hash-object -w --stdin) $(echo <string> | git hash-object -w --stdin): which only works inside git repositories (got this partial solution from here)
Certainly there must be a way to achieve this, either via some git command or other bash concepts I'm unaware of. Any help would be great.
Thanks!
The reason that solution 1. isn't working is that /dev/fd/3 and /dev/fd/4 are symlinks and git diff does not follow symlinks but instead uses their link target string as their "content".
The only way to pass a string to git diff directly instead of a file is as stdin - which obviously only works for one of the files. So I see only two possible solutions to your problem:
write the strings to (temporary) files first, then pass them to git diff
use another tool, as suggested by #B--rian in the comment
Another, shorter version of 1. using process substitution would be:
git diff --no-index --numstat <(echo "<string1>") <(echo "<string2>")
Which unfortunately doesn't work either for the same reason/because git diff does not support process substitution, see https://stackoverflow.com/a/49636553/11932806
I am trying to compile grammar parser https://github.com/RichardMoot/Grail into Linux program according to instructions https://github.com/RichardMoot/Grail/blob/master/README and http://www.labri.fr/perso/moot/tutorial/install.html. There is manual how to create Linux executable from SWI-Prolog code http://www.swi-prolog.org/FAQ/UnixExe.html. All that is fine. But I can not find in the Makefile https://github.com/RichardMoot/Grail/blob/master/Makefile any compilation command. SWI-Prolo uses swipl command for compilation but this Makefile swipl calls only once - for displaying the version of the swipl.
I experience some hardship in installation and compilation, that is fine, I can execute/debug Makefile line by line and arrive at the result. But there is problem in my case - I can not see the ultimate goal in my makefile: which lines are responsible for the production of object files (if necessary) and which lines are responsible for the creation of the final Linux executable.
This is windowed program. The source code and documentation contains warnings about incompatibility with the SWI-Prolog 7, but that is fine, I can resolvem them myself, but as I said - I can not see the Makefile lines for creation of exe.
The source code is created by eminent scientist and I certainly don't want to disturb him by so low-level technical question. I would be happy if he continues work on theory and doesn't waste time on low level programming questions. Hope, that there are SWI-Prolog experts.
I am using latest (7.x) SWI-Prolog on Ubuntu 16.x and I have already installed all the mentioned prerequisites.
If you look closely at the provided Makefile, you'll find that the rules all and install are defined as follows (comments added by me):
all:
-cd source ; $(edit) g3 > g3.tmp # Replaces placeholders for your
# ... GRAIL_ROOT install directory.
-cd source ; mv -f g3.tmp g3 # Overwrites `g3` with the filled file.
cd source ; chmod a+x g3 # Makes it executable.
install: # Essentially copies all files to
-mkdir $(datarootdir) # ... your install directory.
-mkdir $(datadir)
cp -f $(images) $(datadir)
-mkdir $(bindir)
cp -f source/insertdot $(bindir)
chmod a+x $(bindir)/insertdot
cp -f $(resources) $(datadir)
cp -f source/*.pl $(bindir)
cp -f source/g3 $(bindir)
If you then do the common make && make install you'll end up with two folders installed in your Grail directory: bin and share. Inside the binary directory you'll have the g3 file that, regardless of being a SWI-Prolog source, has this initial line:
#!/usr/bin/swipl -q -g start -f
% [... prolog code.]
This header should allow your console terminal to determine what interpreter to use for this script (in this case, swipl). In my case, executing Grail with ./g3 returned a SWI-Prolog message indicating that wrong options/command arguments were used.
According to the man, Unix systems have to use option -s at the end of the header (but this didn't work either in my case):
From the manual:
-s file
Load file as a script. This option may be used from the shell to
make Prolog load a file before entering the toplevel.
It is also used to turn a file into an executable Prolog script
on Unix systems using the following first line
#!/usr/bin/swipl option ... -s
If you want to run this program, simply call the same command from your terminal:
swipl -q -g start -s g3
We currently have a challenge where the ideal solution would be to symlink a file to a web URL...
image.jpg -> http://www.host.com/images/image.jpg
Is this possible?
Maybe a named pipe that you feed with a wget for the file?
Edit - Not wget. You can work with linx -dump. So -
mkfifo reddit
links -dump reddit.com > reddit
cat reddit
There are several nice and interesting solutions here. I especially like #ArjunShankar's fuse solution. In the spirit of keeping it simple though, perhaps a file in /etc/cron.daily with
#!/bin/sh
cd /your/dir && wget -N http://www.host.com/images/image.jpg
would be a lot simpler and Good Enough(TM)?
On mac I successfully used this great tool by maxogden, which also using FUSE:
https://github.com/maxogden/mount-url
brew install osxfuse
npm install -g mount-url
Then
mount-url "https://url-to-10-gb-video-file-on-some-external-cloud-storage/video.mp4?xxx=yyy"
This would create a symlink for the file named video.mp4 in the current directory.
Not too fast access speed, but works.