User commands can be given the -complete=shellcmd option. This turns out to be quite disappointing, since instead of working in the same way as vim's built-in completion for its :! command, it just repeatedly completes the names of shell commands in the path.
I'd like to write a command that completes command names and then files. Is there a convenient way to do this, either with built-in vim functionality or via an addon?
Why not leverage bash-completion while you're at it? You should be able to fork a bash shell with /etc/bash_completion sourced and talk to it over pipes to interrogate it. That way you'd have all the shell goodies.
Of course, you could do as I do and C-z suspend Vim to drop back to your shell instead.
I ended up writing a vim addon called shell_complete. It's not really super-megazord-awesome, but it does do pretty much what I was looking for, which is to more or less emulate the completion used by :!. This is to say that it's sort of "pseudo shell completion": it completes commands for the first "word", and files from there out. Basic \ escaping of spaces seems to work okay.
pseudo shell completion
Installation of shell_complete is sort of complicated, especially for those not familiar with vim-addon-manager. The addon has a couple of dependencies, and none of them are "published", which is to say that VAM doesn't know about them yet. You can let VAM know about them by installing (via git clone; see the docs for more info) another addon I've called tt_addons. Anyway, once you've done this, you should be able to just :ActivateAddons tt_addons and then :ActivateAddons shell_complete.
If you're not using VAM, you'll have to download (or git clone, more likely) all of the related modules, and then mix them into your vim directory, or make them pathogen bundles, or what have you. If you actually end up wanting to do this, it's likely worth your while to start using VAM. If you're some sort of curmudgeonly Luddite who refuses to do so, let me know and I might put the thing up on vim.org if there's decent interest.
I think it's pretty obvious (at least from the docs) how to use shell_complete, but if you would like to see it In Action you can check out my reput addon, which uses it to do completion on its :RePutShell command. reput is also currently only available through github, and the same caveats apply with respect to installation via VAM.
Actual shell completion using the shell
For the record, I think sehe's suggestion about using the shell itself to do completion is totally flash. I actually spent quite a while figuring out how to do this, and have determined that it's possible, at least in theory. I initially thought it would be easier to do it this way than by doing what shell_complete does, but it turns out that bash (like vim) doesn't provide any programmatic way to access its completion facilities, so you end up basically having to reimplement them in bash, after scraping the configuration using grep and friends. Bash sucks a lot for this sort of thing, so I'm cowardly refusing to fight that battle, at least for the moment.
Should someone be so brave/foolish as to take up this standard, they may avail themselves of the chronicles of my travails. I managed to get it to do completions that are handled by custom completion functions. However, this is only a small part of the puzzle, because bash also provides about 6½ other ways to do completion. It does sort of complement the functionality provided by shell_complete, so it might be worthwhile to synergistically merge the two into a sort of awkward, drunken Voltron of vim shell completion.
Related
I am looking for a way to edit the source code of common Linux commands (passwd, cd, rm, cat)
Ex. Every time the 'cat' command is called (by any user), it performs its regular function, but also prints "done" to stdout after.
If you're only looking to "augment" the commands as in your example, you can create e.g. /opt/bin/cat.sh:
/bin/cat && echo "done"
and then either:
change the default PATH (in /etc/bash.bashrc in Ubuntu) as follows:
PATH=/opt/bin:$PATH
or rename cat to e.g. cat.orig and move cat.sh to /bin/cat.
(If you do the latter, your script needs to call cat.orig not cat).
If you want to actually change the behavior, then you need to look at the sources here:
https://ftp.gnu.org/gnu/coreutils/
and then build them and replaces them.
All this assumes, of course, that you have root permissions, seeing how you want to change that behavior for any user.
The answer to how to modify the source is not to, unless you have a REALLY good reason to. There’s a plethora of reasons why you shouldn’t, but a big one is that you should try to avoid modifying the source of anything that could receive an update. The update breaks, if not erases, your code and you’re left with a lot of work.
Alternatively, you can use things like Alias for quick customizations and write scripts that call and rely upon the command being available, instead of worrying about its implementation. I’ve over explained, but that’s because I’m coming to you as someone with only a little experience with Linux but much more in Development, and what I’ve said extends beyond an Operating Systems CLI capabilities and lands further into general concepts of development.
I recently started picking up vi, going through some tutorials and trying to get used to it. But I still have some questions about it.
It seems to be nice for small one file changes, but as soon as I start to try doing bigger things it seems to be lacking. For example I'm used to have code formatting, import organizing, simple overview over all packages and other things that an IDE gives me. I saw some tutorials on how to use vi as an IDE, but it felt awkward at best.
Now I'm just wondering, what are the typical use cases for vi? Is it typically used to edit small files, or can it be used for larger projects? And if you use it in larger projects, how do you make it work? Or would it be a lot easier to use an IDE with vi keybindings?
People use non-IDE editors like Vi(m) for coding due to the following reasons,
They are non-distractive, allows you to concentrate on the job.
They do not clutter you screen area, offers you more code space
They are faster
They have better/faster/cooler text manipulation at the stroke of the finger
You happen to move your hands out of the keyboard less to hold the mouse, drag it here and there and click.
They also have the flexibility to support other tools like debugger, document viewer, etc.
They also have ways to get things like code folding, etc.
For a normal programmer whose ideal work cycle is sit, write code, test, debug, more code, test debug.. Vi offers a simplistic yet powerful environment to get the work done faster and more efficiently.
For someone who had years of using some IDE, it might be like using some prehistoric tool, but once they have been through the initial days, then there is no looking back. They'd feel like there is no better thing.
Why, oh WHY, do those #?#! nutheads use vi?
I've haven't done anything that you'd call a big project in python (only little test scripts), but I use Vim exclusively for writing large embedded C applications and I have never really felt the need for an IDE.
Vim is fast to start up, extremely fast to use and (with a bit of customisation) can do most things that an IDE can do. It'll do code completion, code auto-indentation and reformatting and it is very good at refactoring. The project plugin http://www.vim.org/scripts/script.php?script_id=69 makes it very easy to manage projects with lots of files and the taglist plugin http://www.vim.org/scripts/script.php?script_id=273 is great for browsing source code. It also allows you to have the C code open side-by-side with python code, assembly code and latex documentation without having to use a different tool for each.
Overall though, I think there is one really valuable thing that Vim gives you that very few other editors do and I would find it very hard to lose that: I can have a window split into three parts like this:
Each of the subwindows can either show a separate file or (as in the screenshot) a separate part of the file. I imagine emacs can do this (although I could be wrong), but I doubt many IDEs can. This can be invaluable for refactoring and for referring to other parts of the file.
I've used VI(M) and Emacs as my primary editors for years... I've tried switching to IDEs but find they can't get out of my way enough. I always end up back in VI(M) or Emacs after a while. One major reason is that I find my hands need to leave the keyboard too much in IDEs as they require the mouse too often... And I'm too lazy to setup my own shortcuts for everything.
Here's why I use it.
it's fast to start up
it's available across multiple platforms and is on all Unix machines
it's fast to use (keystrokes for common operations, operations based around words/paragraphs etc.)
However, I use IDEs for large scale development work, since I can't believe they can be beaten for productivity, given their code-awareness and refactoring capability. I use Eclipse, but I plug a VI editor emulator into it. See this answer for more details.
I saw some tutorials on how to use VI as an IDE, but it felt awkward at best.
There's one thing to do at the very beginning: throw your sense of aesthetics overboard. You will never get the same kind of graphical experience in VIM as you do in a decent IDE.
That said, VIM actually does offer many of the features of a full-blown IDE and has a lot of advantages besides. I use VIM for almost all of my developing work and all of my text documents (using Vim-LaTeX) – even though I've actually paid money for Apple’s office suite, iWorks.
There's one point that's still nagging me, though: I can't get IntelliSense to work. For me, that's a huge problem, especially when using languages such as VB, C# or Java, for which excellent IDE support exists.
So, the learning curve for VIM is steep but once you've passed a certain point it's everything but awkward. In fact, compared to VIM's editing experience, you will find that it's the IDEs that suddenly feel awkward because while they're good at bookkeeping stuff, they suck at editing.
I frequently work on a remote system, programming for a cluster, or editing config files on a headless box. All of these could be done with a file transfer, a regular IDE, and another file transfer, but it is so much faster to just use VIM through SSH.
You can find it (or easily install it) almost anywhere - Runs on all systems that can implement the standard C library, including UNIX, Linux, DOS, Windows, Mac, BeOS, and POSIX-compliant systems.
The ability to log into a remote server and confidently edit a config file is priceless.
Vim is not very good at code awareness. What it is good at is text manipulation. It provides you with the tools for you to edit text, not to edit for you. If you just do "small" edits and don't read up on the occasional "vim commands you wish you knew" you'll never understand the power provided.
code formatting
Formatting is done with =. You can also point equalprg to an external program to do the formatting for you.
import organizing
Vim won't be able to remove unused imports but if you select the import lines
:'<,'>sort
simple overview over all packages
:vs .
This will open a vertical window containing the current working directory :)
In conclusion, vim can't replace your IDE but it will let you edit your text.
As usuall: Choose your tool depending on the work you have to do! ;-) I'm using Vim and Visual Studio. You don't have to choose only one.
If you need an IDE, use an IDE. Vim is made for text manipulation.
It supports any languages. Try to edit a Ruby script with Visual Studio.
Performance on large files is much better than for most IDEs. Try to analyze a logfile having 100k lines and more using your IDE.
It runs on many plattforms and you can use it via a remote console, if you have to edit files on a server.
...
IDEs are specialized tools, which are good for problems they are made for. Vim is an incredible flexible and powerfull generall purpose tool.
This is obviously a hot topic so I'm going to give a list of reasons why I prefer IDE editor for coding.
1) I prefer to be able to fold sections of my code (I don't know whether Vi(m) can) without having to remember the command to do so or type it.
2) I prefer one click compile button/command as opposed to having to remember my compile command and it's library of options
3) I prefer the easier highlight copy paste operations without having to........ aah sod it!!
Well by now you get the picture. I'm not knocking Vi(m) I just think having your the elements and functionality of your coding environment visually represented makes for a clearer head and encourages exploration of options you may not know exists.
I much prefer vim over vi and prefer not to lump them together. Vim provides some features that are really handy and not always immediately obvious. Already mentioned was screen splitting. Also you may want to checkout ctags or exctags. These allow you to jump around the code base from within vim. I can place the cursor over a called method, jump to the definition, jump to a definition within that method, etc. Very powerful for tracing down bugs. Cscope is another similar program.
Vim will also allow you to run arbitrary shell commands within the environment and has powerful search and replacement features.
So an IDE will provide most of those, what will vim provide over a traditional GUI based IDE? That's easy, it runs on the CLI.
This allows me to login to my dev box, run screen (definitely something to checkout for the unfamiliar: http://www.gnu.org/software/screen/) and run multiple vim sessions within my screen session on my dev box. Now when I leave work for the day, I can leave my work session just as I left it, even while running builds. When I come into work the next day I can connect from my environment as if I never left.
Another reason to like vim or vi in particular, is that it can be found on virtually any Unix environment.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I use bash, and have done so for over a decade - but occasionally I wonder whether there has been any significant new developments in the world of Linux shells.
A few years back Microsoft released PowerShell, which seemed very interesting. Is there any comparable innovation going on in Linux shells?
You do realize bash 4 has very recently been released with a load of new features and language additions?
Shell options globstar (**/foo) does a recursive search, dirspell fixes typos during pathname expansion.
Associative arrays, map strings to strings, instead of just numbers to strings.
The autocd shell option allows changing directories by just typing the directory path instead of having to put cd in front.
Coprocesses
&>> and |& redirection operators that redirect both stdout and stderr
Loads of additions to existing builtins for improved scripting convenience.
Check out:
The "official" changelog: http://tiswww.case.edu/php/chet/bash/CHANGES
A short guide to some of the new features: http://bash-hackers.org/wiki/doku.php/bash4
I'd take a look at zsh or fishshell.
One of the least touted features of Bash (and several other shells) is the ability to write your own loadables, and have the shell run them as builtins.
Lets say you write the loadable 'on' .. and you want it to work like this:
on node 123 run some command
on class nodes run some command
on all nodes run some command
... etc ..
You can follow simple examples on how to write a loadable, then enable it as a bash built in via enable -f /path/to/loadable loadable_name
So in our case, enable -f /opt/bash/loadables/on on
... in your bashrc , and you've got it.
So, if you want to have bash interpret your spiffy new language natively, you would write a loadable named 'use' or 'switch_to', then modify the parser to load a different grammar / runtime if a certain environment variable was set.
I.e.:
#/bin/bash
switch_to my-way-cool-language
funkyfunc Zippy(int p) [[
jive.wassup(p) ]]
Most people are not going to want to hack their shell, however. I did want to point out that facilities exist to take Bash and make it the way you want it, without fiddling too much with core code.
See /path-to-bash-source/examples/loadables, you might be able to get that to fly where you work, since you're still using Bash.
You can run PowerShell on Linux via Pash. It uses Mono the way PowerShell uses .NET.
I think the "original improved shell" is ksh93. bash came into existence at a time when the ksh source code was proprietary; if ksh had been open-source then, it might not have been deemed necessary to have a new shell (though with the FSF you never know). ksh is worth studying, especially for its ability to be extended through C modules, but it's not a clear win over bash. bash's autocompletion is clearly superior, which may be enough to make bash a win overall. In any case bash and ksh have made substantial effort to converge, so differences are minor.
The other interesting shell is zsh, which attempts to be everything that ksh is while also including csh. Since I never saw any point or use to csh, I am not the right person to advocate for zsh. I will point out one unusual incompatibility: by default, in zsh a variable $var always expands to a single token, even if it contains spaces. This behavior is incompatible with all other sh-derived shells, and it is occasionally inconvenient, but really it makes a lot more sense than the original, and it saves a hell of a lot of quoting.
csh was the first shell to have job control, but in my mind it (and its successors) has been superseded by bash and ksh. It was never mucn fun to write scripts in.
Finally, there are many tiny shells designed for rescue floppies (!) and other Spartan environments, but it sounds like you have little interest in those.
(In the matter of innovation, I should add that more than half the scripts I used to write as shell scripts are now Lua scripts. Others could say the same for Python or Ruby, or back in the day, Perl or Tcl. So I think the real innovation is migration away from the shell for programmable interaction at the command line.)
IIRC, Powershell is Object Oriented, whereas most unix shells and utilities operate on text. On that regard, Squirrel Shell might interest you. I've never used it, though.
If you’re willing to lose sh compatibility, you could look at using a scripting language like Python or Tcl as your shell. rlwrap can be very handy if the interpreter doesn't provide line editing, command history, completion, etc.
One philosophy regarding shells is that they should primarily only be used to connect processes with files (here is one page that espouses that approach). That said, people have written some remarkably complex software using them.
Shells don't come much more inovative than the Scheme Schell. All the power of Scheme combined with the ability to run Unix commands and an embedded awk interpreter (written in Scheme, of course). The only drawback is that it needs a tiny bit of patching to build on 64 bit Linux.
It's not exactly Bourne-shell, but it's different. Of course, you have to learn Scheme - bonus!
if you like ruby, you can use rush (ruby-unix shell, not irb)
see the presentation here
http://www.slideshare.net/adamwiggins/rush-the-ruby-shell-and-unix-integration-library
or official website to see more examples
http://rush.heroku.com/
I have recently started moving into the world of Linux development. I wanted to learn some new things and thought bash might be fun. As I learn more about bash programming I have found that there are quite an assortment of useful tools to be used (such as grep, tr, awk, etc.) There are so many that I just do not know which ones are "vital" to learn.
Shell scripting commands depend heavily on the configuration of the system itself, and can change drastically over time, unlike most programming languages (where a core library ships with the language itself and represents the "core" set of commands that a programmer would use when interacting with the outside world). Therefore,
As a modern Linux shell script programmer, which command line tools should I be familiar with?
Compressing and uncompressing various archives.
Using the man pages
alias is always helpful
as mentioned by others sed & grep (RegEx is good to know in general), sort, head, tr, cut
echo & printf (their differences and when to use what)
Getting the return value (not as useful but still handy when writing scripts) via $?
top, ps, kill, how to background/foreground/suspend a process
The important thing is combining the many tools that exists and where most become extremely useful. Using man whenever you are stuck is probably the most important thing.
I'd recommend especially that you become familiar with locate, grep and find. sed, awk and vim are next, and around these are cat, less, tail / head, ls (yes, ls!), and the many ways in which bash can help you.
Especially about Bash: beware of bashisms!
Depends on what you're doing, obviously, but I get a lot of mileage out of find, grep, rsync, and ssh. The simple ones are useful, too: cat, tail, wc, ps. There's a lot you can do with a for loop, too, and wildcard syntax is essential. For example,
$ for i in {app,web}{01,02}; do ssh $i date; done
That will ssh into hosts app01, app02, web01, and web02 and execute the date command on each one.
Try looking at commandlinefu. People come up with all sorts of things there, and you're bound to find examples of stuff which may be useful in the future.
But generally, top used commands, by John are nice as a guidance.
And of course, here be dragons, list of stuff you shouldn't do: deadly ones
You should know some console-based text editor. Pico might suffice. I myself am a vi guy, though Emacs is also acceptable. (Though I will recommend vi: that is a de-facto standard on nearly any platform of Unix, and things like grep/sed behave very similar to vi.)
Others:
Screen: extremely useful when you don't have a GUI or don't want to/can't open up many terminal windows or PuTTY sessions. Allows you to have multiple shell sessions open, and you can toggle between them (and many other things.
top: good for monitoring processes, CPU usage, and memory usage
watch: runs a command every "n" seconds and displays its output. E.g., watch -n 1 "ls -aio" executes "ls -aio" every time 1 second.
you should probably know everything on this list:
http://www.faculty.ucr.edu/~tgirke/Documents/UNIX/linux_manual.html
maybe not everything is essential all the time, but knowing at least a cursory overview of each can help a lot for basic functionality.
perl, xargs, lsof, find, grep, bash, tar, gzip, tr, tail, diff, patch, and bc.
And everything that is in SUS2 (Single UNIX Specification).
Like you mentioned, learn awk, sed and grep. They will be very good friends of yours.
Also, very important, learn to use properly a text editor such as vim.
I would also recommend you to get familiar with a good scripting language such as perl or python.
Don't worry about the commands directly. Rather when you find yourself struggling with something try a few quick Google and man page searches and see how you can improve what you're trying to do right then and there. Keep it relevant and you will get more useful results.
I want to stop losing precious time when dealing with Linux/Unix's shell.
If I could get to understand it well, that would be great. Otherwise:
I may end up losing a day just for setting up a crontab.
I'll keep wondering why the shebang in this script doesn't work.
I'll keep wondering what the real difference is between:
. run.sh
./run.sh
. ./run.sh
sh run.sh
sh ./run.sh
You see, it's these kind of things that cripple my Linux/Unix life.
As a programmer, I want to get much better at this. I suppose it's best to stay with the widely-used bash shell, but I might be wrong. Whatever tool I use, I need to understand it down to its guts.
What is the ultimate solution?
Just for fun:
. run.sh --- "Source" the code in run.sh. Usually, this is used to get environment variables into the current shell processes. You probably don't want this for a script called run.sh.
./run.sh --- Execute the run.sh script in the current directory. Generally the current directory is not in the default path (see $PATH), so you need to call out the relative location explicitly. The . character is used differently than in item #1.
. ./run.sh --- Source the run.sh script in the current directory. This combines the use of . from items #1 and #2.
sh run.sh --- Use the sh shell interpretor on run.sh. Bourne shell is usually the default for running shell scripts, so this is probably the same as item #2 except it finds the first run.sh in the $PATH rather than the one in the current directory.
sh ./run.sh --- And this is usually the same as #2 except wordier.
Command line interfaces, such as the various shell interpretors, tend to be very esoteric since they need to pack a lot of meaning into a small number of characters. Otherwise typing takes too long.
In terms of learning, I'd suggest using bash or ksh and don't let anyone talk you into something else until you are comfortable. Please don't learn with csh or you will need to unlearn too much when you start with a Bourne-type shell later.
Also, crontab entries are a bit trickier than other uses of shell. My guess is you lost time because your environment was set differently than on the command line. I would suggest starting somewhere else if possible.
Man pages are probably the "ultimate" solution. I'm always amazed at what gems they contain.
You can even use man bash to answer some of the questions you raise in this question.
Try the Linux Documentation Project's BASH pages here [tldp.org].
PS: The difference is that . is the same as the source command. This only requires read permission. To use ./run.sh, you need execute permissions. When you use 'sh', you are explicitly specifying the command you want to run the script with (here, you only need read permission). If you are using a shell to execute it, there shouldn't be a problem there. If you want to use a different program, such as 'python', you could use python run.py. Another trick is to add a line #!<program> to the beginning of your script. In your case, #!/bin/sh would do, and for Python, #/usr/bin/env python is best.
I think immersing yourself in it is the answer. It's like learning to walk, to type, to use an ergonomic keyboard, or type in dvorak. Commit to it completely. And yes, you will absolutely slow down. And you'll be forced to look at your hands or google things constantly. But eventually it will come to you.
Funny story, I killed my apartment's internet access by doing an ifconfig release. Completely didn't know that ifconfig renew was not a command. Had to call a friend when google didn't load ;) dhcpcd later and I was back to googling everything.
Oreily has an older book that teaches you about bash, and bash scripting at the end. Most everything you need to know is on the web, but spread out.
The best way to learn is by reading, and then trying things out. MAN pages are often very helpful, and there are tons of Shell scripting tutorials out there. If shell scripting is what you are after, just read, and then practice the things you read by writing little scripts that do something neat or fun. If you are looking for more info on all of the different command line applications that can be run from a shell, those are more distro dependent, so look in the documentation for your favorite distro.
There are some decent online guides that will help you feel more comfortable in the shell(s). For example:
Steve Parker's Unix/Linux Shell Scripting Tutorial
UNIX Shell Programming, by Stephen G. Kochan and Patrick H. Wood, available online at Google Books. Also available in hardback. Amazon has it.
UNIX shell scripting with sh/ksh. Apparently used as part of a class at Dartmouth College. ksh is the Korn shell, and it's close enough to bash that the info will be useful.
Spend some time reading those kinds of tutorials and, above all, playing in the shell. Pretty soon, it'll start to feel like $HOME. (Okay, sorry for the bad pun...)
use the shell a lot, type "[prog-name] --help" a lot, type "man [prog-name]" a lot, and make notes on what works and what does not -- even if those notes seem obvious at the time. By tomorrow, they might not be so obvious, again. OTOH, in a couple of weeks they definitely should be!
Have a gander at some of the many books on working with shells, like From Bash to Z Shell (which covers Bash and Z), slog through a shell-scripting tutorial, or Gnu's Bash manual.
While sticking with Bash might be best, while just learning, the fish shell might be a little bit easier to use. I played around with it for a few weeks, and while it didn't seem as powerful as bash, it did seem pretty user friendly.
The way I really learned my way around in Linux was to install gentoo. It takes forever but you begin to see how everything ties together.
Go grab the latest instructions and start following them. Do it enough times and it starts to stick.
After a while you get comfortable building everything from scratch.
I find trying to learn a new command using the man pages sometimes a bit overwhelming.
I prefer man pages for refreshing my memory.
When I want to learn a command I look for examples then use the man pages to fine tune what I want it to do.