Using Emacs for big big projects - search

Maybe is a often repeated question here, but i can't find anything similar with the search.
The point is that i like to use Emacs for my personal projects, usually very small applications using C or python, but i was wondering how to use it also for my work, in which we have project with about 10k files of source code, so is veeeery big (actually i am using source insight, that is very nice tool, but only for windows), questions are:
Searching: Which is the most convenient way to search a string within the whole project?
Navigating throught the function: I mean something like putting the cursor over a function, define, var, and going to the definition
Refactoring
Also if you have any experience with this and want to share your thoughts i will consider it highly interesting.
Br

The "traditional" way of navigating C source files is to use "etags" to make a file called TAGS, then use ALT-. to go to functions across files.
For searching for strings in files, I usually use "grep". You could make a shell script with all the directories you want to search or something if you get tired of typing them in each time.

My projects typically live in git, so I put this together to quickly search them:
;; There's something similar (but fancier) in vc-git.el: vc-git-grep
;; -I means don't search through binary files
(defcustom git-grep-switches "--extended-regexp -I -n --ignore-case"
"Switches to pass to `git grep'."
:type 'string)
(defun git-grep (command-args)
(interactive
(list (read-shell-command "Run git-grep (like this): "
(format "git grep %s -e "
git-grep-switches)
'git-grep-history)))
(let ((grep-use-null-device nil))
(grep command-args)))

There is also the Emacs Code Browser. It makes exploring projects a lot simpler. See here and here for more information.

Regarding searches in the whole project, I find extremely useful the rgrep command.
Also, imenu is quite handy to jump to a function definition in the same file.
These are my 2p.

look to EDE from CEDET - it provide base support for projects...

ECB is too heavyweight for my taste. I have had good results with xcscope. Needless to say it doesn't help too much with Python.
http://www.emacswiki.org/emacs/CScopeAndEmacs

In addition to using TAGS as others have mentioned, I find igrep and igrep-find very useful. There is also Emacs' built in grep and grep-find, but I find their interface more clumsy.
My standard search is:
M-x igrep-find some_regexp RET ~/work_area/*.cxx
Which will look for all *.cxx files under ~/work/area, and show results matching some_regexp. Like all the search utilities, it populates a compilation-like buffer you can navigate using C-x ` (aka M-x next-error).

There are many ways that Icicles can help with projects. Likewise, Bookmark+ and even Dired+.
These libraries can help you create, organize, and manage projects, wherever their files and directories might reside. And they can help you navigate and search in various ways.
Some of the features are unique -- quite different from other approaches. I could list some of the project support here, but this is the best place to start.

Related

Can ack/ag/grep print function name?

Is it possible to print the function or class name in which a keyword occurs when using ack or ag? This is something I have highly desired for quite some time.
I think it would be quite tricky, as different programming languages have different ways of enclosing functions/classes.
Note that my goal right is for searching through C source code, however I would prefer a generic solution which covers more languages/syntax.
Author of ack here. No, I don't know of any greplike tool that understands anything about the text files that it's searching. It's something that people ask for all the time, but I've never even thought about implementing it.
You said "I think it would be quite tricky, as different programming languages have different ways of enclosing functions/classes." You're exactly right. Also, consider things like comments
/* void foo() */
and literal strings
printf( "void foo()" );
that would cause problems for any search tool. Neither of those instances of the string void foo() is actually a function declaration.
Check out the More Tools page on beyondgrep.com. Something like cscope might help you.
As commented by #Inian, it would be difficult to get a robust solution using ack, ag and grep as they are not aware of the grammar of the languages.
However, for my case of looking inside C source code files, I used ack with an OR condition to include lines which are starting with the function definitions. In my case, all my functions were either returning int or nothing. Hence, the following code printed out function definition lines alongwith the lines containing the KEYWORD:
ack 'KEYWORD|^void|^int'
Although none of the programs you listed currently have this functionality, Git uses language-based regexps to implement git grep -L (search within a function name). This blog post describes how it works. The current list of regexps are in the git source tree here, and can be extended as described in the blog above.
Also, ctags provides a universal way to enumerate tags from files of multiple languages, but I haven't (yet) found a way to integrate this output with git grep -L yet.

Finding files using icicles in Emacs under Windows 7

The short version of the question: how can I get icicles to search usefully for files in directories and subdirectories, even if only given a partial match of the filename?
EDIT: The short answer is to use icicle-locate-file in the top level of your directory and use S-Tab (shift-tab) to begin completion rather than plain tab. More details in this answer.
Just as an addendum, I gave up on icicles after this as it took about 10 seconds to find files in the (large) directory tree in question each time I used icicle-locate-file. There may be a way round this delay, perhaps by creating and sets of files 'gathered' by icicles, but I began to feel that the potential benefits were being eroded by the up-front costs of working this out and by the costs of keeping the file sets updated. As the author of icicles points out below, access to *nix's locate command would allow me to use icicle-locate, which is quicker than icicle-locate-file. However, I run on Windows and the Everything utility doesn't work for me. So, back to copious use of IDO and bookmarks that expand to dired buffers.
==========
The longer version... As Emacs (24.3.1) is now my main development environment I have been exploring ways of improving my efficiency, particularly regarding filename completion for some time now. Several excellent answers to this question pointed me to ido-mode and dired-x, both of which I am now using.
Another great recommendation was Emacs' bookmarks. In particular, defining dired buffers as bookmarks and jumping to them using C-x r b mybook1 or even calling C-x r b from the C-x C-f minibuffer (this page was helpful) are two very useful strategies.
A couple of people mentioned anything and its successor helm. I have been unable to get the file location part of either package working on Windows 7. Apparently they depend on the command line version of Everything but this fails for me, as detailed in this question, to which there were some helpful responses but no definitive answers. It seems that it works smoothly on *nix but there's not much discussion of helm-locate etc on Windows. So that counts out helm and anything for filename completion.
Which brings me to icicles. In a question about making find-file search in subdirectories the asker commented that they had taken a look at "ido and icicles, but they seem to work shallowly, only within current directory".
In response to this came a comment on icicles: "you can search for any file on your system, if you want, matching any part of the file name and path" with a pointer to a page on Icicles file name input. While I appreciate the considerable effort that has gone the help pages for icicles, I didn't find this one very useful because it consisted largely of a list of descriptions of icicles functions. What would be useful to me is a tutorial that walks you through finding files.
Let's assume the following.
I am running a Windows 7 installation of Emacs 24.3.1.
I have a top-level directory containing some files and folders. In this case it is c:/iciclestest/.
I know there's a file somewhere in this section of the tree that has "grob" as part of the filename.
I want to use icicles to find this file. I have put (require 'icicles) and (icy-mode 1) in my init file.
So, off we go. Start Emacs in the scratch buffer. Hit C-x C-f. I get a File or directory prompt, with a purple plus sign to the left that I think indicates that this is one of icicles' multi-commands.
Hitting tab gets me this mini-frame, showing me the contents of the current directory.
I hit tab on the folder1 subdirectory and icicles shows me the contents of that, as one would expect.
You can see a couple of files with "grob" in the name.
Right, C-g to clear everything then C-x C-f again. Enter 'grob'. I get a "no prefix completions" message. This surprised me a little because I expected icicles to have some kind of whizzy fuzzy matching like ido-mode.
Okay, maybe I need a different command. Let's try M-x icicle-locate-file, which gives me this prompt:
If I enter 'grob' and hit confirm I just get a new file, as per below.
To recap: what I'd like to be able to do is enter a string and have icicles go and look for files or folders containing that string. My main dev folder has many dozens of directories and thousands of files so a quick find within Emacs would be a godsend.
I realise that the locate command doesn't exist on Windows so functionality will be in some ways limited, but I would have thought a recursive search of files and subdirs from the current dir based on a user-entered string would be straightforward. What am I missing? Am I going about this the right way? Can this be done in icicles?
You need to read the Icicles doc a bit more: use S-TAB instead of TAB for apropos (regexp or substring) completion. That is apparently all you want here: match grob as a substring. (No need for any fuzzy matching for that.)
Since you want files matching grob anywhere under that directory, use icicle-locate-file. Give it that directory as the starting point. (And since you want to match grob anywhere in the file name, use S-TAB for completion.)
Icicles does provide "whizzy fuzzy matching like ido-mode" (in fact a lot whizzier). Ido's "flex" matching is the same as Icicles's "scatter" matching.
You can set the kind of completion you want to be one of the fuzzy-matching types. In the minibuffer, C-( cycles among the prefix-completion methods. M-( cycles among the apropos-completion methods. True fuzzy matching is a prefix completion method. Flex/scatter matching is a poor man's fuzzy matching, and it is an apropos completion method (so use M-( to cycle to it). To change the default matching, so you need not cycle to get the ones you prefer, customize option icicle-S-TAB-completion-methods-alist or icicle-TAB-completion-methods.
In addition to the answer about icicle-locate-file, you can find files that are in marked Dired subdirectories, and their marked subdirectories, etc. recursively, using M-+ C-F (command icicle-visit-marked-file-of-content-recursive), if you use both Icicles and Dired+.
This answer provides the details.

Command-T for Emacs?

I'm a long time Emacs user, but partially switched to MacVim 2 years ago, and have been using it for all my programming work (still using Emacs for other stuff). I decided to switch back to Emacs now, mainly because of the awesome evil-mode. I'm quite happy, but I still haven't found something as good and simple as the Vim Command-T plugin.
The main problem with the Emacs alternatives is that they are either too complicated and or slow.
The closest thing to Command-T is helm/helm-cmd-t, but it doesn't quite work like it.
Let's say you have two paths:
app/controllers/admin/feedback_controller.rb
app/controllers/fee_controller.rb
In Vim's Command-T, you can write:
app/controllers/fe
And it will match both paths.
With helm-cmd-t, if I write app/controllers/fee, it will only match fee_controller. If I want to match both, I have to use a regexp, as in app/controllers/.*, which is not that bad, but requires more keystrokes.
Any suggestions appreciated!
Any Emacs package that uses ido can do this, provided that ido-flex-match is non-nil. Helm-cmd-t deliberately doesn't support this kind of flexible matching.
You can get the behaviour you describe using find-file-in-repository with ido-flex-match set to 't.
I've recently started using flx with ido-mode and projectile it has exceptionally good, ranked fuzzy matching and has a very similar feel to CtrlP and CmdT for Vim.
It can be installed via MELPA.
M-x package-install
flx
For more info see the flx project.
Here's a screengrab to illustrate...
Here it is, but required helm to be installed first: https://github.com/emacs-helm/helm-cmd-t
EDIT: See my Helm guide. Read why it is powerful. See helm with projectile in action in that section.
With fuzzy searcher like Ido + flx or the like in Vim, you have to type thing in order.
With Helm, you can perform out of order matching.I called it out of order because whether I enter "main.c x86" or "x86 main.c", I get the same set of results for the either query. But it also makes sense to call it multi-steps search. Without having to enter the search strings (called "patterns", which are actually regexp) in an orderly fashion, Helm gives me greater freedom: I can enter the thing I wanted first in my mind without having to remember its complex path; if there are many candidates from the target I wanted, I narrow it down further with more details (patterns).
The above example to illustrate the two cases from the advantage above:
If my desire target is not unique, fine I can narrow further.
If my desire target is unique, I can immediately get it.
As you can see, the Linux kernel source tree contains more than 40kfiles, and I narrowed it down to a few files immediately.
Visual Studio also implements this mechanism in their project search, but it's not generic as Helm. In Helm, you can reuse the same Helm interface for many other things; for example, see helm-semantic-or-imenu. You have something like an outline tree, but you can interactively and incrementally narrow to the candidates you want with a few simple patterns.
Finally, fyi, flx author - Le Wang - is using Helm.
Did you try LustyExplorer? It's based on the Vim plugin with the same name.
Also, it looks like you are not using Command-T to the best of its capabilities: acfe should be enough for it to match those two files. What you do is not particularly better than :e app/con*/**/fe<Tab>.
I've once run across gpicker which advertises speed as one of its advantages over Ido and other "native" elisp packages. Never got to try it out on real-world projects though, but it might work for you.
Long story short - there's nothing quite like command-T for Emacs. Best options are gpicker or Peepcode's peepopen, but they are external applications, and I find them to be distracting after using command-T for so long. I'll stick to MacVim for now, only because of command-T, but might look into implementing something that behaves just like command-T as an exercise.
Thank you all for the answers and comments!
Give textmate.el a try :)
https://github.com/defunkt/textmate.el
You'll just have to use "Command+T" instead of "Ctrl+P" :)
https://github.com/bling/fzf.el
This is the best solution out there in my opinion. Here's why:
Uses projectile to determine project root if you're in a project.
Otherwise it very quickly indexes the file in the current directory.
Fuzzy matching.
Can be customized (top or bottom placement, number of records, etc.)

Opening the header file to a C/C++ source file with vim from multiple directories and multiple extension

First off, I was about to write a long list of if/else statements in vim and realized that 1) there was a better way to do what I was trying to do and 2) SO would be ripe with help on the subject. So! I have a variety of files spread about like
foo/src/file01.C
foo/src/file02.cc
foo/src/file03.c
foo/include/file01.hh
foo/include/file02.h
foo/include/file03.h
If you notice that the C/H, cc/hh, c/h extension may or may not match then you are keen and I'd like you to please help. I've look at things like the following vim scripts from the Vim wiki for "Easily switch between source and header file" and although I only dumped a few hours into a.vim without success, it doesn't seem that the others would work via the docs on that page. So can anyone help out on how to make this work?
A good lead I had was a quick How to Easily Switch between Header and Source topic, but still couldn't make it work.
I guess what I really want is how to avoid the multiple if statements and use real matching to do what I want. I want to look into another directory and if look for a header file of the same name with any familiar extension if it was a source C/C++ file, or look for a source file of any regular extension if it was a header file. Thanks for your help!
UPDATE: I specifically want to open the file in a new tab. I live on vim tabs!
I recommend using the FSwitch plugin. https://github.com/derekwyatt/vim-fswitch
This does exactly what you need out of the box. It is better than a.vim in more than one way, being a rewrite of the idea behind a.vim.
The link you posted presents it as a solution, too.
I have just installed it to my vim configuration and it does its job well.
Enjoy.
Just to make sure I was using the most current version, I downloaded the latest a.vim script (2.18) and copied it into my ~/.vim/plugin directory.
You can define certain variables in your ~/.vimrc file to get a.vim to recognize alternate file extensions.
To get the files in your example to match their alternates I added the following to my ~/.vimrc:
let g:alternateExtensions_C = "H,hh"
let g:alternateExtensions_hh = "C"
These are global variables that allow you to override what's already defined. You'll have to define
both relationships (they don't work both ways).
You can see what the current mappings are by typing:
:echo g:alternateExtensionsDict
If you need to define other mappings, follow the same pattern. What comes after the underscore is the file extension you're editing. What's in the double quotes is a comma-separated list of alternate extensions.
let g:alternateExtensions_<ext> = "<list,of,alt,ext>"
If you have a different directory structure, you can define what paths to search by overriding the g:alternateSearchPath variable. ../src and ../include are already included by default.
:echo g:alternateSearchPath
To open the alternate file in a new tab:
:AT
By the way, the a.vim script is pretty well documented. You might want to open it up and take a look. I found a setting or two that I didn't know about and I've been using it for years ;o)
I hope that helps.
IMO your best option is to adopt existing scripts to use :tabf instead of :e or whatever the scripts use right now to open the counterpart file. You can also try to make the change configurable and submit it to the script author. (I'm pretty sure many would find the enhancement useful.)
That reminded me of a trick I used very long time ago. Instead of guessing where the corresponding source/header files are, I have used at the top of file special comment containing relative path to the counterpart file. Switching was as simple as finding the special comment, extracting file name and opening it. Problem was similar to yours in that file extensions were not predictable. My rational solution was to stop guessing and denote counterparts explicitly in the source code. (This days I would have probably tried to put the relationship table into an external file with a standard name and look it up in VIM using the upward search.)
Two helpful things
:he 'path'
:he tabfind
So you would do
:set path=../,/usr/include/,/home/buildagent/SDKROOT/mysdk/inc
:tabfind error_codes.h
to open error_codes.h from somewhere exotic without having to specify it. Note how vim globbing is very very flexible, so you might not need mucht
:argadd ./**/*.[h,H] | tab sall
will open all header files under the current directory, regardless of how many levels deep. Be careful running this command on a large tree or with symlinks outside the tree

Replace in multiple files - graphical tool for Linux

It needs to be graphical. No sed, awk, grep, perl, whatever. I know how to use those and I do use them now, but I need to cherry-pick each replace in 300+ files.
I want a tool where I can:
type a search string
type a replace string
select a directory and file extension
and it would recursively go into each file in that directory and its sub-directories, open it and scroll to the place where search string is and offer two options:
replace (and find next)
find next
Nothing more. Reg.exp. support is a plus, but not required.
SOLVED: Regexxer is exactly what I needed. In case someone needs it on Slackware, here's what you need to download and how to compile it (choosing correct version of each dependency can be a PITA)
I think regexxer is exactly what you're looking for:
Regexxer
regexxer is a nifty GUI search/replace tool featuring Perl-style regular
expressions. If you need project-wide substitution and you’re tired of
hacking sed command lines together, then you should definitely give it a try.
See also the screenshot, looks a lot like what you're describing:
Emacs + dired + query-replace-regexp
For complete recipe follow this link (it's rather long, covering all possible alternatives),
jEdit does exactly what you need. It is written in Java and works well in Linux, Windows and OS X (probably other operating systems also).
Lately Kate (if you use KDE) can do it, but in a very tricky way. Go to "Edit>Search in Files", and choose the folder within which your files exist.
The trick is that only after the search results appear, you will find a text box and a button called "Replace checked". This button will do what you want.
I use gVim for this task all the time. I open up all the files at once, then use the commands to perform a subsitution on each file, asking for confirmation. Generally I use < 20 files, so I open them as tabs and use this:
:tabdo %s/foo/bar/gc
gVim works fine on Windows :) My coworkers often use Textpad to do this same thing, but I'd say gVim is much more efficient at it.
If you are a KDE user there's also kfilereplace.

Resources