Related
I'm attempting to test an application which has a heavy dependency on the time of day. I would like to have the ability to execute the program as if it was running in normal time (not accelerated) but on arbitrary date/time periods.
My first thought was to abstract the time retrieval function calls with my own library calls which would allow me to alter the behaviour for testing but I wondered whether it would be possible without adding conditional logic to my code base or building a test variant of the binary.
What I'm really looking for is some kind of localised time domain, is this possible with a container (like Docker) or using LD_PRELOAD to intercept the calls?
I also saw a patch that enabled time to be disconnected from the system time using unshare(COL_TIME) but it doesn't look like this got in.
It seems like a problem that must have be solved numerous times before, anyone willing to share their solution(s)?
Thanks
AJ
Whilst alternative solutions and tricks are great, I think you're severely overcomplicating a simple problem. It's completely common and acceptable to include certain command-line switches in a program for testing/evaluation purposes. I would simply include a command line switch like this that accepts an ISO timestamp:
./myprogram --debug-override-time=2014-01-01Z12:34:56
Then at startup, if set, subtract it from the current system time, and indeed make a local apptime() function which corrects the output of regular system for this, and call that everywhere in your code instead.
The big advantage of this is that anyone can reproduce your testing results, without a big readup on custom linux tricks, so also an external testing team or a future co-developer who's good at coding but not at runtime tricks. When (unit) testing, that's a major advantage to be able to just call your code with a simple switch and be able to test the results for equality to a sample set.
You don't even have to document it, lots of production tools in enterprise-grade products have hidden command line switches for this kind of behaviour that the 'general public' need not know about.
There are several ways to query the time on Linux. Read time(7); I know at least time(2), gettimeofday(2), clock_gettime(2).
So you could use LD_PRELOAD tricks to redefine each of these to e.g. substract from the seconds part (not the micro-second or nano-second part) a fixed amount of seconds, given e.g. by some environment variable. See this example as a starting point.
Is there a good 'how-to' or 'getting started' guide for getting started using g++ and gdb?
Some background. Decent programmer, but so far I have done everything on Windows in Visual Studio.
I have a little experience using terminal to compile files (not much beyond a .h and 1 or 2 .cpp). But nothing beyond that.
Anyone know of a good primer on on how to get started coding on Linux?
Read some good books, notably Advanced Linux Programming and Advanced Unix Programming. Read also the advanced bash scripting guide and other documentation from Linux Documentation Project
Obviously, install some Linux distribution on your laptop (not in some VM, but on real disk partitions). If you have a debian like distribution, run aptitude build-dep gcc-4.6 gedit on it to get a lot of interesting developers packages.
Learn some command line skills. Learn to use the man command; after installing manpages and manpages-dev packages, type man man (use the space bar to "scroll text", the q key to quit). Read also the intro(2) man page. When you forgot how to use a command like cp try cp --help.
Use a version control system like git, even for one person tiny projects.
Backup your files.
Read several relevant Wikipedia pages on Linux, kernels, syscalls, free software, X11, Posix, Unix
Try hard to use the command line. For instance, try to do everything on the command line for a week or more. Avoid using your desktop, and possibly your mouse. Learn to use emacs.
Read about builder programs like GNU make
Retrieve several free software from their source code (e.g. from sourceforge or freecode or github) and practice building and compiling them. Study their source code
Basic tips to start (if a command is not found, you need to install the package providing it) in command line (in a terminal).
run emacs ; there is a tutorial menu; practice it for half an hour.
edit a helloworld.c program (with a main calling some hello function)
compile it with gcc -g -Wall helloworld.c -o helloworld; improve your code till no warnings are given. Always pass -Wall to gcc or g++ to get almost all warnings.
run it with ./helloworld
debug it with gdb ./helloworld, then
use the help command
use the b main command to add a breakpoint in main and likewise for your hello function.
run it under gdb using r
use bt to get a backtrace
use p to print some variable
use c to continue the execution of the debugged program.
write a tiny Makefile to be able to build your helloworld program using make
learn how to call make (with M-x compile) and gdb (with M-x gdb) from inside Emacs
Learn more about valgrind (to detect most memory leaks). Perhaps consider using Boehm's GC in some of your applications.
You've got a lot of things to learn. I won't give you details, but as someone who's done unix and c/c++ development for a couple decades now I'll try to give you some topics to start with.
My main advice is to start experimenting. Write the most trivial program you can in C or C++ (something that prints "Hello there, world!" is traditional) and figure out how to compile and run it from the command line. Then, once you've got a compiled version, start it up under the debugger and play around with breakpoints, printing expressions, etc, etc. Once you've got this simplest program up and running and you sort of understand what the debugger is telling you, add a class, function, struct, or whatever else feels like a good small step and go through the cycle again. You'll proceed much faster this way than if you start with a very large program.
Still at a very high level, here are a handful of topics you'll need to figure out at least a bit about. Note that the "learn by starting small" approach works well for any of the topics below.
Running g++: it has pretty good online documentation for the command line syntax, and although you're bound to find it intimidating at first, try to look for the simplest starting point.
Find a text editor to use. Vim and emacs are traditional (and very very powerful) but both have a relatively steep learning curve. If you have someone around to help you, that's so much the better. There are other alternatives, but as an emacs user myself, I'm afraid I'm not that familiar with them.
Get familiar with gdb. It's an incredibly powerful tool for understanding your program. Again, it has extensive online documentation that will repay close reading.
Some familiarity with standard unix commands will be useful: ls, cd, and moving basics of the navigating unix directories; grep for quickly searching source files.
You'll have to get used to the command line approach versus the ide approach. The former is the traditional unix developer model, where you put together the operations you want from a collection of other tools, rather than having the ide hide most of this knowledge from you.
If your project is multi-file, and especially if it's a full-semester project, you might also consider learning something about the following topics.
Make is a tool for describing how to compile and link a multi-file project so that you don't have to remember how to do it by hand each time. Make, unfortunately, has a well-deserved reputation for being tricky to use, but this is mostly true in very large projects spanning multiple directories, and there are probably good simple examples online.
I would strongly consider making use of a source code control system such as git or hg, even for a few relatively small project. It's so much safer to have an archived version of what you've done so that you can back up quickly. Both git and hg are overkill for a small one-shot project, but they are worth learning on their own. Conventional wisdom as I understand it today is that they're very similar in philosophy and core functionality, but that hg is definitely a bit more consistent at the command line level, and therefore easier to start with.
I suspect this is rather intimidating, especially if you've got effectively no exposure to a unix command environment before. I re-emphasize my first piece of advice above: learn by starting simple and experimenting. This minimizes the amount of new stuff you're having to wrap your head around at any given point in time.
User commands can be given the -complete=shellcmd option. This turns out to be quite disappointing, since instead of working in the same way as vim's built-in completion for its :! command, it just repeatedly completes the names of shell commands in the path.
I'd like to write a command that completes command names and then files. Is there a convenient way to do this, either with built-in vim functionality or via an addon?
Why not leverage bash-completion while you're at it? You should be able to fork a bash shell with /etc/bash_completion sourced and talk to it over pipes to interrogate it. That way you'd have all the shell goodies.
Of course, you could do as I do and C-z suspend Vim to drop back to your shell instead.
I ended up writing a vim addon called shell_complete. It's not really super-megazord-awesome, but it does do pretty much what I was looking for, which is to more or less emulate the completion used by :!. This is to say that it's sort of "pseudo shell completion": it completes commands for the first "word", and files from there out. Basic \ escaping of spaces seems to work okay.
pseudo shell completion
Installation of shell_complete is sort of complicated, especially for those not familiar with vim-addon-manager. The addon has a couple of dependencies, and none of them are "published", which is to say that VAM doesn't know about them yet. You can let VAM know about them by installing (via git clone; see the docs for more info) another addon I've called tt_addons. Anyway, once you've done this, you should be able to just :ActivateAddons tt_addons and then :ActivateAddons shell_complete.
If you're not using VAM, you'll have to download (or git clone, more likely) all of the related modules, and then mix them into your vim directory, or make them pathogen bundles, or what have you. If you actually end up wanting to do this, it's likely worth your while to start using VAM. If you're some sort of curmudgeonly Luddite who refuses to do so, let me know and I might put the thing up on vim.org if there's decent interest.
I think it's pretty obvious (at least from the docs) how to use shell_complete, but if you would like to see it In Action you can check out my reput addon, which uses it to do completion on its :RePutShell command. reput is also currently only available through github, and the same caveats apply with respect to installation via VAM.
Actual shell completion using the shell
For the record, I think sehe's suggestion about using the shell itself to do completion is totally flash. I actually spent quite a while figuring out how to do this, and have determined that it's possible, at least in theory. I initially thought it would be easier to do it this way than by doing what shell_complete does, but it turns out that bash (like vim) doesn't provide any programmatic way to access its completion facilities, so you end up basically having to reimplement them in bash, after scraping the configuration using grep and friends. Bash sucks a lot for this sort of thing, so I'm cowardly refusing to fight that battle, at least for the moment.
Should someone be so brave/foolish as to take up this standard, they may avail themselves of the chronicles of my travails. I managed to get it to do completions that are handled by custom completion functions. However, this is only a small part of the puzzle, because bash also provides about 6½ other ways to do completion. It does sort of complement the functionality provided by shell_complete, so it might be worthwhile to synergistically merge the two into a sort of awkward, drunken Voltron of vim shell completion.
Is there a programming language which can be programmed entirely in interactive mode, without needing to write files which are interpreted or compiled. Think maybe something like IRB for Ruby, but a system which is designed to let you write the whole program from the command line.
I assume you are looking for something similar to how BASIC used to work (boot up to a BASIC prompt and start coding).
IPython allows you to do this quite intuitively. Unix shells such as Bash use the same concept, but you cannot re-use and save your work nearly as intuitively as with IPython. Python is also a far better general-purpose language.
Edit: I was going to type up some examples and provide some links, but the IPython interactive tutorial seems to do this a lot better than I could. Good starting points for what you are looking for are the sections on source code handling tips and lightweight version control. Note this tutorial doesn't spell out how to do everything you are looking for precisely, but it does provide a jumping off point to understand the interactive features on the IPython shell.
Also take a look at the IPython "magic" reference, as it provides a lot of utilities that do things specific to what you want to do, and allows you to easily define your own. This is very "meta", but the example that shows how to create an IPython magic function is probably the most concise example of a "complete application" built in IPython.
Smalltalk can be programmed entirely interactively, but I wouldn't call the smalltalk prompt a "command line". Most lisp environments are like this as well. Also postscript (as in printers) if memory serves.
Are you saying that you want to write a program while never seeing more code than what fits in the scrollback buffer of your command window?
There's always lisp, the original alternative to Smalltalk with this characteristic.
The only way to avoid writing any files is to move completely to a running interactive environment. When you program this way (that is, interactively such as in IRB or F# interactive), how do you distribute your programs? When you exit IRB or F# interactive console, you lose all code you interactively wrote.
Smalltalk (see modern implementation such as Squeak) solves this and I'm not aware of any other environment where you could fully avoid files. The solution is that you distribute an image of running environment (which includes your interactively created program). In Smalltalk, these are called images.
Any unix shell conforms to your question. This goes from bash, sh, csh, ksh to tclsh for TCL or wish for TK GUI writing.
As already mentioned, Python has a few good interactive shells, I would recommend bpython for starters instead of ipython, the advantage of bpython here is the support for autocompletion and help dialogs to help you know what arguments the function accepts or what it does (if it has docstrings).
Screenshots: http://bpython-interpreter.org/screenshots/
This is really a question about implementations, not languages, but
Smalltalk (try out the Squeak version) keeps all your work in an "interactive workspace", but it is graphical and not oriented toward the command line.
APL, which was first deployed on IBM 360 and 370 systems, was entirely interactive, using a command line on a modified IBM Selectric typewriter! Your APL functions were kept in a "workspace" which did not at all resemble an ordinary file.
Many, many language implementations come with pure command-line interactive interpreters, like say Standard ML of New Jersey, but because they don't offer any sort of persistent namespace (i.e., when you exit the program, all your work is lost), I don't think they should really count.
Interestingly, the prime movers behind Smalltalk and APL (Kay and Iverson respectively) both won Turing Awards. (Iverson got his Turing award after being denied tenure at Harvard.)
TCL can be programmed entirely interactivly, and you can cetainly define new tcl procs (or redefine existing ones) without saving to a file.
Of course if you are developing and entire application at some point you do want to save to a file, else you lose everything. Using TCLs introspective abilities its relatively easy to dump some or all of the current interpreter state into a tcl file (I've written a proc to make this easier before, however mostly I would just develop in the file in the first place, and have a function in the application to resources itself if its source changes).
Not sure about that, but this system is impressively interactive: http://rigsomelight.com/2014/05/01/interactive-programming-flappy-bird-clojurescript.html
Most variations of Lisp make it easy to save your interactive work product as program files, since code is just data.
Charles Simonyi's Intentional Programming concept might be part way there, too, but it's not like you can go and buy that yet. The Intentional Workbench project may be worth exploring.
Many Forths can be used like this.
Someone already mentioned Forth but I would like to elaborate a bit on the history of Forth. Traditionally, Forth is a programming language which is it's own operating system. The traditional Forth saves the program directly onto disk sectors without using a "real" filesystem. It could afford to do that because it didn't ran directly on the CPU without an operating system so it didn't need to play nice.
Indeed, some implementations have Forth as not only the operating system but also the CPU (a lot of more modern stack based CPUs are in fact designed as Forth machines).
In the original implementation of Forth, code is always compiled each time a line is entered and saved on disk. This is feasible because Forth is very easy to compile. You just start the interpreter, play around with Forth defining functions as necessary then simply quit the interpreter. The next time you start the interpreter again all your previous functions are still there. Of course, not all modern implementations of Forth works this way.
Clojure
It's a functional Lisp on the JVM. You can connect to a REPL server called nREPL, and from there you can start writing code in a text file and loading it up interactively as you go.
Clojure gives you something akin to interactive unit testing.
I think Clojure is more interactive then other Lisps because of it's strong emphasis of the functional paradigm. It's easier to hot-swap functions when they are pure.
The best way to try it out is here: http://web.clojurerepl.com/
ELM
ELM is probably the most interactive you can get that I know of. It's a very pure functional language with syntax close to Haskell. What makes it special is that it's designed around a reactive model that allows hot-swapping(modifying running code(functions or values)) of code. The reactive bit makes it that whenever you change one thing, everything is re-evaluated.
Now ELM is compiled to HTML-CSS-JavaScript. So you won't be able to use it for everything.
ELM gives you something akin to interactive integration testing.
The best way to try it out is here: http://elm-lang.org/try
I recently started picking up vi, going through some tutorials and trying to get used to it. But I still have some questions about it.
It seems to be nice for small one file changes, but as soon as I start to try doing bigger things it seems to be lacking. For example I'm used to have code formatting, import organizing, simple overview over all packages and other things that an IDE gives me. I saw some tutorials on how to use vi as an IDE, but it felt awkward at best.
Now I'm just wondering, what are the typical use cases for vi? Is it typically used to edit small files, or can it be used for larger projects? And if you use it in larger projects, how do you make it work? Or would it be a lot easier to use an IDE with vi keybindings?
People use non-IDE editors like Vi(m) for coding due to the following reasons,
They are non-distractive, allows you to concentrate on the job.
They do not clutter you screen area, offers you more code space
They are faster
They have better/faster/cooler text manipulation at the stroke of the finger
You happen to move your hands out of the keyboard less to hold the mouse, drag it here and there and click.
They also have the flexibility to support other tools like debugger, document viewer, etc.
They also have ways to get things like code folding, etc.
For a normal programmer whose ideal work cycle is sit, write code, test, debug, more code, test debug.. Vi offers a simplistic yet powerful environment to get the work done faster and more efficiently.
For someone who had years of using some IDE, it might be like using some prehistoric tool, but once they have been through the initial days, then there is no looking back. They'd feel like there is no better thing.
Why, oh WHY, do those #?#! nutheads use vi?
I've haven't done anything that you'd call a big project in python (only little test scripts), but I use Vim exclusively for writing large embedded C applications and I have never really felt the need for an IDE.
Vim is fast to start up, extremely fast to use and (with a bit of customisation) can do most things that an IDE can do. It'll do code completion, code auto-indentation and reformatting and it is very good at refactoring. The project plugin http://www.vim.org/scripts/script.php?script_id=69 makes it very easy to manage projects with lots of files and the taglist plugin http://www.vim.org/scripts/script.php?script_id=273 is great for browsing source code. It also allows you to have the C code open side-by-side with python code, assembly code and latex documentation without having to use a different tool for each.
Overall though, I think there is one really valuable thing that Vim gives you that very few other editors do and I would find it very hard to lose that: I can have a window split into three parts like this:
Each of the subwindows can either show a separate file or (as in the screenshot) a separate part of the file. I imagine emacs can do this (although I could be wrong), but I doubt many IDEs can. This can be invaluable for refactoring and for referring to other parts of the file.
I've used VI(M) and Emacs as my primary editors for years... I've tried switching to IDEs but find they can't get out of my way enough. I always end up back in VI(M) or Emacs after a while. One major reason is that I find my hands need to leave the keyboard too much in IDEs as they require the mouse too often... And I'm too lazy to setup my own shortcuts for everything.
Here's why I use it.
it's fast to start up
it's available across multiple platforms and is on all Unix machines
it's fast to use (keystrokes for common operations, operations based around words/paragraphs etc.)
However, I use IDEs for large scale development work, since I can't believe they can be beaten for productivity, given their code-awareness and refactoring capability. I use Eclipse, but I plug a VI editor emulator into it. See this answer for more details.
I saw some tutorials on how to use VI as an IDE, but it felt awkward at best.
There's one thing to do at the very beginning: throw your sense of aesthetics overboard. You will never get the same kind of graphical experience in VIM as you do in a decent IDE.
That said, VIM actually does offer many of the features of a full-blown IDE and has a lot of advantages besides. I use VIM for almost all of my developing work and all of my text documents (using Vim-LaTeX) – even though I've actually paid money for Apple’s office suite, iWorks.
There's one point that's still nagging me, though: I can't get IntelliSense to work. For me, that's a huge problem, especially when using languages such as VB, C# or Java, for which excellent IDE support exists.
So, the learning curve for VIM is steep but once you've passed a certain point it's everything but awkward. In fact, compared to VIM's editing experience, you will find that it's the IDEs that suddenly feel awkward because while they're good at bookkeeping stuff, they suck at editing.
I frequently work on a remote system, programming for a cluster, or editing config files on a headless box. All of these could be done with a file transfer, a regular IDE, and another file transfer, but it is so much faster to just use VIM through SSH.
You can find it (or easily install it) almost anywhere - Runs on all systems that can implement the standard C library, including UNIX, Linux, DOS, Windows, Mac, BeOS, and POSIX-compliant systems.
The ability to log into a remote server and confidently edit a config file is priceless.
Vim is not very good at code awareness. What it is good at is text manipulation. It provides you with the tools for you to edit text, not to edit for you. If you just do "small" edits and don't read up on the occasional "vim commands you wish you knew" you'll never understand the power provided.
code formatting
Formatting is done with =. You can also point equalprg to an external program to do the formatting for you.
import organizing
Vim won't be able to remove unused imports but if you select the import lines
:'<,'>sort
simple overview over all packages
:vs .
This will open a vertical window containing the current working directory :)
In conclusion, vim can't replace your IDE but it will let you edit your text.
As usuall: Choose your tool depending on the work you have to do! ;-) I'm using Vim and Visual Studio. You don't have to choose only one.
If you need an IDE, use an IDE. Vim is made for text manipulation.
It supports any languages. Try to edit a Ruby script with Visual Studio.
Performance on large files is much better than for most IDEs. Try to analyze a logfile having 100k lines and more using your IDE.
It runs on many plattforms and you can use it via a remote console, if you have to edit files on a server.
...
IDEs are specialized tools, which are good for problems they are made for. Vim is an incredible flexible and powerfull generall purpose tool.
This is obviously a hot topic so I'm going to give a list of reasons why I prefer IDE editor for coding.
1) I prefer to be able to fold sections of my code (I don't know whether Vi(m) can) without having to remember the command to do so or type it.
2) I prefer one click compile button/command as opposed to having to remember my compile command and it's library of options
3) I prefer the easier highlight copy paste operations without having to........ aah sod it!!
Well by now you get the picture. I'm not knocking Vi(m) I just think having your the elements and functionality of your coding environment visually represented makes for a clearer head and encourages exploration of options you may not know exists.
I much prefer vim over vi and prefer not to lump them together. Vim provides some features that are really handy and not always immediately obvious. Already mentioned was screen splitting. Also you may want to checkout ctags or exctags. These allow you to jump around the code base from within vim. I can place the cursor over a called method, jump to the definition, jump to a definition within that method, etc. Very powerful for tracing down bugs. Cscope is another similar program.
Vim will also allow you to run arbitrary shell commands within the environment and has powerful search and replacement features.
So an IDE will provide most of those, what will vim provide over a traditional GUI based IDE? That's easy, it runs on the CLI.
This allows me to login to my dev box, run screen (definitely something to checkout for the unfamiliar: http://www.gnu.org/software/screen/) and run multiple vim sessions within my screen session on my dev box. Now when I leave work for the day, I can leave my work session just as I left it, even while running builds. When I come into work the next day I can connect from my environment as if I never left.
Another reason to like vim or vi in particular, is that it can be found on virtually any Unix environment.