Related
So I have a computer. It has programs on it already. If I delete those programs, I would be left with an operating system that is able to run commands. I could create my own programs from that point, but I would be limited to the constraints of the operating system already loaded onto the machine. What I would like to do is remove the operating system from the computer entirely and be left only with a blank screen and a cursor where I could type whatever I want. I want to be able to create my own program without having to run an operating system program behind it. I do not understand how the physical machine would be able to process the strings of characters that I type into it and produce its own response, which would then be displayed on the screen, but obviously someone has done it before, otherwise I would not have the machine that I am typing on right now.
(I apologize for the run on sentences but I do not know how to say what I want to say right now.)
My goal here is to have a computer, kind of like the Apple 2, where the only thing that I could do with it is type into a text line and see characters pop up on the screen. My goal on top of that goal is to develop a program that would hide in the background of the machine, so that there would still only be a cursor on screen, but the program would make it so that when I type any simple question into the screen, such as, "How are you feeling today?", I would receive a response like, "I am doing quite well, thank you. How are you?".
Does anybody have any idea how I would be able to start this project properly?
If you need to ask this question, you need to learn more than one answer on SO can provide.
Operating system is needed even to get the cursor thingy on screen.
If you are serious about the idea - you might want to start with a microcontroller, such as Arduino. They are more powerful than Apple 2 and they will allow you to write programs and boot straight into them. Adding some kind of terminal IO will not be hard - at least comparing to bootstrapping a program on an actual PC.
A good starting point for a project like this is to learn about operating systems in general. It is a vast topic but you don't have to know everything.
When we speak of an operating system we have in mind a large system that provides capabilities like managing memory, reading and writing files to permanent storage and interacting with input and output such as keyboards and displays. We are also usually thinking of a large number of higher level software applications. Think about commands like dir or ls as programs that come along with the operating system. Of course with a GUI based OS we also have windows and buttons and a wide variety of controls to consider.
The good news is that in order to get started you don't need to be an expert on everything and you certainly don't have to start with a full-blown OS.
The other good news is that the topic can be broken down into byte-sized pieces. A great introduction to the fundamentals you will need is Charles Petzold's Code The Hidden Language of Hardware and Software
Petzold begins with discussions of the inventions of Morse code and Braille, adds electricity, number systems, Boolean logic, and the resulting epiphanies required to put them all together economically. With these building blocks he builds circuits, relays, gates, switches, discusses the inventions of the vacuum tube, transistors, and finally the integrated circuit.
The last portion of the book contains a grab bag of subjects such as implementation of floating point math, operating systems, and the various refinements that have occurred in the latter half of the twentieth century.
Once you have a feel for the fundamentals a good next step in learning about operating systems is to study one that provides as few capabilities as possible. Take a look at MINIX
MINIX originally was developed in 1987 by Andrew S. Tanenbaum as a teaching tool for his textbook Operating Systems Design and Implementation. Today, it is a text-oriented operating system with a kernel of less than 6,000 lines of code. MINIX's largest claim to fame is as an example of a microkernel, in which each device driver runs as an isolated user-mode process—a structure that not only increases security but also reliability, because it means a bug in a driver cannot bring down the entire system.
Have fun.
I am a non CS/IT student, but having knowledge of C, Java, DS and Algorithms. Now-a-days I am focusing on operating system and had gained some of its concepts. But I want some practical knowledge of it. Merely writing algo code in java/c has no fun in doing. I have gone through many articles where they mentioned we can customize source code of Linux-kernel.
I want to start customizing the kernel as I move ahead in the learning of OS concepts and apply the same. It will make two goals achievable 1. I will gain practical idea of the operating system 2. I will have a project.
Problem which I face-
1. From where to get the source code? Which source code should I download? Also the documentation if possible.
https://www.kernel.org/
I went in there but there are so many of them which one will be better?
2. How will I customize the code once I have it?
Please give me suggestions with detail about how I should start this journey (of changing source code to customize Linux).
Moreover I am using Windows 8.
I recommend first reading several books on OSes and on programming. You need a broad CS culture (if possible get a CS degree)
I am a non CS/IT student,
You'll better become one, or else spend years of work to learn all the stuff a CS graduate student has learnt.
First, you need to be very familiar with Linux programming on user side (application programs). So read at least Advanced Linux Programming and study the source code of several programs, including shells (and some kind of servers). Read also carefully syscalls(2). Explore the state of your kernel (e.g. thru proc(5)...). Look into https://kernelnewbies.org/
I also recommend learning several programming languages. You should in particular read SICP, an excellent introduction to programming. Read also some book like programming language pragmatics. Read something about continuation and continuation passing style. Read the Dragon book. Read some Introduction to Algorithms. Read something about computer architecture and instruction set architecture
Merely writing algo code in java/c has no fun in doing.
But the kernel is also written in C (mostly) and full of algorithmic code. What makes you think you'll get more fun in it?
I want to start customizing the kernel as I move ahead in the learning of OS concepts and apply the same.
But why? Why don't you also consider studying and contributing to some user-level code
I would recommend first reading a good book on OSes in general, notably Operating Systems: Three Easy Pieces. Look also on OSdev.
At last, the general advice about kernel programming is don't. A common mistake is to try adding code inside the kernel to solve some issue that can and should be solved in user-land.
How will I customize the code once I have it?
You probably should not customize the kernel, but if you did you'll use familiar tools (a good source code editor like emacs or vim, a compiler and linker on the command line, a build automation tool like make). Patching the kernel is similar to patching some other free software. But testing your kernel is harder (because you'll often reboot).
You'll also find several books explaining the Linux kernel.
If you still want to customize the kernel you should first try to code some kernel module.
Moreover I am using Windows 8.
This is a huge mistake. You first need to be an advanced Linux user. So wipe out Windows from your computer, and install some Linux distribution -I recommend Debian- (and use only Linux, no more Windows). Become familiar with command line.
I seriously recommend to avoid working on the kernel as your first project.
I strongly recommend looking at some existing user-land free software project first (there are thousands of them, notably on github, e.g. choose some package in your distribution, study its source code, work on it, propose the patch to the community). Be able to build from source code a lot of things.
A wise man once said you "must act your way into right thinking, as you cannot think your way into right acting". In your case, you'll need to act as an experienced programmer would act, which means before we write any code, we need to answer some questions.
What do we want to change?
Why do we want to change it?
What are the repercussions of this change (ie what other functions - out of all the 10's of millions of lines of source code - call this function)?
After we've made the change, how are we going to compile it? In other words, there is a defined process for this. What is it?
After we compile our new kernel/module, how are we going to test it?
A good start, in addition to the answer that was just posted, would be to run LFS (Linux from Scratch). Get a successful install of that and use it as a starting point.
Now, since we're experienced programmers, we know that tinkering with a 10M+ line codebase is a recipe for trouble; we need a bit more direction than that. Here's a list of bugs that need to be fixed: https://bugzilla.kernel.org/buglist.cgi?chfield=%5BBug%20creation%5D&chfieldfrom=7d
I, for one, would be glad to see the one called "AUFS hangs on fanotify" go away, as I use AUFS with Docker on a daily basis.
If, down the line, you decide you'd rather hack on something besides the kernel, there are plenty of other options.
From your question it follows that you've already gained some concepts of an operating system. However, if you feel that it's still insufficient, it is OK to spend more time on learning. An operating system (mainly, a kernel) has certain tasks to perform like memory management (or memory protection), multiprogramming, hardware abstraction and so on. Neither of the topics may be neglected - they are all as important. So, if you have some time, you may refer to such useful books as "Modern Operating Systems" by Andrew Tanenbaum. Special books like that will shed much light on all important aspects of a modern OS. Suffice it to say, Linux kernel itself was started by Linus Torvalds because of a strong inspiration by MINIX - an educational project by A. Tanenbaum.
Such a cumbersome project like an OS kernel (BSD, Linux, etc.) contains lots of code. Many people are collaborating to write or enhance whatever parts of the kernel. So, there is a common and inevitable need to use a version control system. So, if you have an intention to submit your code to the kernel in future, you also have to have hands on with version control. Particularly, Linux relies on Git SCM (software configuration management - a synonym for version control).
So, once you have some knowledge of Git, you can install it on your computer and download Linux source code: git clone https://github.com/torvalds/linux.git
Determine your goals at Linux kernel modification. What do you want to achieve? Perhaps, you have a network card which you suspect to miss some features in Linux? Take a look at the other vendors' drivers and make an attempt to fix the driver of interest to include the features. Of course, this will require some knowledge of the HW, and, if the features are HW dependent, you will unlikely succeed to elaborate your code without special knowledge. But, in general, - if you are trying to make an enhancement, it assumes that you are an experienced Linux user yourself. Otherwise, how will you understand that some fixes/enhancements/etc. are required? So, I can't help but agree with the proposal to postpone Windows 8 for a while and start using some Linux distribution (eg. Debian).
If you succeed to determine your goals (eg. if you find a paper describing some desired changes in Linux kernel or if you decide to enhance some device drivers / write your own), you will be able to try it hands on. However, you still might need some helpful books, but, in this case, some Linux-specific ones. Also, writing C code for the kernel itself will require one important detail - you will need to comply with a so called coding standard, otherwise Linux kernel maintainers will not be able to accept your patches.
So, I made an attempt to outline some tips based on your current question. Of course, the job of kernel development has far more broad prerequisites, but these are which are just obvious.
everybody out there!
I'm currently seeking for a technical solution to create a nice literate programming environment. Unfortunately, most editors are too much hard coded, and their functionalities just cover most famous needs, and can't cleanly cover special needs.
I came to Emacs (later after some others), but I also came to numerous troubles with Emacs (I will not talk about these, this is not the topic).
However, there is one thing I like with Emacs and which was indeed matching what I was looking for: it exposes a full text model to a scriptable environment, and the overall UI is designed so it is well suited to either graphical UIs or text UIs (because it is mostly text based). And last but not least, this is scriptable with a kind of LISP, and LISP indeed seems a good choice to me, in the area of text manipulation and interpretation.
I've searched the web for a text editor which would expose a full text model to a scriptable environment, but I have not found anything. I guess this is not an everyday request on the web, so it is probably better to ask some humans about it, better than to ask a robot.
I was, but in short, I'm looking for: an editor which exposes a full text model [*], and which exposes this model to a script engine (preferably LISP, but I would enjoy Python as well, or any others after all).
[*] Talking about text model, I mean: text attributes (optionally font face), text visibility, text read-write property, and text content iteration, at a level as low as the character basis.
Have a nice day! :)
--
Yannick Duchêne
JEdit seems to be very scriptable with Java, BeanShell, Jython and other languages targeting the JVM. Most of its functionality is implemented with OSGI plugins. If you really like LISP, maybe you could even try with Clojure! :-)
Emacs, Climacs, Portable Hemlock (and to some extent Hemlock).
I am sure there are other editors around that exposes a full text model to a script engine that are NOT in "the emacs family", but I don't know them.
Oh, yes, there's the VMS editor framework, but I cannot recall its name.
What Vatine said, plus there's a very minimal Scheme editor built into Fluxus, which I extended with Emacs key-bindings (in my personal copy), so I know it would work as something close to a stubbed implementation (if you rip out all the OpenGL stuff).
Edit:
Looks like I was working with fluxus-0.8, which doesn't even seem to be on the site anymore. If you end up needing to go that low-level to start, let me know and I'll send it over.
Not sure if this is useful, but there is a long list of Emacs-like editors: http://www.finseth.com/emacs.html
Btw., Craig A. Finseth also wrote a book on implementing an Emacs-like editor: http://www.finseth.com/craft/
The Book as PDF.
Report of an (unsuccessfully) ending quest :
Although a possible technical choice I could figure will not work for me (see later), I still point it here, if this can ever be useful to someone running UNIX-Like (I'm running Windows).
Context and state of the “ art ” : near to all (or all) so called Emacsen and Emacs clone, has nothing to compare with Emacs. They just mimics terms like major mode an minor mode, mimics key-bindings, and most of time also, the UI look and feel. But the core is not there. I've learned these are called “ Emacs Ersatz ”.
Disclaimer : for some reasons, I have not tested Climax and Hemlock, so the latter comment does not apply to these.
EFuns : the last one I came to, was EFuns, but unfortunately, I could not compile it on Windows (I suspect something is wrong with the sources, some directory are missing in the archive). Interested parties may get it here : EFuns, an Emacs-like scripted in OCaml. Fortunately for UNIX-Like users, binaries are provided (not for Windows).
Implementations List : to complete the list Rainer Joswig pointed to, here is another one, shorter, but more up-to-date : [ Sorry I can't post this link, it seems I'm not allowed to post more than one link - I'm sorry for interested parties (sad) ]
I have written a little script which retrieves pictures and movies from my camera and renames them based on their date and then copies them on my harddrive, managing conflicts automatically (same name? same size? same md5?)
Works pretty well.
But this is ONE script.
From time to time I need to check if a picture is already hidden somewhere in a volume, so I'd like to apply the "conflict manager" only. I guess if I had properly followed the unix spirit of tiny single-task tools, I could do that.
What are the best resources, best practices and your experience on the subject?
Thank you.
Edit : Although I'd love to read unix books and have a deep understanding of the subject, I am looking for the Great Principles first. Plus I tend to limit myself to online resources.
I would look at the book called The Art of Unix Programming.
I've found that most code doesn't start out being reusable, it evolves to be. Take your existing code and factor out the "conflict manager" portion into its own function or program, then call that program instead of having it be a part of your original application. After that you'll be able to reuse that part of your code that you have a need to reuse. Sometimes it's impossible to design software up front for reusability because you simply don't know which parts you'll want to reuse.
As for resources, it seems like the store shelves are packed with books for Linux desktop users and system administrators, but it's hard to find good Linux programming books. A few good ones:
Beginning Linux Programming
Professional Linux Programming
Linux Programming by Example: The Fundamentals
The Linux Programmer's Toolbox
Lastly, Eric Raymond has made The Art of Unix Programming available online for free.
Check out this book:
The Art of Unix Programming by Eric S. Raymond
http://www.amazon.com/UNIX-Programming-Addison-Wesley-Professional-Computing/dp/0131429019
Here is his website:
http://www.catb.org/~esr/writings/taoup/
Personally, whenever I see that the script I'm planning to write will be longer than a dozen lines, I use python instead of shell script. One neat trick with python scripting is that it's very easy to program in a style where you create both "unix spirit" command-line tools and libraries. E.g. for your "conflict manager", create a file (python module) and put the functionality in functions and/or classes, and then at the end you can put a python "main" function (the usual if __name__=='__main__': dance) where you parse command line options (use the builtin OptionParser module for this, it's very nice!) and use the functionality in the functions/classes.
This way you can use the utility both as a stand-alone command line program, or you can import the module in another python script and use the functionality defined there via functions/classes rather than parsing input.
Start with wikipedia (Dataflow programming)
The book Software Tools (amazon) by Kernighan and Plauger is a classic on this subject. I think it should be required reading for any serious student of software development.
- the art of UNIX programming - is quite a nice book ok "the unix way", in so far as one exists. OTOH if the way is "do as little work as gets your job done", you may already be there. :)
I think some of the keys for good gnu code, are:
Handling the system signals properly, like deattaching hard drive files if SIGTERM is received.
Proper use of pipes and standard input/output
Follwing common command line flag rules
I would also recommend this book. Pretty old, but I think is quite clear explaining the principles of unix.
I'm thinking about creating a small language that is very easy to type on a mobile phone (J2ME),
What is the more appropriate language to implement in order to run it inside a mobile phone (j2me always)? Appropriate meaning, small/easy syntax, easy to type in a mobile phone.
Is it lisp? Some sort of Basic/Python/Ruby (I think not...)? Or another new (can you propose a new syntax?)?
I am the author of just such a language: Hecl, at http://www.hecl.org . In order to make quite applications easier, I also created a site where you can build simple apps through a web interface: http://www.heclbuilder.com . I also wrote an article discussing the implementation of the language:
http://www.welton.it/articles/hecl_implementation
Other languages that are worth looking at include Lua, and Javascript, both of which have mobile implementations.
If you include editor support (nesting structures, indented display, balancing, ...) then some form of LISP would be relatively straightforward to implement and use. I've seen screenshots (but can't find them now) of a LISP-based language for live interactive-performance programming. It used indented, shaded rectangular areas on the screen (instead of parentheses) to show nesting of structure.
I would think the design of the editor would be the biggest consideration, not the language. For instance, supporting some kind of "intellisense"-like autocompletion would be vital for saving thumbstrokes. Some kind of language sensitivity in the editor would help a lot too. For instance, when a C user types "for" the autocomplete should show an option for filling out the syntax of a loop:
for (;;) {
}
You might want to look into Hecl: http://www.hecl.org/
I'm not sure what's easy to type on a mobile phone, but the language I know with the most computing power per character is APL. As a source of syntactic or design ideas, you might prefer its modern successor, the J programming language.
On a mobile phone, you should also consider languages like Scratch (smalltalk), because the non-typing interface would be easy to use.
Also on the smartphones with drag&drop capability, it would be something good.
On the other hand, the IDE would be a lot heavier on CPU & other resources.
Forth is usually considered a legitimate contender for these kinds of requirements. And it's about as terse as can be imagined. Extensible, small and malleable. Built-in small screen editor, too.
If you want super-compact, try nano-False http://www.aldweb.com/pages/winikoff/#false
It isn't very usable, although more so than the deliberately painful Brainfuck and Whitepace. Think of it as Forth with the easy syntax made more concise ;-)
I found Quartus Forth reasonably easy to use, provided you can think in stacks, and with more Intellisense support for the API it would have been much more productive. For prototyping little algorithms on the Palm I preferred Plua or Lispme. The LispMe environment is worth studying anyway because it provided good use of lists for finding keywords and so eased GUI programming
The big decision you have to make is whether you expect users to just use a phone numeric keypad or be able to type in reasonable approximations to a full keyboard. One of the huge benefits of the Palm was the high-quality full-size folding keyboards which I sadly miss (and hope someone makes an iPhone accessory to connect). If you don't have a full keyboard, make use of selectors for verbs so they can use picking actions rather than having to type in words. Consider the amount of code typed in traditional code for the framework classes and methods compared to the user code.
When I go about dreaming about a language, I think about what features are important to me at the time I'm dreaming. Only once you figure out what features are important to you can you come up with the best answer to what syntax. For example, if you want named parameters, it greatly influences your design choice about how method calls look (a la Objective-C or Python).
Designing a language can be a really fun task. I encourage you to step back and ask yourself "Do I really like how this is done in X?" (substituting some language name). If that's something you've always loved, steal it. If not, look elsewhere. Create your ultimate mashup of what you love, and leave out what you hate!
Lisp would be difficult to type because of all the ()s, although joel.neely's answer demonstrates one way of working around that problem.
So if you want to use an existing language you might want to look at which ones use least unusual characters.
Then there's the screen size issue. The more verbose the language the less code you're going to be able to fit onto the screen at once. What kind of devices are you aiming at? Smartphones with big screens (a limited audience) or 240x240 pixel feature phones?
Bear in mind that the interpreter/VM for your language will have to fit into a small amount of memory and performance may not be very good.
Brainfuck has only 8 characters -- very easy to type in on a mobile phone.
Of course, understanding and doing stuff with it... not so easy. But it satisfies the requirement....
Basic is very easy.
I would stay away from lisp. Unless you want to give your mobile users a headache on top of the headache they have from radio waves.