Where can I start looking to better understand how computers work? - security

I've been trying to figure out what computer field I want to go into later on in life. College is just around the corner for me and I've considered looking into Computer Engineering, Software Engineering, etc.
Lately, I've been looking into computer security systems and exploitations of such (purely for educational purposes, on my own property). Unfortunately, it seems to me like 99% of the people out there have no idea what they're talking about. Oftentimes, it's just "run this" or "run that" or "you can find a program that will do all that for you" - no one knows how these programs work or what exactly they do.
I find no fun or interest in using something that someone else created simply to call myself a "hacker" as most people do. In fact, I'm not even interested in hacking systems as much as HOW they do it.
My question all comes down to this.
I want to learn the ins, outs, ups, and downs of computers - everything from abstract concepts such as the internet and data transferral, to hardware. I want to know how computers store data (how the bites are organized, etc.) and what processors, etc. actually do. What is WIFI, really? Do computers communicate with light (something I picked up from a magazine that I read on a plane).
I have multiple years of computer/programming experience, but so much of what I know about computers in general is very broad. Computers send packages of information back and forth between one another, each with a header and content. Computers are composed of multiple components, each with their own function (processor, video card, RAM, hard drive(s), etc.), which I have some basic understanding of already. etc. etc. etc.
There is just so much to a computer and I don't know where to start. I'm sure some of my college classes will clear things up for me, but I'm so curious that I want to start learning as much as I can now.
This question is probably all over the place, so please ask me to clarify when necessary. I'm a little jet lagged at the moment, but I tried to write my thoughts in the quickest, most coherent way possible (I could have completely failed in the process, though).
Thanks in advance for any advice!
Justian Meyer
Please, feel free to edit the tags for this question. The current ones are terrible.
EDIT:
All these comments are making me excited :). So much to learn, so much to explore :).

To help you choose which specialization to go into, I would very highly recommend computer engineering(Known as CMPE or CE in college course books). Your classes will take you to everything you just listed, and with electives you can delve deeper into whichever aspects you wish(such as security and networking).
In CMPE you will learn both software(C, C++, and some C#) and then hardware( maybe two electrical engineering classes). Once you get to assembly programming, you will start to learn how the two combine to make up everything else in any computer or embedded system. It will take you down to the bit level of memory, CPU, data buses, I/O, and so many other things. I am just starting to do Digital Design, and its ****ing glorious. From what you described, you will enjoy being a CMPE major greatly.
There's computer science majors and software engineers; there's electrical engineers; but there is no cell phone, GPS, or computer designed without computer engineers!

Structured Computer Organization, Tanenbaum
It is a great book and explains everything from a transistor to a Java virtual machine.

These two helped me understand how the OS and memory in general works.
I believe a lot of things are derived out of these 'simple mechanics.
1.Anatomy of a program in memory
2.Pushing the limits on Windows memory

Steve Gibson of security now has been doing a series of podcasts on computer basics.
http://www.grc.com/securitynow.htm Episode 233 "Let's Design a Computer (part 1)" up to the most recent one "What We'll Do for Speed".
Every other episode he does listener feedback and those are good to listen to too.
a few times (like right now) they interrupted the series if a important security news item comes up (like when that big SSL thing broke a few months ago)
Its a really good show and I recommend starting on 233 and working your way up, then starting over on episode 1. Has also done very good series on how a computer network works and how cryptography works. (Ep 203 will blow your mind when he talks about the Boyer & Moore
method of searching)

Since you are deciding where to go exactly, to be in software development or to become expert in hardware and networking, I would like to point out that in my opinion it is two different occupations and they require two different mindsets. Good hardware experts are usually not good programmers and good programmers almost always not experts in hardware and networking. So I would say don't try to embrace both, stick to one direction which is most suitable to your mindset. To pursue two rabbits would result in catching no one.
#Justian
I see, sorry I somewhat misunderstood you. Desire to understand intricacies of how code gets processed inside of hardware is a very natural one. When in college I was reading the book "How computer works" - it is fairly simple, even somewhat primitive book about general hardware functionality. But it can get you a broad look on the topic.
Another analogy came to mind. Say linguists research internal mechanics of language, but it is neuroscientists who research on how language signals get processed in brain. Two very different occupations. This is not to discourage you from learning hardware though, this is just to underline difference between two realms.

Related

Where to Start Learning to Develop Security Systems

I'm currently a student attending College and in the future, I would really love to work on (develop) security systems like Antivirus, Firewalls, etc. I'm not sure where to start or what to start learning so that I can be successful in this field. Any advice?
Right now, I know (not necessarily good):
C, C++, Java, Python
use Linux (Ubuntu)
I know that my question might be too broad but I really have no idea what core concepts I need to familiarize/master.
My advice would be to start watching every talk that you can find from DEFCON, The Black Hat Conference, Chaos Communication Congress, ShmooCon, and other computer security conferences. This is the cheapest, easiest and fastest way to learn.
Start by taking up a Networks, Algorithms and Cryptography course. If you already have, go for more advanced versions of the same. Knowledge of networks, good algorithmic and cryptographic skills are much more important than the languages known in this case.
There are also some great Wargaming sites out there that you can jump on. In particular, I spend some time on these:
- http://hackthissite.org
- http://smashthestack.org
- http://intruded.net
- http://securityoverride.com
Those will give you a kind of 30,000 ft. view of the field of "offensive security".
I'd also recommend trying to track down hacker blogs and hackers on Twitter. (Twitter is actually a remarkably effective way to stay informed about security stuff, in my opinion.) The links above should hopefully put you on the trail.
Good luck!

Is non-programmatic software development feasible? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm currently faced with a very unusual design problem, and hope that a developer wiser than myself might be able to offer some insight.
Background
Without being too specific, I've been hired by a non-profit organisation to assist with the redevelopment of their legacy, but very valuable (in terms of social value) software. The development team is unlike any I've encountered in my time as a software developer, and is comprised of a small number of developers and a larger group of non-programming domain experts. What makes the arrangement unusual is that the domain experts (lets call them content creators), use custom tooling, some of which is based around a prolog expert system engine, to develop web based software components/forms.
The Problem
The system uses a very awkward postback model to perform logical operations server side and return new forms/results. It is slow, and prone to failure. Simple things, like creating html forms using the existing tooling is much more arduous than it should be. As the demand for a more interactive, and performant experience grows, the software developers are finding increasingly that they have to circumvent the expert system/visual tooling used by the content creators, and write new components by hand in javascript. The content creators feel increasingly that their hands are tied, as they are now unable to contribute new components.
Design approach: Traditional/Typical
I have been advocating for the complete abandonment of the previous model and the adoption of a typical software development process. As mentioned earlier, the project has naturally evolved towards this as the non-programmatic development tooling has become incapable of meeting the needs of the business.
The content creators have a very valuable contribution to make however, and I would like to see them focusing on formally specifying the expected behaviour of the software with tools like Cucumber, instead of being involved in implementation.
Design approach: Non-programmatic
My co-worker, who I respect a great deal and suspect is far more knowledgeable than me, feels that the existing process is fine and that we just need to build better tooling. I can't help but feel however that there is something fundamentally flawed with this approach. I have yet to find one instance, either historical or contemporary where this model of software development has been successful. COBOL was developed with the philosophy of allow business people/domain experts to write applications without the need for a programmer, and to my mind all this did was create a new kind of programmer - the COBOL programmer. If it was possible to develop effective systems allowing non-programmers to create non-trivial applications, surely the demand for programmers would be much lower? The only frameworks that I am aware of that roughly fit this model are SAP's Smart Forms and Microsoft's Dynamix AX - both of which are very domain specific ERP systems.
DSLs, Templating Languages
Something of a compromise between the two concepts would be to implement some kind of DSL as a templating language. I'm not even sure that this would be successful however, as all of the content creators, with one exception, are completely non technical.
I've also considered building a custom IDE based on Visual Studio or Net Beans with graphical/toolbox style tooling.
Thoughts?
Is non-programmatic development a fools errand? Will this always result in something unsatisfactory, requiring hands on development from a programmer?
Many thanks if you've taken the time to read this, and I'd certainly appreciate any feedback.
You say:
Something of a compromise between the
two concepts would be to implement
some kind of DSL as a templating
language. I'm not even sure that this
would be successful however, as all of
the content creators, with one
exception, are completely non
technical.
Honestly, this sounds like exactly the approach I would use. Even "non-technical" users can become proficient (enough) in a simple DSL or templating language to get useful work done.
For example, I do a lot of work with scientific modeling software. Many modelers, while being much more at home with the science than with any form of engineering, have been forced to learn one or more programming languages in order to express their ideas in a way they can use. Heck, as far as I know, Fortran is still a required course in order to get a Meteorology degree, since all the major weather models currently in use are written in Fortran.
As a result, there is a certain community of "scientific programming" which is mostly filled with domain experts with relatively little formal software engineering training, expertise, or even interest. These people are more at home with languages/platforms like Matlab, R, and even Visual Basic (since they can use it to script applications like Excel and ESRI ArcMap). Recently, I've seen Python gaining ground in this space as well, mainly I think because it's relatively easy to learn.
I guess my point is that I see strong parallels between this field and your example. If your domain experts are capable of thinking rigorously about their problems (and this may not be the case, but your question is open-ended enough that it might be) then they are definitely capable of expressing their ideas in an appropriate domain-specific language.
I would start by discussing with the content creators some ideas about how they would like to express their decisions and choices. My guess would be that they would be happy to write "code" (even if you don't have to call it code) to describe what they want. Give them a "debugger" (a tool to interactively explore the consequences of their "code" changes) and some nice "IDE" support application, and I think you'll have a very workable solution.
Think of spreadsheets.
Spreadsheets are a simple system that allows non-technical users to make use of a computer's calculation abilities. In doing so, they have opened up computers to solve a great number of tasks which normally would have required custom software developed to solve them. So, yes non-programmatic software development is possible.
On the other hand, look at spreadsheets. Despite their calculational abilities you really would not want as a programmer to have to develop software with them. In the end, many of the techniques that make programming languages better for programmers make them worse for the general population. The ability to define a function, for example, makes a programmer's life much easier, but I think would confuse most others.
Additionally, past a certain point of complexity trying to use a spreadsheet would be a real pain. The spreadsheet works well within the realm for which it was designed. Once you stray too far out that, its just not workable. And again, its the very tools programmers use to deal with complexity which will prevent a system being both widely usable and sufficiently powerful.
I think that for any given problem area, you could develop a system that allows the experts to specify a solution. It will be much harder to develop that system then to solve the problem in the first place. However, if you repeatedly have similiar problems which the experts can develop solutions for, then it might be worthwhile.
I think development by non-developers is doomed to failure. It's difficult enough when developers try it. What's the going failure rate? 50% or higher?
My advice would be to either buy the closest commercial product you can find or hire somebody to help you develop a custom solution with your non-developer maintenance characteristics in mind.
Being a developer means keeping a million details in mind at once and caring about details like version control, deployment, testing, etc. Most people who don't care about those things quickly tire of the complexity.
By all means involve the domain experts. But don't saddle them with development and maintenance as well.
You could be putting your organization at risk with a poorly done solution. If it's important, do it right.
I don't believe any extensive non-programmer solution is going to work. Programming is more than language, it's knowing how to do things reasonably. Something designed to be non-programmer friendly will still almost certainly contain all the pitfalls a programmer knows to avoid even if it's expressed in English or a GUI.
I think what's needed in a case like this is to have the content creators worry about making content and an actual programmer translate that into reasonable computer code.
I have worked with two ERP systems that were meant for non-programmers and in both cases you could make just about every mistake in the book with them.
... Simple things, like creating html forms using the existing tooling is much more arduous than it should be...
More arduous for whom? You're taking a development model that works (however badly) for the non-programming content creators, and because something is arduous for someone you propose to replace that with a model where the content creators are left out in the cold entirely? Sounds crazy to me.
If your content creators can learn custom tooling built around a Prolog rules engine, then they have shown they can learn enough formalism to contribute to the project. If you think other aspects of the development need to be changed, I see only two honorable choices:
Implement the existing formalism ("custom tooling") using the new technology that you think will make things better in other ways. The content creators contribute exactly as they do now.
Design and implement a domain-specific language that handles the impedance mismatch between what your content creators know and can do and the way you and other developers think the work should be done.
Your scenario is a classic case where a domain-specific language is appropriate. But language design is not easy, especially when combined with serious usability questions. If you are lucky you will be able to hire someone to help you who is expert in both language design and usability. But if you are nonprofit, you probably don't have the budget. In this case one possibility is to look for help from another nonprofit—a nearby university, if you have one.
I'd advise you to read this article before attempting to scrap the whole system. I look at it this way. What changed to prompt the redevelopment? Your domain experts haven't forgotten how to use the original system, so you already have some competent "COBOL programmers" for your domain. From your description, it sounds like mostly the performance requirements have changed, and possibly a greater need for web forms.
Therefore, the desired solution isn't to change the responsibilities of the domain experts, it's to increase the performance and make it easier to create web forms. You have the advantage of an existing code base showing exactly what your domain experts are capable of. It would be a real shame not to use it.
I realize Prolog isn't exactly the hottest language around, but there are faster and slower implementations. Some implementations are designed mostly for programmer interactivity and are dynamically interpreted. Some implementations create optimized compiled native code. There are also complex logical programming techniques like memoization that can be used to enhance performance, but probably no one learns them in school anymore. A flow where content creators focus on creating new content and developers focus on optimization could be very workable. Also, Prolog is ideally suited for the model layer, but not so much for the view layer. Moving more of your view layer to a different technology could really pay off.
In general though, 2 thoughts:
You cannot reduce life to algorithms. Everything we know (philosophically, scientifically) and experience demonstrates this. (Sorry, Dr. Minsky).
That said, a Domain-Specific tool that allows non-programmers to express a finite language is definitely doable as several people have given examples. Another example of this type of system is Mathematica and especially Simulink which are used very successfuly over a range of applications. However, the failure of Expert Systems, Fuzzy logic, and Japan's Fifth Generation computer project of the 80's to take-off demonstrate the difficulty in doing this.
Labview is a very successful none programatic programming environment.
What an interesting problem.
I would have to ultimately agree with you, and disagree with your colleague.
The philosophy and approach of Domain Driven Development/Design exists exactly for your purposes, in that it puts paramount importance on the specific knowledge of the experienced domain experts, and on communicating that knowledge to talented software developers.
See, in your issue, there are two distinct things. The domain, and the software. The domain should be understood and specified first and foremost without software development in mind.
And then the transformation to software happens between the communication between domain experts and programmers.
I think trying to build "programming" tools for domain experts is a waste of time.
In Domain Driven Development your domain experts will continue to be important, and you'll end up with better software.
In your colleague's approach you're trying to replace programmers with tool.... maybe in the future, like, start trek future, that will be possible, but today I don't think so.
I am currently struggling with a similar problem in trying to enable healthcare providers to write rules for workflows, which isn't easy because they aren't programmers. You're a programmer not because you went to programming school -- you're a programmer because you think like a programmer. Fortunately, most hospitals have some anesthesiologist or biomedical engineer who thinks like a programmer and can manage to program. The key is to give the non-programmers-who-think-like-programmers a language that they can learn and master.
In my case, I want doctors to be able to formulate simple rules, such as: "If a patient's temperature gets too low, send their doctor a text message". Of course it's more complicated than that because the definition of "too low" depends on the age of the patient, a patient may have many doctors, and so on, but a smart doctor will be able to figure out those rules. The real issue is that the temperature sensor will often fall off the patient and read ambient temperature, meaning that the rule is useless unless you can figure out how to determine that the thermometer is actually reading the patient's temperature.
The big problem I have is that, while it's relatively easy to create a DSL so that doctors can express IF [temperature] [less-than] [lookup-table [age]] THEN [send-text-message], it's much harder to create one that can express all the different heuristics you might need to try before coming up with the right way to make sure the reading is valid.
In your case, you may want to consider how VB became popular: It has a form drawing tool that anybody can easily use to draw forms and set properties on form items. Since not everything can be specified by form properties and data binding, there's a code-behind mode that lets you do complex logic. But to make the tool accessible to beginning programmers, the language is BASIC, so users didn't have to learn about pointers, memory allocation, or data structures.
While you probably wouldn't want to give your users VB, you might consider a hybrid approach. You would have one "language" (it could be graphical, like VB's form designer or Labview) where inexperienced users can easily do the simple stuff, and another language to enable expert users to do the complicated stuff.
I had this as a comment previously but I figured it deserved more merit.
There are definitely a number of successful 'non-programmatic' tools around, off hand I can think of Labview, VPL and graph based (edit: I just noticed this link has more far more than just shaders on it) shaders which are prevalent in 3D applications.
Having said that, I don't know of any which are suited to web based dev (which appears to be your case).
I dout very much the investment on developing such tools would be worth it (unless maybe you could move to sell it as a product as well).
I agree with you - non-technical people will not be able to program anything non-trivial.
Some products try to create what's basically a really simple programming language. The problem is that programming is an aptitude as well as a skill. It takes a certain kind of mindset to think in the sort of logic used by computers, which just can't be abstracted away by any programming language (at least not without without making assumptions that it can't safely make).
I've seen this in action with business people trying to construct workflows in MS Dynamics CRM. Even though the product was clearly intended to allow them to do it without a programmer they just couldn't figure out how to make a loop or an if-else condition work, no matter how "friendly" the UI tried to make it. I watched in amazement as they struggled with something that seemed completely self-obvious to me. After a full day (!) of this they managed to produce a couple of very basic workflows that worked in some cases, but didn't handle edge cases like missing values or invalid data. It was basically a complete waste of time.
Granted, Dynamics CRM isn't exactly the epitome of user-friendliness, but I saw enough to convince me that this is, indeed, a fool's errand.
Now, if your users are not programmers, but still technical people they might be able to learn programming, but that's another story - they've really become "new programmers" rather than "non-programmers" then.
This is a pretty philosophical topic and difficult to answer for every case, but in general...
Is non-programmatic development a fools errand?
Outside of a very narrow scope, yes. Major software vendors have invested billions over the years in creating various packages to try to let non-technical users create & define workflows and processes with limited success. Your best bet is to take advantage of what has been done in that space rather than trying to re-invent it.
Edit:
Sharepoint, InfoPath, some SAP stuff are the examples I'm talking about. As I said above, "a very narrow scope". It's possible to let non-programmers create workflows, complex forms, some domain processes, but that's it. Anything more general-purpose is simply trying to make non-programmers into programmers by giving them very crude tools.
Non programatic software development IS feasable, as long as you are realistic about what non-developers can reasonaby achieve - its all about a compromise between capability and ease of use.
The key is to break the requirements down cleanly into things that the domain experts need to be able to do, and spend time implementing those features in a foolproof way - The classic mistake is to try to let the system to do too much.
For example suppose you want domain experts to be able to create a form with a masked text input:
Most developers will look at that requirement and create a fancy control which accepts some sort of regular expression and lets the domain expert do anything.
This is the classic developer way of looking at things, however it's likely that your domain expert does not understand regular expressions and the developer has missed the point of the reqirement which was for the domain expert to be able to create this form.
A better solution might have been to create a control that can be confgiured to mask either Email addresses or telephone numbers.
Yes this control is far less capable than the first control, and yes the domain experts have to ask developers to extend it when they want to be able to mask to car registration numbers, however the domain experts are able to use it.
It seems that the problem is of organizational nature and cannot be solved by technical means alone.
The root is that content creators are completely non-technical, yet have to perform inherently technical tasks of designing forms and writing Prolog rules. Various designers and DSLs can alleviate their problems, but never solve them.
Either reorganize system and processes so knowledge carriers actually enter knowledge - nothing else; or train content creators to perform necessary development with existing tools or may be DSLs.
Non-programmatic development can save from low-level chores, but striving to set up system once and let users indefinitely and unrestrictedly expand it is certainly a fools errands.
Computer games companies operate like this all the time, so far as I can tell: a few programmers and a lot of content creators who need to be able to control logic as well (like level designers).
It's also probably a healthy discipline to be able to separate your code from the data and rules driving it if you can.
I'm therefore with your colleague, but of course the specifics may not make this general solution appropriate!

How to be prepared for industry? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 7 years ago.
Improve this question
The amount of available programming languages is both a bless and a curse, I think.
I know a lot of programming languages already, some at syntax-level only and some good enough to do actual coding (Python, C, C++, Haskell, Perl, BASH, PHP, and lots of others). I have been programming for almost as long as I've been intensivly using computers (6 years), in almost every paradigm (functional, imperative, object oriented), but I don't feel prepared for the software industry.
I've been writing a lot of bigger programs in a lot of different languages, mostly network based, including large multithreaded server/clients, and I still don't feel prepared!
Currently I'm obsessed with my "3-tier" plan, which includes a high level language like Haskell, an interpreted language like Python and a low level language like C, yet I don't feel good enough!
I know how to work in teams, and how to work along given guidelines, but I'm unsure.
Am I prepared?
Please, kind people of stackoverflow, help me out of this mess! :(
Thanks for all the answers, I wish I could chose more answers as THE answer :)
Sounds like you know an awful lot about programming, but you don't mention anything else. Being a software developer requires more than just programming as a technical skill. Brush up on topics such as source code control, unit testing/test-driven development, continuous integration, etc. Hopefully you'll land in a job where at least one of those is in use. Try and learn as many useful time-savers as you can with your tools; try to become as flexible and efficient with your IDE as possible.
Elsewhere, don't forget to develop the more personal skills; attitude and work ethic, and more related to your field, issues such as eliciting requirements, documenting issues and describing problems and solutions. Don't worry too much about these if you're going in afresh, because you're not expected to have a huge knowledge of them, but if you're at least aware of them and trying to improve, then you have a greater chance of doing so.
Try to appraise yourself of general software development issues that aren't directly coding, if you haven't already - general attitudes to security-oriented development (and testing), good design and similar best practices.
Don't sweat too much about being perfect right off the bat. If you've got no room for improvement, you aren't going to enjoy your career very long, and burning out as a programmer ain't much fun.
You know enough - there is a minimum threshold of knowledge required in the industry (which is above what some developers have), but it sounds like you are already there.
For anyone with the aptitude, new programming languages, techniques, etc, are easy to learn. A good company to work for will hire you based on your abilities, not knowledge (which can go stale very quickly).
If you want to stand out as a software developer, ensure you have rock solid communication skills for reports, e-mail, telephone, meetings, etc. That is a rarer gift in the software field, and although it is not necessary more valuable at the junior levels, it pays off in the long run.
The single most important thing I can think of to be successful in the industry is to be able to respond quickly and efficiently to change.
I recently took a programming test which I thought was a good and fair test. I passed it without a great deal of effort. I was told that 50% of the people (these are all people with programmer on the resume) don't even know where to start. Your earnestness and desire will most likely put you in the top third of most places to start with.
Knowning languages is not all you can do.
If you can, a placement/internship will do wonders. Anyone can program. Real world experience will teach you more than any tutorials, self learning or schooling will.
Naturally, gaining an internship requires some experience, so it's very much catch twenty two.
If going for an internship is not possible, get involved with an open source project. You'll find you'll learn loads by working with people smarter than you.
True knowledge exists in knowing that you know nothing.
Socrates some smart dude
I think this is pretty common among developers. Imo it´s a way better sign then if you would come to the conclusion that you were fully trained.
The only way to know for sure if you're prepared is to try. Sometimes being thrown in the deep end actually helps and you'll find you learn more in that first real world job than you did in all the books/etc that you read in the years before. Also, knowing multiple languages helps you understand underlying semantics of programming in general, but in a real job you'll likely be sticking to one or two languages day to day, so don't get hung up on knowing every language out there.
It's better to try & fail than to spend your life wondering if you're ready.
Go to dice or monster or whatever your favorite job site is and see what people are looking for. It's not Haskell, it's C++. Learn that well and you're ready to go. Once you're out in the real world, you'll learn quickly enough the things that are important. These are mostly the soft skills that school doesn't teach you. Things like how to get along with the clueless, how to present your ideas so they'll actually be considered, and how to see the forest even though you're stuck under a rock.

How to protect your software code? [duplicate]

This question already has answers here:
Closed 12 years ago.
Possible Duplicates:
How do you protect your software from illegal distribution?
Best practice to prevent software copy
Hypothetical situation:
Lets say I have built a software product from the scratch and it does wonderful things. The only problem is that, once someone takes a look at the code, they will understand it very easily and they can easily build it up themselves.
Now, the thing is that I built the code from the scratch 100% and uses a mixture of API calls.
Nobody else is involved in the development of the code.
If I want to sell this product, what is the guarantee that someone much smarter than me will reverse engineer the whole thing and come up with better product?
Right now I am thinking of fragmenting the whole code. Adding lots of redundant code and tonnes of comments.
Is there any software which encrypts the software code, that will make debugging, troubleshooting, and understanding how the code works virtually impossible? and yet runs as usual? so that the developer can have peace of mind?
Very few things in a program are truly novel. Almost everything that you are likely to put into your code, someone else could invent on their own. Generally more easily than they could learn it by reading your code. Reading code is harder than writing it, and most programmers don't really like doing it anyway.
So it's much more likely that they will look at your app and think "I could do that", then "That's cool, I'm gonna read that code and then copy it!". Even if they understand it, you will still own the copyright, you still get to market first.
I recommend that you just forget about it.
once someone takes a look at the
code, they will understand it very
easily and they can easily build it up
themselves.
So don't give anybody the source code.
If I want to sell this product, what
is the guarantee that someone much
smarter than me will reverse engineer
the whole thing and come up with
better product?
(a) So start selling it now and capture the market. Reverse engineering takes time, during which you are capturing market and 'mind-share'. (b) Put a provision in your licence agreement that prohibits reverse-engineering. (c) Make sure everybody who gets the product signs the agreement.
Right now I am thinking of fragmenting
the whole code. Adding lots of
redundant code and tonnes of comments.
That only has a point if you're going to distribute the source code. In which case nobody even needs to reverse-engineer. They have your source code. Don't give it to them.
Is there any software ...
There's lots of software that purports to do this job. However it is a technical solution to a business problem. All software can be reverse-engineered, because at some point or other it all has to be decrypted and de-obfuscated to the point where the CPU will understand it. At that point it is essentially plaintext. So no technical solution is formally speaking possible (short of something like code that executes in a tamper-proof HSM).
I will add that there is another business mechanism you can use to defend against business loss, which is what this is all about: price. Make the price so high that the licensees will value their copy and not permit it to be inspected, or make it so low that reverse-engineering is cost-infeasible; or make it free and make your money on the support contract.
Once you actually have the knowledge and experience to write such a codebase, it will be clear to you that obfuscation is meant to deter casual IP infringement.
Someone who wants to know your code is going to know your code.
If it becomes an issue of monetary loss, the courts are your protection.
That's how it works.
Someone will always be able to understand and work out your code. Heck, if you had 0 way getting to the code, even just using the system is enough for someone to be able to replicate the process.
Example: I take a jug of water and pour it into the cup, while my back is facing to another person. This other person knows that water and gravity are awesome at making things fall into other containers, so they can then work out a process of lifting a jug to let gravity (API call) work in their favour. They mightn't know exact what angle you used in your forearm and any super-sneaky cup-holding techniques you used, but they can replicate the same process and improve on it over time.
tl;dr: You can't protect code.
The thing to do is invent even more wonderful things while the competition is reverse-engineering your current stuff. It's called competing through innovation.
I am not a lawyer
if you are really worried about it, to the point you are willing to invest money in it, dont protect your code (beyond something reasonable like obfuscation or encryption) but rather patent your idea and your art. Then if someone does take it, reverse engineer it and make a better process based of yours, you have legal grounds to get your money.
There are tons of things you will have to do, include proving they took your idea (which isnt easy), but if this is the solution to world hunger and all of humanities problems its the thing to do.
Now for the downside, I will guess, and probably be 90% right that your method is:
Not patentable, for various reasons (I was amazed at the number of already patented ideas, and how difficult it was to identify original art)
Not new, or unique (i.e. there is already established art for it)
Not worth patenting because the expense far outways the benefits
An IP lawyer can tell you for sure, and the expense of a consult is not that much. Overall it will be cheaper to consult with them then to invest a lot of time in hiding code.
Good luck.
Don't even bother. If your code really "does wonderful things" be assured that it'll get hacked. And be it just for curiosity.
There is no 100% way to protect your code from reverse engineering. What language are we talking about? If this is C/C++ then it is pretty hard to reverse engineer, more you could strip it from debugging information etc. But if this is for example Java then even if you obfuscate the code, there are some pretty cool tools (like JAD) that will reveal much of your work anyway.
Despite all of this I think you should try to change your attitude. Big companies pay a lot of money for simple solutions and it seems that nowadays service is the most important thing, not the software (hence the success of open-software based companies). So, if you have a great software don't be scared that someone might steal it, rather think how to sell it good.
Is there any software which encrypts the software code, that will make debugging, troubleshooting, and understanding how the code works virtually impossible? and yet runs as usual? so that the developer can have peace of mind?
This is the totally wrong mindset IMO. What happens if you get hit by a bus? Your company goes bankrupt? All your data gets destroyed in a fire? For every single one of your customers, the value of their investment in your software will drop, and eventually reach zero, because the software can't be developed, or troubleshot, any further without you. I have seen so much money wasted that way, I think it's a horrible business model.
I earn my bread with making software myself so I know the hardships of making a living with it. Still, obfuscation can't be the way to go nowadays. Impose strict license agreements on your customers, scare the hell out of them so they don't even think about redistributing the software, but leave it open.
This is futile. There is always someone smarter than you and therefore they will be able to reverse engineer your obfuscation.
Usually someone smart enough to hack your code and use it in a meaningful way is smart enough to do it on their own, and probably thinks they can do it better than you did, so they won't bother stealing your stuff.
Don't worry about the people who can hack your code but not make meaningful use of it. If you've done a good job, this can only reinforce the quality of the job you've done (think of all the crappy touchscreen phone imitators).
They are going to reverse-engineer your code. Nothing can stop them.. The only thing you can do is make it harder. This ranges from obfuscating code that is inheritely "open" such as PHP and Javascript, all the way down to littering your code with a crap load of self-modification.
In a lot of ways, I think, the thing that makes a piece of software valuable, is not the crazy technological advancement that it provides, but rather the things that we think might think of as being tertiary to the piece of software itself. Like the fact that you'll be there to support it. Or that it's provided as a web service and you'll be there to make sure the server is running. Or that it's a community, and you'll be there to moderate and build the community.
While you may be actually selling code, the value you that your code has isn't intrinsic to the code itself, but rather derives from the features and ecosystem that surrounds your code.

Roadmap to a better programmer [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Its always said that more you program, the better you become. Sounds good and true.
But I was wondering if there is a proven route to becoming a better programmer.
Something like:
Learn a
Learn b
Learn c > 'Now you are good to burn the engines'
Try stuff around based on your learning.
The answer might be similar to a CS course roadmap, but I want to hear from successful programmers who might want to pitch in with something notable.
Thanks
It's not true that practice makes perfect.
It's perfect practice that makes perfect.
If all you do is keep repeating the same bad practices again and again, you'll only make it possible to create bad code faster.
By all means keep coding. But at the same time be critical of everything you do. Always have a jaundiced eye that looks for ways to do things better. Read widely to get new ideas. Talk to others about how they do things. Look at other people's code, good and bad.
There's no "sure" way to learn anything that I know of. If there was, anyone could master this.
All questions are rhetorical and meant to stimulate thought.
Technical parts:
Design Patterns - There are probably some specific to a domain but generally these are useful ways of starting parts of an application. Do you know MVC or MVP?
Basic algorithm starting points - Divide and conquer, dynamic programming, recursion, creating special data types like a heap, being greedy, etc.
Problem solving skills - How easily can you jump in and find where a bug is? Can you think of multiple solutions to the problem?
Abstract modelling - How well can you picture things in your head in terms of code or classes when someone is describing a problem?
High level versus low level - How well do you understand when one wants something high or low? This is just something I'd toss out there as these terms get through around a lot, like a high level view of something or a low level language.
Process parts:
Agile - Do you know Scrum, XP, and other new approaches to managing software projects? How about principles like YAGNI, DRY and KISS? Or principles like SOLID? Ideas like Broken Windows?
Developer Environment - How well do you know the IDE you use? Source Control? Continuous Integration? Do you know the bottle necks on your machine in terms of being productive?
xDD - Do you know of TDD, BDD, and other developments driven from a paradigm?
Refactoring - Do you go back over your old code and make it better or do you tend to write once and then abandon your code?
Soft skills:
Emotional Intelligence - Can be useful for presentations and working with others mostly.
Passions/Motivation - Do you know what gets your juices flowing and just kick butt in terms of being productive? Do you know what you would like to do for many many years?
My main piece of advice would be: don't be afraid to rewrite your own code. Look at stuff you wrote even a month ago and you will see flaws and want to rewrite stuff.
Make sure that you understand some fundamentals: collections, equality, hashcodes etc. These are useful across pretty much all modern languages.
Depending on the language you use - use lint and metric tools and run them over your code. Not all their suggestions will be applicable but learning which are important and which are not is important. E.g FindBugs, PMD etc for Java.
Above all refine and keep refining your work. Don't treat your work as abandonware!
Learn your 1st programming language a new programming paradigm or a
find a mentor you can learn from
Apply what you've learnt in a real world project
Learn from your mistakes and successes and goto step one
The trick is knowing what to learn first:
Programming languages - this is the place to start bcause you cannot write software without knowing at least one of these. After you've mastered one language try learning another.
Programming paradigm - i.e. object oriented, dynamic/functional programming etc. Try to learn a new one with each new language.
Design concepts - S.O.L.I.D, design patterns as well as architectural concepts.
People skills - learn to communicate your ideas.
Team leadership - learn how to sweep others and how to become a team or technological lead.
After that the sky is the limit.
I would look at improving roughly in this order, in iterations with each building on the previous one:
Programming concepts. Understand things like memory management, pointers, stacks, variable scope, etc.
Languages. Work on mastering several modern languages.
Design concepts. Learn about design patterns. Practice using them.
Communication. Often-overlooked. You can only become a highly valued Software Engineer if you can communicate effectively with non-tech people. Learn to listen and understand the needs that people are expressing, translate that into a set of requirements and a technical design, but then explain what you understood (and designed) back to them, in terms they can understand, for validation before you code. This is not an easy one to master, but it is essential.
Architectural concepts. Learn to understand the big picture of large, complex systems.
Learning a programming language is in many ways similar to learning a spoken language. The only way to get good at it is to do it as often as possible. In other works
Practice, practice, read and then practice more
Take time to learn about all sorts of coding techniques, tools and programming wisdom. This I have found to be crucial to my development. It's to easy to just code away and feel productive. What about what could be if you just had some more knowledge / weaponry under your belt to bang out that next widget.
Knowledge/know how is our real currency. The more we know the more we can make a better decision about how something should be done and do it faster.
For example, learn about:
•Development Practices, Software Design, Estimation, Methodologies Business Analysis Database Design (there are a lot of great books out there and online resources)
•Read Code - Open Source Projects are a good place for this. Read
Programming blogs
•Try to participate on Open Source
Projects.
•Look for programming user groups in
your town and/or someone who can mentor you.
And yes, as mentioned practice. Don't just read, do and watch how you will improve. :)
Practice, practice, practice.
Once you're over the basic hump of being able to program, you can also read useful books (i.e. Code Complete, Effective Java or equivalents, etc.) for ideas on how to improve your code.
First and foremost write code. Write as much as you can. Tackle hard problems. If you want to be a really good programmer you need to get into the guts of what you are doing. Spend a lot of time in debuggers looking at how things work. If you want to be a good programmer who really understands what is going on you need to get down to the metal and write highly async code, learn about how processors work and why SSE is so awesome. Understand threading primitives and be able to write them as well as describe what is actually happening in the processor. I could keep going here but you get the idea.
Second find someone who knows a lot more than you and learn. This relationship will work better if you are already deeply immersed in writing lots of code.
Third, spend some time in a large high quality open source code base. I learned a ton from the Quake I and Quake II code. Helped me be a better programmer.
Fourth take on hard problems. Push your limits. Build things that you thought were impossible. Right now I am writing a specialized compiler. I have learned so much just working on this for the last couple of months.
Sure, strictly speaking, the more you practice programming, the better you become at solving those sorts of problems. But is that what you really want?
Programming is a human activity more than a technological one, at its heart. It's easy to improve your computer skills, not so hard to improve your interpersonal skills.
Read "Journey of the Software Professional" by Hohmann. One of the concepts the concepts Hohmann describes is the "cognitive library," which includes both programming skills and non-programming skills. Expand your cognitive library, and your programming skill will improve too.
Read a lot of non-programming books too, and observe the world around you. Creating useful metaphors is an essential skill for the successful programmer. Why do restaurants do things how they do? What trade-offs is the garbage department making when they pick up the garbage every few days instead of every day? How does scaling affect how a grocery store does business? Be an inquisitive human to be a better programmer.
For me, there has to be a reason to learn something new... that is, unless I have a project in mind or some problem I need to solve, there's no hope. If that prerequisite is met, then I usually try to get "Hello, world" working, and after that the sky's the limit. So much of development these days is just learning new APIs. Occasionally there's some kind of paradigm shift that blows your mind, but that's not as common as people like to think, IMHO.
Find a program that intrigues you, one that solves a problem, or one that would simplify many of your tasks. Try to write something similar. You'll get up to speed very quickly and have fun doing it at the same time.
You can try learning one thing really well and then expanding out to programming areas that are associated with the things that you have learnt, so that you can offer complete solutions to customers.
At the same time, devote part of your time to explore things outside your comfort zone.
One you have learned something, try to learn something a little harder. Read and practice a lot about things that seem confusing at first time (lambda functins, threading, array manipulation, etc). It will take its time, but once you have practiced enough, what seemed confusing at first, will be familiar and easy.
In addition to the rest of the great advice already given here, don't be afraid to read about coding and good practice, but also take everything with a grain of salt and see what works best for you. A lot of advice is opinion.
Good sites to read:
-thedailywtf.com
-joelonsoftware.com
-codinghorror.com
-blogs.msdn.com/oldnewthing
A great place to get practice is programming competition websites. Those will help you learn how to write good algorithms, not necessarily maintainable code, but they're still a good place to start for learning.
The one I used to use (back when I had time) was:
http://uva.onlinejudge.org/
Learn more than one language. One at a time, definitely, but ultimately you should be fluent in a couple. This will give you a better perspective I think, and help you to become an expert at programming, rather than being an expert at a certain language.
Learn the ins and outs of computers at all levels, hardware, os, etc. Ideally you should be able to build your own system, install multiple operating systems on it, and diagnose just about every problem that can arise. I know many programmers who are not "computer tech people" and their failure to understand what is happening at every level becomes a major hindrance in diagnosing and fixing unusual bugs or performance issues.
As well as looking at 'last weeks code', talk to users of your work after delivery - be one yourself if possible.
Its not my bag, but some of the best coders I know have spent time supporting applications. The experience improved their product I'm sure.
eat breath dream the programming language your using (no seriously, it helps)
There are two kinds of learning -
1. Informal (like how you learned how to function in society- through interaction with peers and family)
2. Formal (like your high school training- through planned instruction)
If you want an entry-level programming job, formal training via an undergrad Computer Science/Engineering degree is the way to go. However, if you want to become a rock-star developer, it is best done by informal training- make unintentional mistakes and have senior developers curse at you, learn a design pattern because an app you are updating uses it, almost cry because a bad developer wrote a huge messy program lacking documentation and best practices and now you have to do several updates to it ASAP; thing of these nature.
It is hard for anyone to give you a list of all you need to know. It varies per area (e.g. a web developer vs. to a desktop developer) and it varies per company (e.g. Microsoft that sells software vs. General Motors that mainly just use it in their cars.) Informal traiing and being engaged in trying to learn to do your job better and get promoted is your best bet in my opinion.
To prove my point, everyone here has great answers but they all differ. Ask a rock-star developer how he learned something or when, why; they may not know- things just happen.
Practice, individually and collectively
Keep an open mind, always learn new things, don't limit yourself to what's familiar. Not solely from a tech perspective, ui design, people skills, ... Don't be afraid of what's new
Peer review, talk to people about your code, let people talk to you about their code, everyone has a unique way of looking at a problem and you will learn a great deal from peers
Love coding. If you love what you're doing, putting in alot of time seems effortless. Every coder needs the drive!
One small addition to these good answers. When I work on someone else's code, usually I pick up something new. If you have the opportunity to work with someone else that is of equal or greater skill, noticing their programming style can teach you tons.
For example, in C++ & Javascript I no longer use if() statements without braces. The reason is that it's just too easy to mistakenly put:
while (true) {
if (a > b)
print a
print b
}
This is an obvious typo, but very easy to introduce, especially if you're editing existing code. I just call it defensive programming in my mind, but little tricks like this are valuable at making you better.
So, find a peer or mentor, and work on their code.
I am not sure if the OP was looking for general advice on how to be a good programmer, but rather something more specific.
I know I am reviving this thread, but I found it because I was trying to see if anyone asked this question already.
What I had in mind was, can we come up with a "knowledge-map" of programming concepts similar to the map that Khan Academy uses.
As a programmer, I want to be able to visualize the dependencies and relationships between different ideas, so that I can understand what skill level I am currently at; what I need to know before tackling a challenging subject; and be able to visualize my progress.
The very belief in the roadmap's existence blocks the road to perfection.

Resources