I am looking for resources that can provide me with a better understanding of what kind of things shaders are used for in games, what they can do, and maybe even more importantly, what they cannot. I understand how the graphics pipeline works and all that, and I have made some very basic shaders in GLSL (mostly just to replace the fixed-function pipeline functionality), but I don't yet fully understand which things are only possible with custom shaders, which things can be done more efficiently, etc. I have been able to find some examples of certain techniques, most notably lighting, but I am looking for a more higher-level overview of their usage.
Links to and explanations of certain interesting techniques, as opposed to an overview, are also appreciated (but less than an overview ;) ), preferrably in GLSL or pseudocode.
Well considering that DirectX and OpenGL have both moved towards a shader-only (i.e. no fixed-function) system, the answer to your "which effects are only possible with shaders" question could be "everything".
Some techniques that I believe were not possible/feasible without programmable shaders (or by using very specific black-box APIs) though, are:
Per-pixel lighting
Shadow mapping.
GPU skinning (i.e. matrix palette skinning) for animated meshes.
Any of the various post-process effects that are common today: bloom, SSAO, depth of field, etc.
Deferred shading.
Implementing arbitrary/"other" lighting models like Oren–Nayar, Cook–Torrance, Rim-Lighting, etc.
and the list can go on, and I'm sure some people will disagree with my assessment that these couldn't be achieved with fixed-function functionality (through hacks or various fixed-functions extensions).
What it boils down to is, before programmable shaders, a given effect had to be implemented in the hardware/driver by the vendor and it and it had to be something that could be reasonably expressed through the API. Now you can execute effectively any user-defined code you want (within the constraints of the different shader stages and other limitations of the hardware) so you have the flexibility to greatly customize your rendering pipeline and invent new techniques as you see fit.
Take a look at the first couple GPU Gems books (which can be read for free on Nvidia's website) to get a feel for the types of techniques that were showing up once programmable hardware was available.
Related
Should I learn scala and AKKA for a simulation project. Are these technologies a good fit / worth the investment? The task is to perform https://www.dropbox.com/s/3lby24y26wp60to/assignment.pdf?dl=0 an event-based simulation to simulate an IOT edge data center and implement some scheduling algorithms.
If yes, which libraries would you suggest? https://github.com/scalation/scalation does not seem to be a parallel library.
This is an opinion-based question. Should not be asked here.
Anyways, I'm going to try to give you some pointers. Akka is a generic framework: you can build anything from it, but nothing in particular is an immediate fit (ok, some things fit better than others, but still).
In your case, while Akka is a valid fit (actors = agents), I'd look more into specialized softwares for ABM (Agent based modeling), you can find a massive list here .
In particular, I recommend Netlogo: it's a bit counterintuitive in terms of syntax if you have never used something akin to Lisp or other immutable variables languages ("let" etc.), but once you get the hang of it it's very powerful for the effort required.
And, if you come from a CS background, it should be super easy for you (it's normally used by non-CS people in various fields, and it's designed to be easy).
I'm working on developing controllers for hybrid systems in Haskell.
FRP libraries (right now I'm using netwire, but there are several good ones and a lot of interesting research on future ones) provide a great solution for the continuous-time side of the problem. Augmenting them with signal names, dimensions, preferred units, and so forth gets you a system that has modularity, is self-describing, and has a straightforward path to confidence in correctness.
I'm looking for information, folklore, or papers that provide similar properties for the discrete-time side. In some sense the problem is much easier, state machines are well-studied and simple. In other senses it's more difficult, I'll briefly explain how.
Correctness is obviously the most important thing, and thankfully it's also straightforward.
Self-description is more of a problem. You'd like the controller not just to be in the correct state, but to be capable of telling you what state it's in. Also how it got there. And where it might go next. So you can tack names on to everything, and it works, but it conflicts somewhat with modularity. You'd also like to be able to build complex discrete time behaviors from simpler ones. But when you ask the system what state it's in, generally the high-level answer is more interesting (or at least, as interesting) as the low level answer. How do you get this cleanly? I've tried a few naive approaches and have wrapped myself in spaghetti a few different ways, but it seems like there must be elegant solutions?
Another problem I've had with self-description is that I'd like to have a list of self-describing conditions (generally comparisons: has it been 10 seconds? am I within 3 feet of the next waypoint? has the battery power fallen below 15%? etc) that are being monitored which might trigger the next state transition. There are tricky questions of what even are the desirable semantics here, since it seems like some of these events are better handled "from the bottom up" (e.g. expected termination conditions of whatever low level step you are performing) and some "from the top down" (e.g. equipment failure detection, geofencing, ...). This can lead to spaghetti of its own even if you relax the goal of self-description.
In addition to diagnostics, accurate self-description information here could also be very useful for abstract interpretation, projecting the state of the system into the future by guessing which events are likely to occur when. Many of the event conditions lead themselves to fairly simple guesses (e.g. using velocity made good, fuel consumption rate, timers). Others are more complicated but might still be worth the effort to develop projections for some applications (e.g. expected orders from operators, weather forecasts, projected tracks for moving objects of interest). It would be nice to find a design that annotates conditions not only with names, but also with functions for this sort of stuff.
Does anyone have experience with this that they are willing to share?
Okay, so I would say the "real" answer to your question is that some of things that you are asking for are open areas of research --- in particular I think some of the self-describing features you desire may necessitate some degree of "spaghetti" simply because the problem you are trying to solve is inherently complicated.
That being said, your focus on modularity is exactly the right approach. I would say, take a look at Keymaera as I believe it has the features you are looking for despite being in Java. I would also recommend looking at the publications page on the Keymaera website as this should provide you valuable insight to the problem in general.
If you do not like Keymaera's approach you can also look into using Timed Automata which is another direction modeling-wise that should be sufficient for your problem description.
I'm trying to visualize some simple automatic physical systems (such things as pendulum, robot arms,etc.) in Haskell.
Often those systems can be described by equations like
df/dt = c*f(t) + u(t)
where u(t) represents some kind of 'intelligent control'. Those systems look to fit very nicely in the Functional Reactive Programming paradigm.
So I grabbed the book "The Haskell School of Expression" by Paul Hudak,
and found that the domain specific language "FAL" (for Functional Animation Language) presented there actually works quite pleasently for my simple toy systems (although some functions, notably integrate, seemed to be a bit too lazy for an efficient use, but easily fixable).
My question is, what's the more mature, up-to-date, well-maintained, performance-tuned alternative for more advanced, or even practical applications today?
This wiki page lists several options for Haskell, but I'm not clear about the following respects:
The status of "reactive", the project from Conal Eliott who is (as I understand it) one of the inventers of this programming paradigm, looks a bit stale. I love his code, but maybe I should try other more up-to-date alternatives? What's the primary difference between them, in terms of syntax/performance/runtime-stability?
To quote from a survey in 2011, Section 6, "... FRP implementations are still not efficient enough or predictable enough in performance to be used effectively in domains which require latency guarantees ...". Alghough the survey suggests some interesting possible optimizations, given the fact that FRP is there for more than 15 years, I get the impression that this performance problem might be something very or even inherently difficult to solve at least within a few years. Is this true?
The same author of the survey talks about "time leaks" in his blog. Is the problem unique to FRP, or something we are generally having when programming in a pure, non-strict language? Have you ever found it just too difficult to stabilize an FRP-based system over time, if not performant enough?
Is this still a research level project? Are the people like plant engineers, robotics engineers, financial engineers, etc. actually using them (in whaterver language that suits their needs)?
Although I personally prefer a Haskell implementation, I'm open to other suggestions. For example, it would be particularly fun to have an Erlang implementation --- it would then be very easy to have an intelligent, adaptive, self-learning server process!
Right now there are mainly two practical Haskell libraries out there for functional reactive programming. Both are maintained by single persons, but are receiving code contributions from other Haskell programmers as well:
Netwire focusses on efficiency, flexibility and predictability. It has its own event paradigm and can be used in areas where traditional FRP does not work, including network services and complex simulations. Style: applicative and/or arrowized. Initial author and maintainer: Ertugrul Söylemez (this is me).
reactive-banana builds on the traditional FRP paradigm. While it is practical to use it also serves as ground for classic FRP research. Its main focus is on user interfaces and there is a ready-made interface to wx. Style: applicative. Initial author and maintainer: Heinrich Apfelmus.
You should try both of them, but depending on your application you will likely find one or the other to be a better fit.
For games, networking, robot control and simulations you will find Netwire to be useful. It comes with ready-made wires for those applications, including various useful differentials, integrals and lots of functionality for transparent event handling. For a tutorial visit the documentation of the Control.Wire module on the page I linked.
For graphical user interfaces currently your best choice is reactive-banana. It already has a wx interface (as a separate library reactive-banana-wx) and Heinrich blogs a lot about FRP in this context including code samples.
To answer your other questions: FRP isn't suitable in scenarios where you need real-time predictability. This is largely due to Haskell, but unfortunately FRP is difficult to realize in lower level languages. As soon as Haskell itself becomes real-time-ready, FRP will get there, too. Conceptually Netwire is ready for real-time applications.
Time leaks aren't really a problem anymore, because they are largely related to the monadic framework. Practical FRP implementations simply don't offer a monadic interface. Yampa has started this and Netwire and reactive-banana both build on that.
I know of no commercial or otherwise large scale projects using FRP right now. The libraries are ready, but I think the people aren't – yet.
Although there are some good answers already, I'm going to attempt to answer your specific questions.
reactive is not usable for serious projects, due to time leak problems. (see #3). The current library with the most similar design is reactive-banana, which was developed with reactive as an inspiration, and in discussion with Conal Elliott.
Although Haskell itself is inappropriate for hard real-time applications, it is possible to use Haskell for soft realtime applications in some cases. I'm not familiar with current research, but I don't believe this is an insurmountable problem. I suspect that either systems like Yampa, or code generation systems like Atom, are possibly the best approach to solving this.
A "time leak" is a problem specific to switchable FRP. The leak occurs when a system is unable to free old objects because it may need them if a switch were to occur at some point in the future. In addition to a memory leak (which can be quite severe), another consequence is that, when the switch occurs, the system must pause while the chain of old objects is traversed to generate current state.
Non-switchable frp libraries such as Yampa and older versions of reactive-banana don't suffer from time leaks. Switchable frp libraries generally employ one of two schemes: either they have a special "creation monad" in which FRP values are created, or they use an "aging" type parameter to limit the contexts in which switches can occur. elerea (and possibly netwire?) use the former, whereas recent reactive-banana and grapefruit use the latter.
By "switchable frp", I mean one which implements Conal's function switcher :: Behavior a -> Event (Behavior a) -> Behavior a, or identical semantics. This means that the shape of the network can dynamically switch as it's run.
This doesn't really contradict #ertes's statement about monadic interfaces: it turns out that providing a Monad instance for an Event makes time leaks possible, and with either of the above approaches it's no longer possible to define the equivalent Monad instances.
Finally, although there's still a lot of work remaining to be done with FRP, I think some of the newer platforms (reactive-banana, elerea, netwire) are stable and mature enough that you can build reliable code from them. But you may need to spend a lot of time learning the ins and outs in order to understand how to get good performance.
I'm going to list a couple of items in the Mono and .Net space and one from the Haskell space that I found not too long ago. I'll start with Haskell.
Elm - link
Its description as per its site:
Elm aims to make front-end web development more pleasant. It
introduces a new approach to GUI programming that corrects the
systemic problems of HTML, CSS, and JavaScript. Elm allows you to
quickly and easily work with visual layout, use the canvas, manage
complicated user input, and escape from callback hell.
It has its own variant of FRP. From playing with its examples it seems pretty mature.
Reactive Extensions - link
Description from its front page:
The Reactive Extensions (Rx) is a library for composing asynchronous
and event-based programs using observable sequences and LINQ-style
query operators. Using Rx, developers represent asynchronous data
streams with Observables, query asynchronous data streams using LINQ
operators, and parameterize the concurrency in the asynchronous data
streams using Schedulers. Simply put, Rx = Observables + LINQ +
Schedulers.
Reactive Extensions comes from MSFT and implements many excellent operators that simplify handling events. It was open sourced just a couple of days ago. It's very mature and used in production; in my opinion it would have been a nicer API for the Windows 8 APIs than the TPL-library provides; because observables can be both hot and cold and retried/merged etc, while tasks always represent hot or done computations that are either running, faulted or completed.
I've written server-side code using Rx for asynchronocity, but I must admit that writing functionally in C# can be a bit annoying. F# has a couple of wrappers, but it's been hard to track the API development, because the group is relatively closed and isn't promoted by MSFT like other projects are.
Its open sourcing came with the open sourcing of its IL-to-JS compiler, so it could probably work well with JavaScript or Elm.
You could probably bind F#/C#/JS/Haskell together very nicely using a message broker, like RabbitMQ and SocksJS.
Bling UI Toolkit - link
Description from its front page:
Bling is a C#-based library for easily programming images, animations,
interactions, and visualizations on Microsoft's WPF/.NET. Bling is
oriented towards design technologists, i.e., designers who sometimes
program, to aid in the rapid prototyping of rich UI design ideas.
Students, artists, researchers, and hobbyists will also find Bling
useful as a tool for quickly expressing ideas or visualizations.
Bling's APIs and constructs are optimized for the fast programming of
throw away code as opposed to the careful programming of production
code.
Complimentary LtU-article.
I've tested this, but not worked with it for a client project. It looks awesome, has nice C# operator overloading that form the bindings between values. It uses dependency properties in WPF/SL/(WinRT) as event sources. Its 3D animations work well on reasonable hardware. I would use this if I end up on a project in need for visualizations; probably porting it to Windows 8.
ReactiveUI - link
Paul Betts, previously at MSFT, now at Github, wrote that framework. I've worked with it pretty extensively and like the model. It's more decoupled than Blink (by its nature from using Rx and its abstractions) - making it easier to unit test code using it. The github git client for Windows is written in this.
Comments
The reactive model is performant enough for most performance-demanding applications. If you are thinking of hard real-time, I'd wager that most GC-languages have problems. Rx, ReactiveUI create some amount of small object that need to be GCed, because that's how subscriptions are created/disposed and intermediate values are progressed in the reactive "monad" of callbacks. In general on .Net I prefer reactive programming over task-based programming because callbacks are static (known at compile time, no allocation) while tasks are dynamically allocated (not known, all calls need an instance, garbage created) - and lambdas compile into compiler-generated classes.
Obviously C# and F# are strictly evaluated, so time-leak isn't a problem here. Same for JS. It can be a problem with replayable or cached observables though.
What features could be added to a new programming language
to make it more "intuitive"? When it comes to websites and
desktops, we favor high usability, almost intuitive
usability. It is becoming increasingly expected that your
application should "just work". For a certain class of
applications the idea that one has to RTFM, is a mark
against the effectiveness of the application. People tend to
expect the application to just work the way they "think" it
should work. One could argue that this is a worthy standard
that designers should strive for.
Can the same usability rigor apply to programming languages
and developer environments? I realize there are tools like
IntelliSense that provide hints, and a good IDE provides a
lot of assist. But what about the core language itself? What
could be added (or removed) that makes certain programming
techniques or algorithms more obvious to implement? How does
one make regular expressions or recursion more intiutive? Or
is this just folly?
Take a more concrete example: liquid layouts in HTML, CSS,
or Flex and MXML. In HTML and CSS, the box model is anything
but intuitive given the different implementations of
Internet Explorer and the other browsers. And unless someone
reads the documentation or studies the concept of the box
model it would be difficult to "just get it" when designing
a layout on one's first stab at CSS. I would argue this is
why tables thrived in the early days. The box model was
implicit in the concept of a table cell. With the help of
tools like Dreamweaver one could get their mind around
percentage widths and layout within the constraints of table
cells. Then CSS came into maturity and a whole set of valid
reasons emerged for why tables are not for layout. But to
achieve the same effects designers had to really study the
CSS implementations and the box model, and inject a new
layer of abstraction into their thinking.
In another example, I find when programming lots of things
in ActionScript and MXML, the whole concept of fluid layouts
and percentage based widths of elements not very obvious and
doesn't always follow intuition. I understand the basic
problem in that the Adobe Flash player and the layout need
to understand things in absolute pixel terms. When it comes
to the potential width of a component, I understand why
percentages are not immediately obvious to implement at the
core level of the code. Theoretically speaking the Flash
Player needs to know (or calculate) the exact width of a
component so that it can provide the proper geometry to the
video card when doing a draw on the screen. But when you
introduce some concept of percentages then you introduce the
theoretical possibility of an infinite width. And to find
"infinity - 1" pixels is not something a computer can
directly do without some layer of abstraction and
calculation. The viewport must be referenced. The program
must know its boundaries. So absolute widths are the norm,
although humans might prefer to design in terms of
percentages.
When it comes to programming languages can there be
expressions and features that assist intuition when thinking
about a programming task. Or are we better off "thinking
like a computer" and just RTFM'ing the manual when we need
to understand how to implement some feature or layout in
code?
If you could change the syntax or semantics of your
programming language of choice what would you add, change,
or remove to improve the "intuitiveness" of it?
Addendum, the reason for asking this question is inspired by
seeing example of what "novices" were able to achieve in
Smalltalk in Alan Kay's lecture: Doing with Images Makes
Symbols.
"If you could change the syntax or semantics of your programming language of choice what would you add, change, or remove to improve the "intuitiveness" of it?
"
Programming is hard. Really hard. Syntax changes don't matter much. IDE's are irrelevant to the fundamental challenge of programming.
The thing that is often baffling is the semantics of the language.
I don't know what "intuitive" means with respect to a thing as abstract as a programming language. Indeed, "intuition" is probably a bad thing. Coming to a programming language with intuition means preconceived notions, biases and intellectual junk will take over.
I would never expect to "just get it" for anything on any level anywhere. Programming requires clear thinking -- not "intuition" -- not "expectation".
The only thing we can ever do is read the manual and understand the unique, distinct, novel semantics of the new thing we're confronted with.
I do know this: elegant simplicity is essential. Orthogonality of features. Clarity. Precision. Absence of exceptions or special cases. Above all, simplicity.
Layering on language features is fundamentally bad.
Covering language problems by layering in a complex IDE is worse.
See http://www.cs.utexas.edu/~EWD/transcriptions/EWD08xx/EWD854.html
"when faced with something new and unfamiliar we try to relate it to what we are familiar with. In the course of the process we invent the analogies that enable us to do so.
It is clear that the above way of trying to understand does not work too well when we are faced with something so radically new, so without precedent, that all analogies we can come up with are too weak and too shallow to be of great help. A radically new technology can create such circumstances and the wide-spread misunderstanding about programming strongly suggests that this has happened with the advent of the automatic computer. "
In short, "intuition" and "intellectual baggage" is the problem of the programmer. The best way to understand a technology is to approach it as something fresh, new and otherwise unknown.
Bottom Line.
The complexity is inherent.
You have two choices.
Develop intellectual tools (i.e., abstraction, summarization, etc.) to cope with it.
Get a job in another field.
Asking for the inherently complicated world of computing to morph into something any one person finds "intuitive" can't happen. Computing is too complicated to be "intuitive".
Another field I've seen that addresses the complexity of the "syntax" of a programming languages is that of Visual Programming Languages. The basic idea behind VPLs is to take the constructs of programming languages (decisions, subroutines, functions, etc.) and represent them graphically, typically as a data-flow diagram. One such language that's gaining popularity recently is the Microsoft Visual Programming Language. I have not used it, and cannot make claims as to its power, but I have used LabView to great effect and I can say that you can do pretty much anything you can think of even in LabView -- but you do have to think of it in a very different way.
That said, I find I have a personal preference for code rather than VPLs.
One step folks are taking that has as much to do with base class library as it does the language itself -- although to be honest, the two are often synonymous -- is the concept of a Fluent API. The basic idea is to make code "read like a sentence", the idea being that this makes the code more flexible and maintainable.
I want to play around with some graphics stuff. Simple animations and things. I want to fool around with raytracing too. I need help finding a library that will help me do these things. I have a few requirements:
Must be able to do raytracing
Must be for a high level language (python, .NET, etc.). Please no C/C++
Must have good documentation, preferably with examples.
Does anyone know of a good library i can use to fool around with?
Have a look at blender.org - it's an open-source 3d project with python scripting capabilities.
First thing that come to my mind is the popular open source P.O.V Raytracer (www.povray.org). POV scenes are defined entirely with script files, and some people made Python code to generate them easily.
http://code.activestate.com/recipes/205451/
http://jabas-unblog.blogspot.com/2007/04/easy-procedural-graphics-python-and-pov.html
I'm not aware of any libraries that satisfy your request (at least not unless I decide to publish the code for my own tracer...).
Writing a tracer isn't actually that hard anyway. I'd strongly recommend getting hold of a copy of "An Introduction to Ray Tracing" by Glassner. It goes through the actual math in relatively easy to understand terms, and also has a whole section on "how to write a ray tracer".
In any event, a "library" isn't all that much use on its own - pretty much every ray tracer has its own internal libraries but they're specific to the tracer. They typically include:
a base class to represent 3D objects
subclasses of that for each geometric primitive
vector and matrix classes (3D and 4D)
texturing functions and/or classes
light classes of various types (point light, spot light, etc)
For my own tracer I actually used the javax.vecmath packages for #3 above, but had to write my own code for #1 and #2 based on the Glassner book. The whole thing is well under 2k lines of code, and most of the individual classes are about 40 lines long.
I believe there are few people putting together ray-tracers using XNA Game Studio.
One example of this with code can be seen over at:
Bespoke Software » Ray Tracing - Materials
The well developed raytracers that are open source are
Yafray
Povray
For realtime 3D (it will be language dependant of course) there is JMonkeyEngine (Java) not sure whether that meets your "high level language" requirement.
You could consider a 3D game scripting language too, like GameCore or BlitzBasic