What does "special compiler magic" mean? - nim-lang

In the Nim system module it says:
Most of the routines listed here use special compiler magic.
What do they mean by "special compiler magic"?

Generally it means that the implementations of the routines in that module cannot be expressed in the source language itself. I.e. the compiler recognizes a special set of names and either uses special semantics or compiles in hand-written assembly, etc. The notice is usually to prevent someone from trying to reason about how the interface could be written in the language itself.

Related

What's the meaning of "magic" pragma in nim?

When going through the code in Nim project itself, I find that some proc decorated by "magic" pragma misses proc definition (example). There's no doc to explain the pragma, I guess the proc's definition is somewhere else and is merged while compiling.But I cannot still find the definition by searching the whole project.
Do I misunderstand the "magic" pragma? What's the meaning of it? And how to use it?
The {.magic.} pragma is used to define built-in operations and types, i.e. anything that requires compiler magic in order to work (hence the name). It is not intended to be used outside the system modules.

Compiled Language with Dynamic Typing

I'm a bit confused when it comes to a compiled language (compilation to native code) with dynamic typing.
Dynamic typing says that the types in a program are only inferred at runtime.
Now if a language is compiled, there's no interpreter running at runtime; it's just your CPU reading instructions off memory and executing them. In such a scenario, if any instruction violating the type semantics of the language happens to execute at runtime, there's no interpreter to intercept the execution of the program and throw any errors. How does the system work then?
What happens when an instruction violating the type semantics of a dynamically typed compiled language is executed at runtime?
PS: Some of the dynamically typed compiled languages I know of include Scheme, Lua and Common Lisp.
A compiler for a dynamically typed language would simply generate instructions that check the type where necessary. In fact even for some statically typed languages this is sometimes necessary, say, in case of an object oriented language with checked casts (like dynamic_cast in C++). In dynamically types languages it is simply necessary more often.
For these checks to be possible each value needs to be represented in a way that keeps track of its type. A common way to represent values in dynamically typed languages is to represent them as pointers to a struct that contains the type as well as the value (as an optimization this is often avoided in case of (sufficiently small) integers by storing the integer directly as an invalid pointer (by shifting the integer one to the left and setting its least significant bit)).

Does "The whole language always available" hold in case of Clojure?

Ninth bullet point in Paul Graham's What Made Lisp Different says,
9. The whole language always available.
There is no real distinction between read-time, compile-time, and runtime. You can compile or run code while reading, read or run code while compiling, and read or compile code at runtime.
Running code at read-time lets users reprogram Lisp's syntax; running code at compile-time is the basis of macros; compiling at runtime is the basis of Lisp's use as an extension language in programs like Emacs; and reading at runtime enables programs to communicate using s-expressions, an idea recently reinvented as XML.
Does this last bullet point hold for Clojure?
You can mix runtime and compile-time freely in Clojure, although Common Lisp is still somewhat more flexible here (due to the presence of compiler macros and symbol macros and a fully supported macrolet; Clojure has an advantage in its cool approach to macro hygiene through automagic symbol resolution in syntax-quote). The reader is currently closed, so the free mixing of runtime, compile-time and read-time is not possible1.
1 Except through unsupported clever hacks.
It does hold,
(eval (read-string "(println \"Hello World!!\")"))
Hello World!!
nil
Just like emacs you can have your program configuration in Clojure, one project that I know Clojure for is static which allows you to have your template as a Clojure vector along with arbitrary code which will be executed at read time.

Is there a compiled* programming language with dynamic, maybe even weak typing?

I wondered if there is a programming language which compiles to machine code/binary (not bytecode then executed by a VM, that's something completely different when considering typing) that features dynamic and/or weak typing, e.g:
Think of a compiled language where:
Variables don't need to be declared
Variables can be created during runtime
Functions can return values of different types
Questions:
Is there such a programming language?
(Why) not?
I think that a dynamically yet strong typed, compiled language would really sense, but is it possible?
I believe Lisp fits that description.
http://en.wikipedia.org/wiki/Common_Lisp
Yes, it is possible. See Julia. It is a dynamic language (you can write programs without types) but it never runs on a VM. It compiles the program to native code at runtime (JIT compilation).
Objective-C might have some of the properties you seek. Classes can be opened and altered in runtime, and you can send any kind of message to an object, whether it usually responds to it or not. In that way, you can implement duck typing, much like in Ruby. The type id, roughly equivalent to a void*, can be endowed with interfaces that specify a contract that the (otherwise unknown) type will adhere to.
C# 4.0 has many, if not all of these characteristics. If you really want native machine code, you can compile the bytecode down to machine code using a utility.
In particular, the use of the dynamic keyword allows objects and their members to be bound dynamically at runtime.
Check out Anders Hejlsberg's video, The Future of C#, for a primer:
http://channel9.msdn.com/pdc2008/TL16/
Objective-C has many of the features you mention: it compiles to machine code and is effectively dynamically typed with respect to object instances. The id type can store any class instance and Objective-C uses message passing instead of member function calls. Methods can be created/added at runtime. The Objective-C runtime can also synthesize class instance variables at runtime, but local variables still need to be declared (just as in C).
C# 4.0 has many of these features, except that it is compiled to IL (bytecode) and interpreted using a virtual machine (the CLR). This brings up an interesting point, however: if bytecode is just-in-time compiled to machine code, does that count? If so, it opens to the door to not only any of the .Net languages, but Python (see PyPy or Unladed Swallow or IronPython) and Ruby (see MacRuby or IronRuby) and many other dynamically typed languages, not mention many LISP variants.
In a similar vein to Lisp, there is Factor, a concatenative* language with no variables by default, dynamic typing, and a flexible object system. Factor code can be run in the interactive interpreter, or compiled to a native executable using its deploy function.
* point-free functional stack-based
VB 6 has most of that
I don't know of any language that has exactly those capabilities. I can think of two that have a significant subset, though:
D has type inference, garbage collection, and powerful metaprogramming facilities, yet compiles to efficient machine code. It does not have dynamic typing, however.
C# can be compiled directly to machine code via the mono project. C# has a similar feature set to D, but again without dynamic typing.
Python to C probably needs these criteria.
Write in Python.
Compile Python to Executable. See Process to convert simple Python script into Windows executable. Also see Writing code translator from Python to C?
Elixir does this. The flexibility of dynamic variable typing helps with doing hot-code updates (for which Erlang was designed). Files are compiled to run on the BEAM, the Erlang/Elixir VM.
C/C++ both indirectly support dynamic typing using void*. C++ example:
#include <string>
int main() {
void* x = malloc(sizeof(int))
*(int*)x = 5;
x = malloc(sizeof(std::string));
*(std::string*x) = std::string("Hello world");
free(x);
return 0;
}
In C++17, std::any can be used as well:
#include <string>
#include <any>
int main() {
std::any x = 5;
x = std::string("Hello world");
return 0;
}
Of course, duck typing is rarely used or needed in C/C++, and both of these options have issues (void* is unsafe, std::any is a huge performance bottleneck).
Another example of what you may be looking for is the V8 engine for JavaScript. It is a JIT compiler, meaning the source code is compiled to bytecode and then machine code at runtime, although this is hidden from the user.

What is the difference between statically typed and dynamically typed languages?

What does it mean when we say a language is dynamically typed versus statically typed?
Statically typed languages
A language is statically typed if the type of a variable is known at compile time. For some languages this means that you as the programmer must specify what type each variable is; other languages (e.g.: Java, C, C++) offer some form of type inference, the capability of the type system to deduce the type of a variable (e.g.: OCaml, Haskell, Scala, Kotlin).
The main advantage here is that all kinds of checking can be done by the compiler, and therefore a lot of trivial bugs are caught at a very early stage.
Examples: C, C++, Java, Rust, Go, Scala
Dynamically typed languages
A language is dynamically typed if the type is associated with run-time values, and not named variables/fields/etc. This means that you as a programmer can write a little quicker because you do not have to specify types every time (unless using a statically-typed language with type inference).
Examples: Perl, Ruby, Python, PHP, JavaScript, Erlang
Most scripting languages have this feature as there is no compiler to do static type-checking anyway, but you may find yourself searching for a bug that is due to the interpreter misinterpreting the type of a variable. Luckily, scripts tend to be small so bugs have not so many places to hide.
Most dynamically typed languages do allow you to provide type information, but do not require it. One language that is currently being developed, Rascal, takes a hybrid approach allowing dynamic typing within functions but enforcing static typing for the function signature.
Type checking is the process of verifying and enforcing the constraints of types.
Statically typed programming languages do type checking at compile-time.
Examples: Java, C, C++.
Dynamically typed programming languages do type checking at run-time.
Examples:
Perl, Ruby, Python, PHP, JavaScript.
Here is an example contrasting how Python (dynamically typed) and Go (statically typed) handle a type error:
def silly(a):
if a > 0:
print 'Hi'
else:
print 5 + '3'
Python does type checking at run time, and therefore:
silly(2)
Runs perfectly fine, and produces the expected output Hi. Error is only raised if the problematic line is hit:
silly(-1)
Produces
TypeError: unsupported operand type(s) for +: 'int' and 'str'
because the relevant line was actually executed.
Go on the other hand does type-checking at compile time:
package main
import ("fmt"
)
func silly(a int) {
if (a > 0) {
fmt.Println("Hi")
} else {
fmt.Println("3" + 5)
}
}
func main() {
silly(2)
}
The above will not compile, with the following error:
invalid operation: "3" + 5 (mismatched types string and int)
Simply put it this way: in a statically typed language variables' types are static, meaning once you set a variable to a type, you cannot change it. That is because typing is associated with the variable rather than the value it refers to.
For example in Java:
String str = "Hello"; // variable str statically typed as string
str = 5; // would throw an error since str is
// supposed to be a string only
Where on the other hand: in a dynamically typed language variables' types are dynamic, meaning after you set a variable to a type, you CAN change it. That is because typing is associated with the value it assumes rather than the variable itself.
For example in Python:
some_str = "Hello" # variable some_str is linked to a string value
some_str = 5 # now it is linked to an integer value; perfectly OK
So, it is best to think of variables in dynamically typed languages as just generic pointers to typed values.
To sum up, type describes (or should have described) the variables in the language rather than the language itself. It could have been better used as a language with statically typed variables versus a language with dynamically typed variables IMHO.
Statically typed languages are generally compiled languages, thus, the compilers check the types (make perfect sense right? as types are not allowed to be changed later on at run time).
Dynamically typed languages are generally interpreted, thus type checking (if any) happens at run time when they are used. This of course brings some performance cost, and is one of the reasons dynamic languages (e.g., python, ruby, php) do not scale as good as the typed ones (java, c#, etc.). From another perspective, statically typed languages have more of a start-up cost: makes you usually write more code, harder code. But that pays later off.
The good thing is both sides are borrowing features from the other side. Typed languages are incorporating more dynamic features, e.g., generics and dynamic libraries in c#, and dynamic languages are including more type checking, e.g., type annotations in python, or HACK variant of PHP, which are usually not core to the language and usable on demand.
When it comes to technology selection, neither side has an intrinsic superiority over the other. It is just a matter of preference whether you want more control to begin with or flexibility. just pick the right tool for the job, and make sure to check what is available in terms of the opposite before considering a switch.
http://en.wikipedia.org/wiki/Type_system
Static typing
A programming language is said to use
static typing when type checking is
performed during compile-time as
opposed to run-time. In static typing,
types are associated with variables
not values. Statically typed languages
include Ada, C, C++, C#, JADE, Java,
Fortran, Haskell, ML, Pascal, Perl
(with respect to distinguishing
scalars, arrays, hashes and
subroutines) and Scala. Static typing
is a limited form of program
verification (see type safety):
accordingly, it allows many type
errors to be caught early in the
development cycle. Static type
checkers evaluate only the type
information that can be determined at
compile time, but are able to verify
that the checked conditions hold for
all possible executions of the
program, which eliminates the need to
repeat type checks every time the
program is executed. Program execution
may also be made more efficient (i.e.
faster or taking reduced memory) by
omitting runtime type checks and
enabling other optimizations.
Because they evaluate type information
during compilation, and therefore lack
type information that is only
available at run-time, static type
checkers are conservative. They will
reject some programs that may be
well-behaved at run-time, but that
cannot be statically determined to be
well-typed. For example, even if an
expression always
evaluates to true at run-time, a
program containing the code
if <complex test> then 42 else <type error>
will be rejected as ill-typed, because
a static analysis cannot determine
that the else branch won't be
taken.[1] The conservative behaviour
of static type checkers is
advantageous when
evaluates to false infrequently: A
static type checker can detect type
errors in rarely used code paths.
Without static type checking, even
code coverage tests with 100% code
coverage may be unable to find such
type errors. Code coverage tests may
fail to detect such type errors
because the combination of all places
where values are created and all
places where a certain value is used
must be taken into account.
The most widely used statically typed
languages are not formally type safe.
They have "loopholes" in the
programming language specification
enabling programmers to write code
that circumvents the verification
performed by a static type checker and
so address a wider range of problems.
For example, Java and most C-style
languages have type punning, and
Haskell has such features as
unsafePerformIO: such operations may
be unsafe at runtime, in that they can
cause unwanted behaviour due to
incorrect typing of values when the
program runs.
Dynamic typing
A programming language is said to be
dynamically typed, or just 'dynamic',
when the majority of its type checking
is performed at run-time as opposed to
at compile-time. In dynamic typing,
types are associated with values not
variables. Dynamically typed languages
include Groovy, JavaScript, Lisp, Lua,
Objective-C, Perl (with respect to
user-defined types but not built-in
types), PHP, Prolog, Python, Ruby,
Smalltalk and Tcl. Compared to static
typing, dynamic typing can be more
flexible (e.g. by allowing programs to
generate types and functionality based
on run-time data), though at the
expense of fewer a priori guarantees.
This is because a dynamically typed
language accepts and attempts to
execute some programs which may be
ruled as invalid by a static type
checker.
Dynamic typing may result in runtime
type errors—that is, at runtime, a
value may have an unexpected type, and
an operation nonsensical for that type
is applied. This operation may occur
long after the place where the
programming mistake was made—that is,
the place where the wrong type of data
passed into a place it should not
have. This makes the bug difficult to
locate.
Dynamically typed language systems,
compared to their statically typed
cousins, make fewer "compile-time"
checks on the source code (but will
check, for example, that the program
is syntactically correct). Run-time
checks can potentially be more
sophisticated, since they can use
dynamic information as well as any
information that was present during
compilation. On the other hand,
runtime checks only assert that
conditions hold in a particular
execution of the program, and these
checks are repeated for every
execution of the program.
Development in dynamically typed
languages is often supported by
programming practices such as unit
testing. Testing is a key practice in
professional software development, and
is particularly important in
dynamically typed languages. In
practice, the testing done to ensure
correct program operation can detect a
much wider range of errors than static
type-checking, but conversely cannot
search as comprehensively for the
errors that both testing and static
type checking are able to detect.
Testing can be incorporated into the
software build cycle, in which case it
can be thought of as a "compile-time"
check, in that the program user will
not have to manually run such tests.
References
Pierce, Benjamin (2002). Types and Programming Languages. MIT Press.
ISBN 0-262-16209-1.
Compiled vs. Interpreted
"When source code is translated"
Source Code: Original code (usually typed by a human into a computer)
Translation: Converting source code into something a computer can read (i.e. machine code)
Run-Time: Period when program is executing commands (after compilation, if compiled)
Compiled Language: Code translated before run-time
Interpreted Language: Code translated on the fly, during execution
Typing
"When types are checked"
5 + '3' is an example of a type error in strongly typed languages such as Go and Python, because they don't allow for "type coercion" -> the ability for a value to change type in certain contexts such as merging two types. Weakly typed languages, such as JavaScript, won't throw a type error (results in '53').
Static: Types checked before run-time
Dynamic: Types checked on the fly, during execution
The definitions of "Static & Compiled" and "Dynamic & Interpreted" are quite similar...but remember it's "when types are checked" vs. "when source code is translated".
You'll get the same type errors irrespective of whether the language is compiled or interpreted! You need to separate these terms conceptually.
Python Example
Dynamic, Interpreted
def silly(a):
if a > 0:
print 'Hi'
else:
print 5 + '3'
silly(2)
Because Python is both interpreted and dynamically typed, it only translates and type-checks code it's executing on. The else block never executes, so 5 + '3' is never even looked at!
What if it was statically typed?
A type error would be thrown before the code is even run. It still performs type-checking before run-time even though it is interpreted.
What if it was compiled?
The else block would be translated/looked at before run-time, but because it's dynamically typed it wouldn't throw an error! Dynamically typed languages don't check types until execution, and that line never executes.
Go Example
Static, Compiled
package main
import ("fmt"
)
func silly(a int) {
if (a > 0) {
fmt.Println("Hi")
} else {
fmt.Println("3" + 5)
}
}
func main() {
silly(2)
}
The types are checked before running (static) and the type error is immediately caught! The types would still be checked before run-time if it was interpreted, having the same result. If it was dynamic, it wouldn't throw any errors even though the code would be looked at during compilation.
Performance
A compiled language will have better performance at run-time if it's statically typed (vs. dynamically); knowledge of types allows for machine code optimization.
Statically typed languages have better performance at run-time intrinsically due to not needing to check types dynamically while executing (it checks before running).
Similarly, compiled languages are faster at run time as the code has already been translated instead of needing to "interpret"/translate it on the fly.
Note that both compiled and statically typed languages will have a delay before running for translation and type-checking, respectively.
More Differences
Static typing catches errors early, instead of finding them during execution (especially useful for long programs). It's more "strict" in that it won't allow for type errors anywhere in your program and often prevents variables from changing types, which further defends against unintended errors.
num = 2
num = '3' // ERROR
Dynamic typing is more flexible, which some appreciate. It typically allows for variables to change types, which can result in unexpected errors.
The terminology "dynamically typed" is unfortunately misleading. All languages are statically typed, and types are properties of expressions (not of values as some think). However, some languages have only one type. These are called uni-typed languages. One example of such a language is the untyped lambda calculus.
In the untyped lambda calculus, all terms are lambda terms, and the only operation that can be performed on a term is applying it to another term. Hence all operations always result in either infinite recursion or a lambda term, but never signal an error.
However, were we to augment the untyped lambda calculus with primitive numbers and arithmetic operations, then we could perform nonsensical operations, such adding two lambda terms together: (λx.x) + (λy.y). One could argue that the only sane thing to do is to signal an error when this happens, but to be able to do this, each value has to be tagged with an indicator that indicates whether the term is a lambda term or a number. The addition operator will then check that indeed both arguments are tagged as numbers, and if they aren't, signal an error. Note that these tags are not types, because types are properties of programs, not of values produced by those programs.
A uni-typed language that does this is called dynamically typed.
Languages such as JavaScript, Python, and Ruby are all uni-typed. Again, the typeof operator in JavaScript and the type function in Python have misleading names; they return the tags associated with the operands, not their types. Similarly, dynamic_cast in C++ and instanceof in Java do not do type checks.
Statically typed languages: each variable and expression is already known at compile time.
(int a; a can take only integer type values at runtime)
Examples: C, C++, Java
Dynamically typed languages: variables can receive different values at runtime and their type is defined at run time.
(var a; a can take any kind of values at runtime)
Examples: Ruby, Python.
In Programming, Data Type is a Classification which tells what type of value a variable will hold and what are the mathematical, relational and logical operations can be done on those values without getting error.
In each programming language, to minimize the chance of getting error, type checking is done either before or during program execution. Depending on the Timing of Type Checking, programming languages are 2 types : Statically Typed and Dynamically Typed languages.
Also depending on whether Implicit Type Conversion happens or not, programming languages are 2 types : Strongly Typed and Weakly Typed languages.
Statically Typed :
Type checking is done at compile time
In source code, at the time of variable declaration, data type of that variable must be explicitly specified. Because if data type is specified in source code then at compile time that source code will be converted to machine code and type checking can happen
Here data type is associated with variable like, int count. And this association is static or fixed
If we try to change data type of an already declared variable (int count) by assigning a value of other data type (int count = "Hello") into it, then we will get error
If we try to change data type by redeclaring an already declared variable (int count) using other data type (boolean count) then also we will get error
int count; /* count is int type, association between data type
and variable is static or fixed */
count = 10; // no error
count = 'Hello'; // error
boolean count; // error
As type checking and type error detection is done at compile time that's why during runtime no further type checking is needed. Thus program becomes more optimized, results in faster execution
If we want more rigid code then choosing this type of language is better option
Example : Java, C, C++, Go, Swift etc.
Dynamically Typed :
Type checking is done at runtime
In source code, at the time of variable declaration, no need to explicitly specify data type of that variable. Because during type checking at runtime, the language system determines variable type from data type of the assigned value to that variable
Here data type is associated with the value assigned to the variable like, var foo = 10, 10 is a Number so now foo is of Number data type. But this association is dynamic or flexible
we can easily change data type of an already declared variable (var foo = 10), by assigning a value of other data type (foo = "Hi") into it, no error
we can easily change data type of an already declared variable (var foo = 10), by redeclaring it using value of other data type (var foo = true), no error
var foo; // without assigned value, variable holds undefined data type
var foo = 10; // foo is Number type now, association between data
// type and value is dynamic / flexible
foo = 'Hi'; // foo is String type now, no error
var foo = true; // foo is Boolean type now, no error
As type checking and type error detection is done at runtime, that's why program becomes less optimized, results in slower execution. Although execution of these type of languages can be faster if they implement JIT Compiler
If we want to write and execute code easily then this type of language is better option, but here we can get runtime error
Example : Python, JavaScript, PHP, Ruby etc.
Statically typed languages type-check at compile time and the type can NOT change. (Don't get cute with type-casting comments, a new variable/reference is created).
Dynamically typed languages type-check at run-time and the type of an variable CAN be changed at run-time.
Sweet and simple definitions, but fitting the need:
Statically typed languages binds the type to a variable for its entire scope (Seg: SCALA)
Dynamically typed languages bind the type to the actual value referenced by a variable.
In a statically typed language, a variable is associated with a type which is known at compile time, and that type remains unchanged throughout the execution of a program. Equivalently, the variable can only be assigned a value which is an instance of the known/specified type.
In a dynamically typed language, a variable has no type, and its value during execution can be anything of any shape and form.
Static typed languages (compiler resolves method calls and compile references):
usually better performance
faster compile error feedback
better IDE support
not suited for working with undefined data formats
harder to start a development when model is not defined when
longer compilation time
in many cases requires to write more code
Dynamic typed languages (decisions taken in running program):
lower performance
faster development
some bugs might be detected only later in run-time
good for undefined data formats (meta programming)
Static Type: Type checking performed at compile time.
What actually mean by static type language:
type of a variable must be specified
a variable can reference only a particular type of object*
type check for the value will be performed at the compile time and any type checking will be reported at that time
memory will be allocated at compile time to store the value of that particular type
Example of static type language are C, C++, Java.
Dynamic Type: Type checking performed at runtime.
What actually mean by dynamic type language:
no need to specify type of the variable
same variable can reference to different type of objects
Python, Ruby are examples of dynamic type language.
* Some objects can be assigned to different type of variables by typecasting it (a very common practice in languages like C and C++)
Statically typed languages like C++, Java and Dynamically typed languages like Python differ only in terms of the execution of the type of the variable.
Statically typed languages have static data type for the variable, here the data type is checked during compiling so debugging is much simpler...whereas Dynamically typed languages don't do the same, the data type is checked which executing the program and hence the debugging is bit difficult.
Moreover they have a very small difference and can be related with strongly typed and weakly typed languages. A strongly typed language doesn't allow you to use one type as another eg. C and C++ ...whereas weakly typed languages allow eg.python
Statically Typed
The types are checked before run-time so mistakes can be caught earlier.
Examples = c++
Dynamically Typed
The types are checked during execution.
Examples = Python
Dynamically typed programming that allows the program to change the type of the variable at runtime.
dynamic typing languages : Perl, Ruby, Python, PHP, JavaScript, Erlang
Statically typed, means if you try to store a string in an integer variable, it would not accept it.
Statically typed languages :C, C++, Java, Rust, Go, Scala, Dart
dynamically typed language helps to quickly prototype algorithm concepts without the overhead of about thinking what variable types need to be used (which is a necessity in statically typed language).
Static Typing:
The languages such as Java and Scala are static typed.
The variables have to be defined and initialized before they are used in a code.
for ex. int x; x = 10;
System.out.println(x);
Dynamic Typing:
Perl is an dynamic typed language.
Variables need not be initialized before they are used in code.
y=10; use this variable in the later part of code

Resources