Implementing Stackless Python - python-3.x

I really admire the functionality of Stackless Python, and I've been looking around for a way to emulate its syntax while still using the standard Python 3 interpreter. An article by Alex J. Champandard in a gamedev blog made it look as though the greenlet library could provide this functionality. I slightly modified his code, but the best makeshift tasklet wrapper I could come up with was a class holding a greenlet inside a variable, as such:
class tasklet():
def __init__(self,function=None,*variables):
global _scheduled
self.greenlet = greenlet.greenlet(function,None)
self.functioncall = function # Redundant backup
self.variables = variables
_scheduled.append(self)
self.blocked = False
The function then emulates Stackless' scheduling by passing the variables to the greenlet when calling its switch() method.
So far this appears to work, but I'd like to be able to call the tasklets in original Stackless syntax, e.g. tasklet(function)(*args), as opposed to the current syntax of tasklet(function,*args). I'm not sure where to look in the documentation to find out how to accomplish this. Is this even possible, or is it part of Stackless' changes to the interpreter?

According to this article from 2010-01-08 (with fixed links):
Stackless Python is an extended version of the Python language (and
its CPython reference implementation). New features include
lightweight coroutines (called tasklets), communication primitives
using message passing (called channels), manual and/or automatic
coroutine scheduling, not using the C stack Python function calls, and
serialization of coroutines (for reloading in another process).
Stackless Python could not be implemented as a Python extension module
– the core of the CPython compiler and interpreter had to be patched.
greenlet is an extension module to CPython providing coroutines and
low-level (explicit) scheduling. The most important advantage of
greenlet over Stackless Python is that greenlet could be implemented
as a Python extension module, so the whole Python interpreter doesn't
have to be recompiled in order to use greenlet. Disadvantages of
greenlet include speed (Stackless Python can be 10%, 35% or 900%
faster, depending on the workflow); possible memory leaks if
coroutines have references to each other; and that the provided
functionality is low-level (i.e. only manual coroutine scheduling, no
message passing provided).
greenstackless, the Python module I've recently developed, provides
most of the (high-level) Stackless Python API using greenlet, so it
eliminates the disadvantage of greenlet that it is low-level. See the
source code and some tests (the latter with tricky corner cases).
Please note that although greenstackless is optimized a bit, it can be
much slower than Stackless Python, and it also doesn't fix the memory
leaks. Using greenstackless is thus not recommended in production
environments; but it can be used as a temporary, drop-in replacement
for Stackless Python if replacing the Python interpreter is not
feasible.
Some other software that emulates Stackless using greenlet:
Concurrence: doesn't support stackless.main, tasklet.next,
tasklet.prev, tasklet.insert, tasklet.remove,
stackless.schedule_remove, doesn't send exceptions properly. (Because
of these features missing, it doesn't pass the unit test above.)
PyPy: doesn't support stackless.main, tasklet.next, tasklet.prev,
doesn't pass the unit test above.

Related

Looking for a Real Time Relative Strength Index (RSI) Indicator function for a Python Script

I would like a python function that would operate similar to: talib.RSI() (https://mrjbq7.github.io/ta-lib/)
The feature that I am looking for is that I can have it in a loop and just feed it the latest stock close price, and it would output the current RSI value. Also no forever growing stock price list would have to be maintained inside our outside of the function.
If anyone can direct me to such or how to build it, I would be very grateful.
Thanks
TA-Lib for python is just a wrapper for TA-Lib library written in C. This library doesn't support incremental calculation of indicators. It requires whole data at once. But there is a fork of TA-Lib - TA-Lib RT that introduces such functionality. Being a fork it's written in C language too and thus requires a python wrapper to be used from Python application. There is experimental adaptation of a original TA-Lib's python wrapper to TA-Lib RT here and related discussion here. You may try to set it up but note performance issues discussion at the end of the thread. You may need *BatchState functions instead of *State.
Or you may look for some python library not based on TA-Lib. I heard about talipp as an alternative. You may check its RSI implementation sources for example.

what's the differences python3 and pypy3

Today I knew that pypy3 is faster than python3 for input() time through any algorithm problem. Performance differences were almost as much as 12 times.
Why is there such a difference?
Kindly check this, when we speak of Python programming language we often mean not just the language but also the implementation. Python is a specification for a language that can be implemented in many different ways.
The default implementation of the Python programming language is Cpython(assuming python3 you mean Cpython). As the name suggests Cpython is written in C language. Cpython compiles the python source code into intermediate bytecode, which is executed by the Cpython virtual machine.
Jython is an implementation of the Python programming language that can run on the Java platform. Jython programs use Java classes instead of Python modules. Jython compiles into Java byte code, which can then be run by Java virtual machine.
PyPy
If you want your code to run faster, you should probably just use PyPy. — Guido van Rossum (creator of Python)
Python is a dynamic programming language. Python is said to be slow as the default CPython implementation compiles the python source code in bytecode which is slow as compared to machine code(native code). Here PyPy comes in.
PyPy is an implementation of the Python programming language written in Python. The Interpreter is written in RPython (a subset of Python).
PyPy uses Just In Time (JIT) compilation. In simple terms, JIT uses compilation methods to make the interpreter system more efficient and fast. So basically JIT makes it possible to compile the source code into native machine code which makes it very fast.
PyPy also comes with default support for stackless mode, providing micro-threads for massive concurrency. It is said to be approximately 7.5 times faster than Cpython.
Hope this will help you.
CPython
It is interpreter for python language written in C and C++. Interpreter convert python code (which is written by humans and can be read by humans) to machine code (which can be read/understood by machines). This process involve various steps.
CPython is the reference implementation of Python, written in C. It compiles Python code to intermediate bytecode which is then interpreted by a virtual machine. CPython provides the highest level of compatibility with Python packages and C extension modules.
If you are writing open source Python code and want to reach the widest possible audience, targeting CPython is best. To use packages which rely on C extensions to function, CPython is your only implementation option.
All versions of the Python language are implemented in C because CPython is the reference implementation.
PyPy
It is JIT compiler for python language written in RPython. JIT compiler execute code which require compilation, i.e. JIT compile code at runtime, just before executing it.
PyPy is a Python interpreter implemented in a restricted statically-typed subset of the Python language called RPython. The interpreter features a just-in-time compiler and supports multiple back-ends (C, CLI, JVM).
PyPy aims for maximum compatibility with the reference CPython implementation while improving performance.
If you are looking to increase performance of your Python code, it’s worth giving PyPy a try. On a suite of benchmarks, it’s currently over 5 times faster than CPython.
PyPy supports Python 2.7. PyPy3, released in beta, targets Python 3.
I assume that when you say python3, you mean CPython which default and widely used implementation of python language.
CPython
It is interpreter for python language written in C and C++. Interpreter convert python code(which is written by human and can read by human) to machine code(which can read/understand by machine/computer). This process involve various steps.
PyPy
It is JIT compiler for python language written in RPython. JIT compiler execute code which require compilation, i.e. JIT compile code at runtime, just before executing it.
This different approach of handling python code of these two implementation is reason behind different int speed. Below link will give you more detail on this.
There are few more implementation of Python language which aims to achieve different goal.
CPython
PyPy
JIT
Design of CPython’s Compiler
PyPy
Alternatives

Is it possible to have a "safe_eval" for primitive math expressions in Python?

While there are many questions on SE regarding sand-boxing CPython, most focus on providing a more or less complete Python environment.
In this case, I'm interested in using Python for basic math expressions, something of this complexity, eg:
(sin(a) * cos(b) / tan(c) ** sqrt(d)) - e
Now I could create my own expression evaluator in Python, however I don't want to sacrifice performance (or have to maintain it and have good Python compatibility for all the corner cases).
I looked into numba and numexpr, both very interesting projects, but neither have security/sand-boxing as a goal.
I considered writing a minimal version of ceval.c which runs a restricted set of CPython's bytecodes, but this is still quite some effort.
Instead I did a quick test that restricts name-space and checks the compiled expressions opcodes before executing, from my initial tests this works well, though I'm not totally confident its secure either.
While this isn't clear-cut, using simple expressions means I don't necessarily need:
multiple lines of code.
import statements.
defining functions & classes.
getattr or getitem access.
for & while loops.
So my question is:
Is there a reliably secure way to execute math expressions in CPython that can co-exist in the same process as complete CPython scripts?
Note, since security may be too vague a term, for the purpose of discussion.
Reading or removing files on the users system.
Running open or any functions in os or shutil.

Safe execution of untrusted Haskell code

I'm looking for a way to run an arbitrary Haskell code safely (or refuse to run unsafe code).
Must have:
module/function whitelist
timeout on execution
memory usage restriction
Capabilities I would like to see:
ability to kill thread
compiling the modules to native code
caching of compiled code
running several interpreters concurrently
complex datatype for compiler errors (insted of simple message in String)
With that sort of functionality it would be possible to implement a browser plugin capable of running arbitrary Haskell code, which is the idea I have in mind.
EDIT: I've got two answers, both great. Thanks! The sad part is that there doesn't seem to be ready-to-go library, just a similar program. It's a useful resource though. Anyway I think I'll wait for 7.2.1 to be released and try to use SafeHaskell in my own program.
We've been doing this for about 8 years now in lambdabot, which supports:
a controlled namespace
OS-enforced timeouts
native code modules
caching
concurrent interactive top-levels
custom error message returns.
This series of rules is documented, see:
Safely running untrusted Haskell code
mueval, an alternative implementation based on ghc-api
The approach to safety taken in lambdabot inspired the Safe Haskell language extension work.
For approaches to dynamic extension of compiled Haskell applications, in Haskell, see the two papers:
Dynamic Extension of Typed Functional Languages, and
Dynamic applications from the ground up.
GHC 7.2.1 will likely have a new facility called SafeHaskell which covers some of what you want. SafeHaskell ensures type-safety (so things like unsafePerformIO are outlawed), and establishes a trust mechanism, so that a library with a safe API but implemented using unsafe features can be trusted. It is designed exactly for running untrusted code.
For the other practical aspects (timeouts and so on), lambdabot as Don says would be a great place to look.

The right language for OpenGL UI prototyping. Ditching Python

So, I got this idea that I'd try to prototype an experimental user interface using OpenGL and some physics. I know little about either of the topics, but am pretty experienced with programming languages such as C++, Java and C#. After some initial research, I decided on using Python (with Eclipse/PyDev) and Qt, both new to me, and now have four different topics to learn more or less simultaneously.
I've gotten quite far with both OpenGL and Python, but while Python and its ecosystem initially seemed perfect for the task, I've now discovered some serious drawbacks. Bad API documentation and lacking code completion (due to dynamic typing), having to import every module I use in every other module gets tedious when having one class per module, having to select the correct module to run the program, and having to wait 30 seconds for the program to start and obscure the IDE before being notified of many obvious typos and other mistakes. It gets really annoying really fast. Quite frankly, i don't get what all the fuzz is about. Lambda functions, list comprehensions etc. are nice and all, but there's certainly more important things.
So, unless anyone can resolve at least some of these annoyances, Python is out. C++ is out as well, for obvious reasons, and C# is out, mainly for lack of portability. This leaves Java and JOGL as an attractive option, but I'm also curious about Ruby and Groovy. I'd like your opinion on these and others though, to keep my from making the same mistake again.
The requirements are:
Keeping the hell out of my way.
Good code completion. Complete method signatures, including data types and parameter names.
Good OpenGL support.
Qt support is preferable.
Object Oriented
Suitable for RAD, prototyping
Cross-platform
Preferably Open-Source, but at least free.
It seems you aren't mainly having a problem with Python itself, but instead with the IDE.
"Bad API documentation"
To what API? Python itself, Qt or some other library you are using?
"lacking code completion (due to dynamic typing)"
As long as you are not doing anything magic, I find that PyDev is pretty darn good at figuring these things out. If it gets lost, you can always typehint by doing:
assert isinstance(myObj, MyClass)
Then, PyDev will provide you with code completion even if myObj comes from a dynamic context.
"having to import every module I use in every other module gets tedious when having one class per module"
Install PyDev Extensions, it has auto-import on the fly. Or collect all your imports in a separate module and do:
from mymodulewithallimports import *
"having to select the correct module to run the program"
In Eclipse, you can set up a default startup file, or just check "use last run configuration". Then you never have to select it again.
"before being notified of many obvious typos and other mistakes"
Install PyDev Extensions, it has more advanced syntax checking and will happily notify you about unused imports/variables, uninitialized variables etc.
Looking just at your list I'd recommend C++; especially because Code Completion is so important to you.
About Python: Although I have few experience with OpenGL programming with Python (used C++ for that), the Python community offers a number of interesting modules for OpenGL development: pyopengl, pyglew, pygpu; just to name a few.
BTW, your import issue can be resolved easily by importing the modules in the __init__.py files of the directory the modules are contained in and then just importing the "parent" module. This is not recommended but nonetheless possible.
I don't understand why nobody has heard of the D programing language?
THIS IS THE PERFECT SOLUTION!!!!
The only real alternative if you desire all those things is to use Java, but honestly you're being a bit picky about features. Is code completion really that important a trait? Everything else you've listed is traditionally very well regarded with Python, so I don't see the issue.
The text editor (not even an IDE) which I use lets you import API function definitions. Code completion is not a language feature, especially with OpenGL. Just type gl[Ctrl+I] and you'd get the options.
I tried using Java3D and java once. I realized Java3D is a typical Java API... lots of objects to do simple things, and because it's Java, that translates to a lot of code. I then moved to Jython in Eclipse to which cleaned up the code, leaving me with only the complexity of Java3D.
So in the end, I went in the opposite direction. One advantage this has over pure python is I can use Java with all of Eclipse's benefits like autocomplete and move it over to python when parts get unwieldy in Java.
It seems like Pydev can offer code completion for you in Eclipse.
I started off doing OpenGL programming with GL4Java, which got migrated to JOGL and you should definately give it (JOGL) a try. Java offers most of the features you require (plus Eclipse gives you the code completion) and especially for JOGL there are a lot of tutorials out there to get you started.
Consider Boo -- it has many of Python's advantages while adopting features from elsewhere as well, and its compile-time type inference (when variables are neither explicitly given a specific type or explicitly duck typed) allows the kind of autocompletion support you're asking about.
The Tao.OpenGL library exposes OpenGL to .NET apps (such as those Boo compiles), with explicit support for Mono.
(Personally, I'm mostly a Python developer when not doing C or Java, but couldn't care less about autocompletion... but hey, it's your question; also, the one-class-per-module convention seems like a ridiculous amount of pain you're putting yourself through needlessly).

Resources