What are the current rules for writing python code that will pass cleanly through 2to3 and what are the practices that seem to be best suited to writing code that will not become mired forever in version 2.
I have read from the SciPy/NumPy forums that "100% test coverage" (unit testing) is important for many people, and I am not sure if that would apply to everybody. Certainly having a reasonable set of unit tests to try your code out with after conversion, seems a sane step.
Are there other things? What are skilled Pythonistas doing if they are writing 2.x code that they hope to have come through "cleanly" in the 2to3 process.
I am looking for specific instances of "[don't] do this" as well as some more general "best-practices", but specific instances of "do's and don'ts" are helpful.
Let's assume that frameworks, libraries (Django, SciPy/NumPy), and every other C Extension we need gets ported to Python3 eventually, and I'm asking about how you write and maintain the pure python language code that you write yourself.
Update: It's possible that what I really want is the "style guide" and list of deprecated features that everybody was already staying away from. I cut my teeth on Python 1.5 and moved to 2.0, and then have not really followed much of the 2.5/2.6 era, used them but really my code is more 2.1 era.
I'd say:
Read the "What's new for Python 3.0". Very informative.
In particular, if you care about Unicode or text encodings at all, take the time to understand what has changed for 3.x. That's probably one of the trickier things to change for Python 3.x.
Get Python 2.6 or 2.7, and run your code with the -3 flag. It will tell you about things in your code that will need changing.
Before using 3rd-party packages, check to see if they have a Python 3.x version. If not, check the package web site, mailing lists, version control repositories etc to see how actively the package is being developed and whether there is a roadmap towards Python 3.x support.
Download Python 3.x and try it out! Admittedly, that might not be practical if you care about code that currently depends on packages that don't yet support Python 3.x (e.g. wxPython or Django).
Related
New to Python and come from a statically typed language background. I want type hints for https://lxml.de just for ease of development (mypy flagging issues and suggesting methods would be nice!)
To my knowledge, this is a python 2.0 module and doesn’t have types. Currently I’ve used https://mypy.readthedocs.io/en/stable/stubgen.html to create stub type definitions and filling in “any”-types I’m using with more information, but it’s really hacky. Are there any safer ways to get type hints?
There is an official stubs package for lxml now called lxml-stubs:
$ pip install lxml-stubs
Note, however, that the stubs are still in development and are not 100% complete yet (although very much usable from my experience). These stubs were once part of typeshed, then curated by Jelle Zijlstra after removal and now are developed as part of the lxml project.
If you want the development version of the stubs, install via
$ pip install git+https://github.com/lxml/lxml-stubs.git
(the project's readme installation command is missing the git+ prefix in URL's scheme and won't work).
Recently I have done much more gap filling based on lxml-stubs with some good progress.
Welcome to check out types-lxml if any late comer are still interested. For most people I think lxml.objectify is the only missing piece lacking from the stubs, which is planned immediately after current release.
I’ve used stubgen to create stub type definitions and filling in “any”-types
This is actually the correct approach if it's not lxml; creating template from mypy stubgen is the starting point for many stub files. But lxml is mostly written in Cython, for which stubgen do not have perfect support yet. Besides as OP noted, this is a python 2.0 era module, and author uses function arguments in a quite polymorphous way. There are lots of unique challenges annotating lxml, as lxml is essentially a python interface for libxml and libxslt in its core.
As an example, the support of both unicode and bytes input complicates matter too; this is the same difficulty found when annotating xml.etree bundled with python, but in a much greater magnitude.
I would not call this "hacky", rather it is gradual typing.
You can take a closer look at lxml-stubs repository. From about:
This repository contains external type annotations (see PEP 484) for the lxml package. Such type annotations are normally included in typeshed, but lxml's annotations were frequently problematic and have therefore been deleted from typeshed. In particular, the stubs are incomplete and it has been difficult to provide complete stubs.
Perhaps it will be useful to you
I've been away from Python for a while (just a normal guy trying to learn it) and wanted to start learning again. I came across Anaconda and I am trying to figure out whether to use it or stick to pip. There are several questions for this, but they are all related to people using Python for Datascience, which I use R for.
So, is there any reason for me to use Anaconda if I am not planning on working with Datascience?
Sorry if this question seems easy to answer by searching, but I cannot find any information about this that isn't related to Datascience.
Off-hand, I'd say no. Anaconda simply clumps many (not all) of the standard data science-y packages with Python. If you go base Python, you get a much smaller install. Now one thing that is useful: conda. I find it's pretty good at managing package versions, which can be pretty problematic otherwise. But you can probably get by without it.
Is there a way I can use the deap library inside grasshopper's Python node
I want to run a genetic algorithm but the fitness function is to be calculated by grasshopper (only the fitness function, all the other things are to be taken of by deap inside the python node)
can it be done?
I am having problem with
importing the deap library in grasshopper's Python interface(I think I will be able to solve it by copying the files manually from Python path)
(major problem) grashopper doesn't allow closed loops so I cant seem to find a way to feed the fitness back into the Python node with the main code
couldnt get it to work, had to make do with the grasshopper pluggins
the problem was that you can only install iron python libraries for grasshopper
These are two well known issues with 'out-of-the-box' grasshopper but there are several plugins that can help overcome them.
Question One
The basic GHPython component uses Iron Python and can limit which libraries are compatible and able to be used. To get around this constraint there is a plugin called 'GH_CPython'. It allows you to set a locally installed python interpreter for your code, and then have access to any libraries available to that local interpreter. So if you install deap Libary locally then it will be available within the grasshopper GH_Cpython editor. Here is a link to download and install GH_CPython: https://www.food4rhino.com/en/app/ghcpython
Question Two
As you noted, Grasshopper is procedural and has limited support for recursive routines. To get around this there are several plugins that support recursion and may be able to help with your implementation. Which plugin would be best for your situation is difficult to say without a deeper description of your goals. Here are several options, each option provides recursive functionality that would allow for 'closed loops' where results of a script can be fed back as input.
Hoopsnake - very basic and has been around the longest
Anemone - A little more flexible and uses multiple components for loop start and end for cleaner-looking scripts. It also has a 'record history' functionality.
Octopus - Has a 'Loop' component that is similar to Hoopsnake. It also has a 'record history' functionality.
I've recently did some web design as a hobby with a primary motivation to learn interesting things. It was certainly nice to learn Python, but I found out there has just been a Great Python Rewrite too late, so I had to learn both Python 3 and 2.6 essentially.
I'm a newbie, so I'd like people to share what they think the strengths/weaknesses of Python 3 are from the perspective of those who do end-user programming rather than language designers. My question would be more of what people are actually liking to the point of using, or shunning as being unproductive or unpythonic.
For me, with statement is definite plus, while breaking print operator is definitely minus.
Clarification edit: there are many posts that ask whether one should learn Python 2 or 3 or whether there is any difference. I see my question is different: the feedback from people who for whatever reason made the choice of using Python 3 but might have an opinion about what works better, what not.
Another clarification: It has been pointed in the answers that with is backported to 2.*. Apologies.
I'm not using Python 3 "in production", yet, but in playing around with it I've found that print being a function is a superb idea -- for example, I can easily put it in a lambda now, where in 2.* I have to use sys.stdout.write("%s\n" % foo), a bit crufty. Plus, the syntax for such tweaks as using an output file different from sys.stdout or removing the final \n is so much more readable than Python 2.*'s!
BTW, with is also in recent Python 2.* versions, it's not a Python 3 - exclusive.
Well a strong point is the clarification between bytes and string. How many times in your short Python experience have you been confused with the unclear UnicodeDecodeError and UnicodeEncodeError? If you never had troubles with unicode vs bytestrings, well chances are that you are using an ascii-only language, (English? ;) ) but this is usually the concept which is the hardest to grasp for beginners. (by the way, if you're still confused, this link should help for Python 2.x)
I really think that this distinction between str, and bytes, is one of the strong points of Python3.0. Read PEP358 for the formal description, and the diveintopython class for something more end-user oriented. This new feature forces developers to maintain a clear distinction between unicode objects, and bytes objects which are encoded in a specific encoding. I believe that this change will help newcomers understanding more easily the difference between the two structures, and will help experienced developers using sane programming methods.
But of course this change has its own inconvenients: porting 2.x applications is quite difficult, and these str+unicode to str+bytes change is the most annoying thing to change if you are not already clearly separating Unicode and byte strings in your 2.x code. Annoying, but long-needed.
Those breaking changes look annoying to a lot of users, and... are annoying to implement for important librairies/solutions. The current force of Python2.x is the numerous third-party applications/modules: but because it is sometimes not-trivial to port to Python3, those third-party apps will need some time to be ported (and because 2.x is still alive, those applications will need to maintain two versions: one aimed to 2.x clients, and one to 3.x... costly maintenance!) For the next year, the number of fully-fledged application running Python3 will likely be quite low, because of the low number of Python3-compatible third parties. But again, I strongly support these breaking changes: have you read this Monkey, banana, Python(3) and fire hose tale? ;)
I think everything they did was for the best, in the long run. They removed a lot of the deprecated ways to do things, thus enforcing "There's Only One Way to Do It" and increasing consistency. Also, the with statement is awesome.
The obvious problem with using Python 3 is its lack of support for a lot of [big] libraries out there (such as Django). If none of your libraries break with Python 3, there's no reason not to use it.
I really like dictionary comprehension:
{k: v for k, v in stuff}
And extended iterable unpacking:
(head, *rest) = range(5)
This is really subjective. Python3.x is certainly an improvement over 2.x. It contains long anticipated changes like: Dictionary comprehensions, ordered dictionary, more powerful string formatting...etc Not to mention cleaner library.
So, I got this idea that I'd try to prototype an experimental user interface using OpenGL and some physics. I know little about either of the topics, but am pretty experienced with programming languages such as C++, Java and C#. After some initial research, I decided on using Python (with Eclipse/PyDev) and Qt, both new to me, and now have four different topics to learn more or less simultaneously.
I've gotten quite far with both OpenGL and Python, but while Python and its ecosystem initially seemed perfect for the task, I've now discovered some serious drawbacks. Bad API documentation and lacking code completion (due to dynamic typing), having to import every module I use in every other module gets tedious when having one class per module, having to select the correct module to run the program, and having to wait 30 seconds for the program to start and obscure the IDE before being notified of many obvious typos and other mistakes. It gets really annoying really fast. Quite frankly, i don't get what all the fuzz is about. Lambda functions, list comprehensions etc. are nice and all, but there's certainly more important things.
So, unless anyone can resolve at least some of these annoyances, Python is out. C++ is out as well, for obvious reasons, and C# is out, mainly for lack of portability. This leaves Java and JOGL as an attractive option, but I'm also curious about Ruby and Groovy. I'd like your opinion on these and others though, to keep my from making the same mistake again.
The requirements are:
Keeping the hell out of my way.
Good code completion. Complete method signatures, including data types and parameter names.
Good OpenGL support.
Qt support is preferable.
Object Oriented
Suitable for RAD, prototyping
Cross-platform
Preferably Open-Source, but at least free.
It seems you aren't mainly having a problem with Python itself, but instead with the IDE.
"Bad API documentation"
To what API? Python itself, Qt or some other library you are using?
"lacking code completion (due to dynamic typing)"
As long as you are not doing anything magic, I find that PyDev is pretty darn good at figuring these things out. If it gets lost, you can always typehint by doing:
assert isinstance(myObj, MyClass)
Then, PyDev will provide you with code completion even if myObj comes from a dynamic context.
"having to import every module I use in every other module gets tedious when having one class per module"
Install PyDev Extensions, it has auto-import on the fly. Or collect all your imports in a separate module and do:
from mymodulewithallimports import *
"having to select the correct module to run the program"
In Eclipse, you can set up a default startup file, or just check "use last run configuration". Then you never have to select it again.
"before being notified of many obvious typos and other mistakes"
Install PyDev Extensions, it has more advanced syntax checking and will happily notify you about unused imports/variables, uninitialized variables etc.
Looking just at your list I'd recommend C++; especially because Code Completion is so important to you.
About Python: Although I have few experience with OpenGL programming with Python (used C++ for that), the Python community offers a number of interesting modules for OpenGL development: pyopengl, pyglew, pygpu; just to name a few.
BTW, your import issue can be resolved easily by importing the modules in the __init__.py files of the directory the modules are contained in and then just importing the "parent" module. This is not recommended but nonetheless possible.
I don't understand why nobody has heard of the D programing language?
THIS IS THE PERFECT SOLUTION!!!!
The only real alternative if you desire all those things is to use Java, but honestly you're being a bit picky about features. Is code completion really that important a trait? Everything else you've listed is traditionally very well regarded with Python, so I don't see the issue.
The text editor (not even an IDE) which I use lets you import API function definitions. Code completion is not a language feature, especially with OpenGL. Just type gl[Ctrl+I] and you'd get the options.
I tried using Java3D and java once. I realized Java3D is a typical Java API... lots of objects to do simple things, and because it's Java, that translates to a lot of code. I then moved to Jython in Eclipse to which cleaned up the code, leaving me with only the complexity of Java3D.
So in the end, I went in the opposite direction. One advantage this has over pure python is I can use Java with all of Eclipse's benefits like autocomplete and move it over to python when parts get unwieldy in Java.
It seems like Pydev can offer code completion for you in Eclipse.
I started off doing OpenGL programming with GL4Java, which got migrated to JOGL and you should definately give it (JOGL) a try. Java offers most of the features you require (plus Eclipse gives you the code completion) and especially for JOGL there are a lot of tutorials out there to get you started.
Consider Boo -- it has many of Python's advantages while adopting features from elsewhere as well, and its compile-time type inference (when variables are neither explicitly given a specific type or explicitly duck typed) allows the kind of autocompletion support you're asking about.
The Tao.OpenGL library exposes OpenGL to .NET apps (such as those Boo compiles), with explicit support for Mono.
(Personally, I'm mostly a Python developer when not doing C or Java, but couldn't care less about autocompletion... but hey, it's your question; also, the one-class-per-module convention seems like a ridiculous amount of pain you're putting yourself through needlessly).