Local directory shadowing 3rd party package - python-3.x

Question

I'm trying to figure out if this is an error in my design, or an error in the redis-py library. Essentially, my understanding of namespace in Python is that packages should be designed such that all components are under the package namespace namespace. Meaning, if I have a queue in packageA and a queue in packageB, there should be no collision since they are namespaced (packageA.queue and packageB.queue). However, I'm running into an error in a package I am building.
This is the directory structure for the package I am building:
○ → tree
.
├── __init__.py
├── net
│ ├── __init__.py
│ ├── rconn.py
└── test.py
The __init__.py files are all empty. Here's the code of my test.py file:
○ → cat test.py
from net import rconn
and here's the code from my net/rconn.py file:
○ → cat net/rconn.py
import redis
Running test.py, everything works, no errors. However, if I add a queue directory in here and create an empty init.py within, here's the new tree:
○ → tree
.
├── __init__.py
├── net
│ ├── __init__.py
│ ├── rconn.py
├── queue
│ ├── __init__.py
└── test.py
Running test.py results in the following error:
Traceback (most recent call last):
File "test.py", line 1, in <module>
from net.rconn import ass
File "/Users/yosoyunmaricon/python_test/net/rconn.py", line 1, in <module>
import redis
File "/usr/local/lib/python3.7/site-packages/redis/__init__.py", line 1, in <module>
from redis.client import Redis, StrictRedis
File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 10, in <module>
from redis._compat import (b, basestring, bytes, imap, iteritems, iterkeys,
File "/usr/local/lib/python3.7/site-packages/redis/_compat.py", line 139, in <module>
from queue import Queue
ImportError: cannot import name 'Queue' from 'queue' (/Users/yosoyunmaricon/python_test/queue/__init__.py)
So, I get what's happening. The Redis code says from queue import Queue, and when I create an empty queue directory (i.e., no Queue), it breaks the package. My question is this: Is that good design? Should the Redis package be more explicit and say something along the lines of from redis.queue import Queue, or is this simply an error in my own design?

Answer 1

It's not the Redis package that should adjust here, because it cannot know or cannot handle the different ways users could integrate the Redis package into their own applications, like how you have a similarly named queue package. Furthermore, there is no redis.queue because that queue, is not part of redis, but the built-in Python queue package. You can go to /usr/local/lib/python3.7/site-packages/redis/_compat.py and print out the queue.__file__, which would give you the path to Python's queue. It expects importing the built-in queue package.
Unfortunately for you, when Python builds the module search paths for resolving imports, it builds it in the following order:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
...which puts your own queue at the start of the list and that's what gets imported. So, yes, getting an ImportError because you shadowed the built-in queue is more of an error in your own design.
You could probably do some tricks here with sys.path or PYTHONPATH, but why bother when you can just rename your queue to something else. Or, what I usually do is to group my own packages into a parent folder, named after the acronym for the project ("abcdlibs") or some app identier or something like "mylibs":
.
├── __init__.py
├── mylibs
│ └── queue
│ ├── __init__.py
├── mynet
│ ├── __init__.py
│ └── rconn.py
└── test.py
That way, you could make it clear that mylibs.queue is different from queue.