This might have been answered before, but I could not find anything that addresses my issue.
So, I have 2 files.
|
|-- test.py
|-- test1.py
test1.py is as below
def fnc():
return np.ndarray([1,2,3,4])
I'm trying to call test1 from test and calling the function like
from test1 import *
x = fnc()
Now naturally I'm getting NameError: name 'np' is not defined.
I tried to write the import both in test and test1 as
import numpy as np
But still, I'm getting the error. This might be silly, but what exactly I'm missing?
Any help is appreciated. Thanks in advance.
Each Python module has it's own namespace, so if some functions in test1.py depends on numpy, you have to import numpy in test1.py:
# test1.py
import numpy as np
def fnc():
return np.ndarray([1,2,3,4])
If test.py doesn't directly use numpy, you don't have to import it again, ie:
# test.py
# NB: do NOT use 'from xxx import *' in production code, be explicit
# about what you import
from test1 import fnc
if __name__ == "__main__":
result = fnc()
print(result)
Now if test.py also wants to use numpy, it has to import it too - as I say, each module has it's own namespace:
# test.py
# NB: do NOT use 'from xxx import *' in production code, be explicit
# about what you import
import numpy as np
from test1 import fnc
def other():
return np.ndarray([3, 44, 5])
if __name__ == "__main__":
result1 = fnc()
print(result1)
result2 = other()
print(result2)
Note that if you were testing your code in a python shell, just modifying the source and re-importing it in the python shell will not work (modules are only loaded once per process, subsequent imports fetch the already loaded module from the sys.modules cache), so you have to exit the shell and open a new one.
mostly you need to have __init__.py in the directort where you have these files
just try creating init.py file like below in the directory where you .py files are present and see if it helps.
touch __init__.py
Related
I have a structure like below:
src/py_meta/
handlers/
internals/
__init__.py
main.py
tests/
__init__.py
test_main.py
pyproject.toml
test_main.py
"""Test the main function
"""
import requests
from src.py_meta import main as py_meta
from src.py_meta.internals.metadata import MetaData
image = requests.get(
"https://file-examples.com/storage/fea8fc38fd63bc5c39cf20b/2017/10/file_example_JPG_500kB.jpg",
timeout=6,
)
with open("demo_file.jpg", "wb") as f:
f.write(image.content)
def test_read():
"""Tests for a MetaData object returned."""
assert isinstance(py_meta.read("demo_file.jpg"), MetaData)
I then get a ModuleNotFound error on the import statements
from src.py_meta import main as py_meta
from src.py_meta.internals.metadata import MetaData
# I have tried without the src.
And relevent contents of pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
------------------------------------
[tool.pytest.ini_options]
pythonpath = "src"
addopts = ["--import-mode=importlib"]
[tool.pylint.ini_options]
pythonpath = "src"
Full code is on github: https://github.com/pyscripter99/python-metadata
I have tried pytest documentation
also followed the python guide. I expect the imports to work without any issues from pylint or pytest, along with future usage as a python package
I have the following python script example.py, which I converted to a Windows Executable.
import pandas as pd
import pickle
def example(df1, df2):
print('Started the executable with example')
df1 = pickle.loads(df1)
df2 = pickle.loads(df2)
print('df1 has {df1.shape[0]} rows')
print('df2 has {df2.shape[0]} rows')
return pickle.dumps(pd.concat([df1,df2]))
if __name__=="__main__":
example(sys.argv[1],sys.argv[2])
Then I use PyInstaller to create an executable named example.exe using pyinstaller example.py -F
Next, I have create random Pandas DataFrames in Python, let's call them df1 and df2.
Now, I would like to use the subprocess module in the main Python script main.py to call this executable and get the results. This is the part with which I need help. Following is the code I wrote, but obviously isn't working.
import subprocess
import pickle
df_dump1 = pickle.dumps(df1)
df_dump2 = pickle.dumps(df2
command = ['./example.exe',df_dump1, df_dump2]
result = subprocess.run(command,
input,=df_dump,
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
shell = True)
print(result.returncode, result.stdout, result.stderr)
The error message I get is this:
TypeError: a bytes-like object is required, not \'str\
Clearly, I am not able to send multiple Pandas dataframes (or even one) to the executable. Any ideas about how to achieve this?
Code below works as expected. It prints 5 random numbers.
import numpy as np
class test_class():
def __init__(self):
self.rand_nums = self.create_rand_num()
def create_rand_num(self):
numbers = np.random.rand(5)
return numbers
myclass = test_class()
myclass.rand_nums
However, the following does not work. NameError: name 'np' is not defined
import numpy as np
from test.calc import create_rand_num
class test_class():
def __init__(self):
self.rand_nums = create_rand_num()
myclass = test_class()
myclass.rand_nums
# contents of calc.py in test folder:
def create_rand_num():
print(np.random.rand(5))
But, this works:
from test.calc import create_rand_num
class test_class():
def __init__(self):
self.rand_nums = create_rand_num()
myclass = test_class()
myclass.rand_nums
# contents of calc.py in test folder:
import numpy as np
def create_rand_num():
print(np.random.rand(5))
Why must I have 'import numpy as np' inside calc.py? I already have this import before my class definition. I am sure I am misunderstanding something here, but I was trying to follow the general rule to have all the import statements at the top of the main code.
What I find confusing is that when I say "from test.calc import create_rand_num," how does Python know whether "import numpy as np" is included at the top of calc.py or not? It must know somehow, because when I include it, the code works, but when I leave it out, the code does not work.
EDIT: After reading the response from #DeepSpace, I want to ask the following:
Suppose I have the following file.py module with contents listed as shown:
import numpy as np
import pandas as pd
import x as y
def myfunc():
pass
So, if I have another file, file1.py, and in it, I say from file.py import myfunc, do I get access to np, pd, and y? This is exactly what seems to be happening in my third example above.
In my third example, notice that np is NOT defined anywhere in the main file, it is only defined in calc.py file, and I am not importing * from calc.py, I am only importing create_rand_num. Why do I not get the same NameError error?
Python is not like C. Importing a module does not copy-paste its source. It simply adds a reference to it to the locals() "namespace". import numpy as np in one file does not make it magically available in all other files.
You have to import numpy as np in every file you want to use np.
Perhaps a worthwhile reading: https://docs.python.org/3.7/reference/simple_stmts.html#the-import-statement
I have a few files with tests and an imported file:
# test1.py
print("running test1.py")
import unittest
import imported
class MyTest1(unittest.TestCase):
def test_method1(self):
assert "love" == "".join(["lo", "ve"])
def test_fail1(self):
assert True, "I failed!"
and
# test2.py
print("running test2.py")
import unittest
import imported
class MyTest2(unittest.TestCase):
def test_ok2(self):
self.assertEqual(True, 2==2, "I will not appear!")
def test_fail2(self):
self.assertFalse(False, "Doh!")
and
# imported.py
import sys
# Here I am accessing the module that has imported this file
importing = sys.modules[sys._getframe(6).f_globals['__name__']]
print(importing.__name__ + " imported this file")
When I run pytest -s test1.py test2.py, I get this result:
=============================== test session starts ================================
platform darwin -- Python 3.6.0, pytest-3.7.1, py-1.5.4, pluggy-0.7.1
rootdir: /Users/ibodi/MyDocs/progs/Python/unittest_test, inifile:
collecting 0 items running test1.py
test1 imported this file
collecting 2 items running test2.py
collected 4 items
test1.py ..
test2.py ..
============================= 4 passed in 0.08 seconds =============================
importing variable in imported.py corresponds to the module that has imported the imported.py module.
And the problem is that the line import imported in test2.py doesn't cause test2 imported this file to appear in the logs. Why is that?
I am trying to put together a unittest to test whether my function that reads in big data files, produces the correct result in shape of an numpy array. However, these files and arrays are huge and can not be typed in. I believe I need to save input and output files and test using them. This is how my testModule looks like:
import numpy as np
from myFunctions import fun1
import unittest
class TestMyFunctions(unittest.TestCase):
def setUp(self):
self.inputFile1 = "input1.txt"
self.inputFile2 = "input2.txt"
self.outputFile = "output.txt"
def test_fun1(self):
m1 = np.genfromtxt(self.inputFile1)
m2 = np.genfromtxt(self.inputFile2)
R = np.genfromtxt(self.outputFile)
self.assertEqual(fun1(m1,m2),R)
if __name__ =='__main__':
unittest.main(exit=False)
I'm not sure if there is a better/neater way of testing huge results.
Edit:
Also getting an attribute error now:
AttributeError: TestMyFunctions object has no attribute '_testMethodName'
Update - AttributeError Solved - 'def init()' is not allowed. Changed with def setUp()!