I tried to import default_collate from torch.utils.data.dataloader but it gives me default_collate is not exsit and another function is exist (_collate_fn_t)
from torch.utils.data.dataloader import default_collate
from torch.utils.data.dataloader import _collate_fn_t
Are these commands are similar
The _collate_fn_t is the type definition of a collate function in general, it is defined here as:
_collate_fn_t = Callable[[List[T]], Any]
default_collate is the default collate function used by the DataLoader class.
Importing these is not equivalent, you should check your spelling and try again with:
from torch.utils.data.dataloader import default_collate
Related
I am working on topic inference on a new corpus given a previously derived lda model. I am able to load the model perfectly, while I am not able to load the id2word file to create the corpora.Dictionary object needed to map the new corpus into numbers: the load method returns a dict attribute error that I don't know why. Below is the minimal code that replicates the situation, and I have attached the code (and packages used) here.
Thank you in advance for your response...
import numpy as np
import os
import pandas as pd
import gensim
from gensim import corpora
import datetime
import nltk
model_name = "lda_sub_full_35"
dictionary_name = "lda_sub_full_35.id2word"
model_for_inference = gensim.models.LdaModel.load(model_name, mmap='r')
print('Successfully load the model')
lda_dictionary = corpora.Dictionary.load(dictionary_name, mmap='r')
I expect to have both the dictionary and the model loaded, but it turns out that when I load the dictionary, I got the below error:
File "topic_inference.py", line 31, in <module>
lda_dictionary = corpora.Dictionary.load(dictionary_name, mmap='r')
File "/topic_modeling/env/lib/python3.8/site-packages/gensim/utils.py", line 487, in load
obj._load_specials(fname, mmap, compress, subname)
AttributeError: 'dict' object has no attribute '_load_specials'```
How were the contents of the lda_sub_full_35.id2word file originally saved?
Only if it was saved by a Gensim corpora.Dictionary object's .save() method should it be loaded as you've tried, with corpora.Dictionary.load().
If, by any chance, it was just a plain Python dict saved via some other method of writing a pickle()-created object, then you would need to load it in a symmetrically-matched way. That might be as simple as:
import pickle
with open(path, 'rb') as f:
lda_dictionary = pickle.load(f)
I have features and a target variable which I am wanting to generate a Decision Tree. However, the code is throwing an error. Since the 'out file' did not generate an error, I figured there wouldn't be an error for the 'Source.from_file' either, but there is one.
import os
from graphviz import Source
from sklearn.tree import export_graphviz
f = open("C:/Users/julia/Desktop/iris_tree.dot", 'w')
export_graphviz(
tree_clf,
out_file=f,
feature_names=sample2[0:2],
class_names=sample2[5],
rounded=True,
filled=True
)
Source.from_file(f)
As noted in the docs, from_file accepts a string path, not a file object:
filename – Filename for loading/saving the source.
Just pass the path in:
import os
from graphviz import Source
from sklearn.tree import export_graphviz
path = "C:/Users/julia/Desktop/iris_tree.dot"
f = open(path, 'w')
export_graphviz(
tree_clf,
out_file=f,
feature_names=sample2[0:2],
class_names=sample2[5],
rounded=True,
filled=True
)
Source.from_file(path)
Code below works as expected. It prints 5 random numbers.
import numpy as np
class test_class():
def __init__(self):
self.rand_nums = self.create_rand_num()
def create_rand_num(self):
numbers = np.random.rand(5)
return numbers
myclass = test_class()
myclass.rand_nums
However, the following does not work. NameError: name 'np' is not defined
import numpy as np
from test.calc import create_rand_num
class test_class():
def __init__(self):
self.rand_nums = create_rand_num()
myclass = test_class()
myclass.rand_nums
# contents of calc.py in test folder:
def create_rand_num():
print(np.random.rand(5))
But, this works:
from test.calc import create_rand_num
class test_class():
def __init__(self):
self.rand_nums = create_rand_num()
myclass = test_class()
myclass.rand_nums
# contents of calc.py in test folder:
import numpy as np
def create_rand_num():
print(np.random.rand(5))
Why must I have 'import numpy as np' inside calc.py? I already have this import before my class definition. I am sure I am misunderstanding something here, but I was trying to follow the general rule to have all the import statements at the top of the main code.
What I find confusing is that when I say "from test.calc import create_rand_num," how does Python know whether "import numpy as np" is included at the top of calc.py or not? It must know somehow, because when I include it, the code works, but when I leave it out, the code does not work.
EDIT: After reading the response from #DeepSpace, I want to ask the following:
Suppose I have the following file.py module with contents listed as shown:
import numpy as np
import pandas as pd
import x as y
def myfunc():
pass
So, if I have another file, file1.py, and in it, I say from file.py import myfunc, do I get access to np, pd, and y? This is exactly what seems to be happening in my third example above.
In my third example, notice that np is NOT defined anywhere in the main file, it is only defined in calc.py file, and I am not importing * from calc.py, I am only importing create_rand_num. Why do I not get the same NameError error?
Python is not like C. Importing a module does not copy-paste its source. It simply adds a reference to it to the locals() "namespace". import numpy as np in one file does not make it magically available in all other files.
You have to import numpy as np in every file you want to use np.
Perhaps a worthwhile reading: https://docs.python.org/3.7/reference/simple_stmts.html#the-import-statement
I have a probleme using bokeh in a class.
The following code is runing when i use the object "Graph" in the same file (.py), but not when i'm calling the class from an other file, and i don't know why.
class Graph():
import pandas as pd
from bokeh.models import HoverTool
from bokeh.plotting import figure, show, output_file, ColumnDataSource
def __init__(self, df, indicators=None):
self.df = df
self.output_file("test.html" , title='test')
....
I have the following error:
TypeError: output_file() got multiple values for argument 'title'
Does anybody know how can I fix it?
For information, the following code:
class Graph():
import pandas as pd
from bokeh.models import HoverTool
from bokeh.plotting import figure, show, output_file, ColumnDataSource
def __init__(self, df, indicators=None):
self.df = df
self.output_file("test.html")
....
returns:
Traceback (most recent call last):
File "Documents/Programmation/python/Trade/Indicators.py", line 50, in <module>
a = TradeGraph(df)
File "/Users/Alex/Documents/Programmation/python/Graph.py", line 29, in __init__
self.output_file("test.html")
File "/anaconda3/lib/python3.6/site-packages/bokeh/io/output.py", line 77, in output_file
root_dir=root_dir
File "/anaconda3/lib/python3.6/site-packages/bokeh/io/state.py", line 166, in output_file
if os.path.isfile(filename):
File "/anaconda3/lib/python3.6/genericpath.py", line 30, in isfile
st = os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not Graph
and the following code:
class Graph():
import pandas as pd
from bokeh.models import HoverTool
from bokeh.plotting import figure, show, output_file, ColumnDataSource
def __init__(self, df, indicators=None):
self.df = df
self.output_file()
....
returns the same (last) error.
Thanks
You have evidently defined a method output_file on your own class, and that is where the problem is. (As an aside, please always try to ask questions with complete minimal code). Based on the output above, the most likely explanation is that you have forgotten to add the self parameter that Python methods should always have. That is, you have something like:
class Graph(object):
def output_file(title):
when you need something like:
class Graph(object):
def output_file(self, title):
However, I would question the value of having an output_file method at all. Unless you are doing something out of the ordinary, you should just call Bokeh's output_file function directly. However, please also note that output_file activates a persistent implicit mode. That is useful especially in interactive environments, but may not be in a program that saves lots of things. There is also a save function that just gives you explicit control wherever you want to perform a save.
Thank you for you answer,
I finnaly import each need module in each methode.
I thought i could all import them one time beteween the class definition and the class initialisation, but doesn't work,
Thanks
I have a project with a function foo in a module my_project.my_functions. I want to pickle that function in a way that I can unpickle it from somewhere else without requiring to import my_project. foo does not have any side effect, so no dependencies outside the function.
I'm using dill to pickle foo, but dill is saving it as a <function my_project.my_functions.foo>, and complains about the unknown my_project module when I try to unpickle it.
Any solution?
I solved it by recreating the function from the code giving and empty globals dictionary.
In /my_project/module.py:
def f(n):
return n+1
In my_project, before pickling the function:
import dill
import types
import module
f = types.FunctionType(module.f.__code__,{})
with open("my_func.pkl", 'wb') as fs:
dill.dump(f, fs)
Somewhere else:
import dill
with open("my_func.pkl", 'rb') as fs:
f = dill.load(fs)