I've tried several perturbations of the following:
datetime.strptime('2018-01-14T23:55:27.337Z',"%Y-%m-%dT%H:%M:%S.%3N%Z")
but get errors like this:
ValueError Traceback (most recent call last)
<ipython-input-45-babd38d0d73f> in <module>()----> 1 datetime.strptime('2018-01-14T23:55:27.337Z',"%Y-%m-%dT%H:%M:%S.%3N%Z")
/usr/lib/python3.5/_strptime.py in _strptime_datetime(cls, data_string, format)
508 """Return a class cls instance based on the input string and the
509 format string."""
--> 510 tt, fraction = _strptime(data_string, format)
511 tzname, gmtoff = tt[-2:]
512 args = tt[:6] + (fraction,)
/usr/lib/python3.5/_strptime.py in _strptime(data_string, format)
333 del err
334 raise ValueError("'%s' is a bad directive in format '%s'" %
--> 335 (bad_directive, format)) from None
336 # IndexError only occurs when the format string is "%"
337 except IndexError:
ValueError: '3' is a bad directive in format '%Y-%m-%dT%H:%M:%S.%3N%Z'
Try doing this.
datetime.strptime('2018-01-14T23:55:27.337Z', '%Y-%m-%dT%H:%M:%S.%fZ')
Take note at the second argument. It's '%Y-%m-%dT%H:%M:%S.%fZ' and not '%Y-%m-%dT%H:%M:%S.%3N%Z'. You are getting a bad directive value error because %3 is not a directive.
In case you need it, here's the documentation for strptime().
Related
I am working on the Jupyter notebook using Python3. I am trying to load an asc file containing 2 columns by using lin_data1 = np.genfromtxt(outdir+"/test_22/mp_harmonic_im_r8.00.ph.asc"), I am getting the following error.
'ValueError Traceback (most recent call last)
<ipython-input-152-5d1a4cbeab20> in <module>
1 # format of the path: SIMULATION-NAME/output-NNNN/PARFILE-NAME
2
----> 3 lin_data1 = np.genfromtxt(outdir+"/test_22/mp_harmonic_im_r8.00.ph.asc")
~/.local/lib/python3.8/site-packages/numpy/lib/npyio.py in genfromtxt(fname, dtype, comments, delimiter, skip_header, skip_footer, converters, missing_values, filling_values, usecols, names, excludelist, deletechars, replace_space, autostrip, case_sensitive, defaultfmt, unpack, usemask, loose, invalid_raise, max_rows, encoding)
2078 # Raise an exception ?
2079 if invalid_raise:
-> 2080 raise ValueError(errmsg)
2081 # Issue a warning ?
2082 else:
ValueError: Some errors were detected !
Line #2 (got 2 columns instead of 3)
Line #3 (got 2 columns instead of 3
I got error when I want to make dataframe after cleaning data! The code is as follows:
data_clean = pd.DataFrame(cleaner_data,columns=['tweet'])
data_clean.head()
and error info :
ValueError Traceback (most recent call last)
<ipython-input-62-1d07a4d30120> in <module>
----> 1 data_clean = pd.DataFrame(cleaner_data,columns=['tweet'])
2 data_clean.head()
~\AppData\Roaming\Python\Python37\site-packages\pandas\core\frame.py in __init__(self, data, index, columns, dtype, copy)
507 )
508 else:
--> 509 raise ValueError("DataFrame constructor not properly called!")
510
511 NDFrame.__init__(self, mgr, fastpath=True)
ValueError: DataFrame constructor not properly called!
I don't know how to solve it. It's said dataframe constructor no properly called.
Do like this:
df_clean = cleaner_data['tweet']
df_clean.head()
How do I pass the output of this prompt to a pandas search by date in excel?
import pandas as pd
TestedDateBegin = pd.to_datetime(input('Input date in mm-dd-yyyy format: '))
For example, if I input 2019-09-08 into above input prompt and run TestedDateBegin I get this output:
Timestamp('2019-09-08 00:00:00')
This search with the date hard coded works fine.
data = df.loc[df['emr_first_access_date'] >= '2019-09-08', ['site_name','subs_num','emr_id', ```'emr_first_access_date']]
But how do I pass the date inputted from the prompt so the user can search by any date?
This doesnt work:
data = df.loc[df['emr_first_access_date'] >= 'TestedDateBegin', ['site_name','subs_num','emr_id',
and throws a exception:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_str_to_tsobject()
pandas/_libs/tslibs/np_datetime.pyx in pandas._libs.tslibs.np_datetime._string_to_dts()
ValueError: Error parsing datetime string "TestedDateBegin" at position 0
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_str_to_tsobject()
pandas/_libs/tslibs/parsing.pyx in pandas._libs.tslibs.parsing.parse_datetime_string()
~\AppData\Local\Continuum\anaconda3\lib\site-packages\dateutil\parser\_parser.py in parse(timestr, parserinfo, **kwargs)
1357 else:
-> 1358 return DEFAULTPARSER.parse(timestr, **kwargs)
1359
~\AppData\Local\Continuum\anaconda3\lib\site-packages\dateutil\parser\_parser.py in parse(self, timestr, default, ignoretz, tzinfos, **kwargs)
648 if res is None:
--> 649 raise ValueError("Unknown string format:", timestr)
650
ValueError: ('Unknown string format:', 'TestedDateBegin')
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in wrapper(self, other)
144 try:
--> 145 other = _to_M8(other, tz=self.tz)
146 except ValueError:
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in _to_M8(key, tz)
77 # this also converts strings
---> 78 key = Timestamp(key)
79 if key.tzinfo is not None and tz is not None:
pandas/_libs/tslibs/timestamps.pyx in pandas._libs.tslibs.timestamps.Timestamp.__new__()
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_to_tsobject()
pandas/_libs/tslibs/conversion.pyx in pandas._libs.tslibs.conversion.convert_str_to_tsobject()
ValueError: could not convert string to Timestamp
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-2-702fd23c14bb> in <module>
----> 1 data = df.loc[df['emr_first_access_date'] >= 'TestedDateBegin', ['site_name','subs_num','emr_id', 'emr_first_access_date']]
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\ops.py in wrapper(self, other, axis)
1714
1715 res_values = dispatch_to_index_op(op, self, other,
-> 1716 pd.DatetimeIndex)
1717
1718 return self._constructor(res_values, index=self.index,
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\ops.py in dispatch_to_index_op(op, left, right, index_class)
1189 left_idx = left_idx._shallow_copy(freq=None)
1190 try:
-> 1191 result = op(left_idx, right)
1192 except NullFrequencyError:
1193 # DatetimeIndex and TimedeltaIndex with freq == None raise ValueError
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\indexes\datetimelike.py in wrapper(self, other)
115 other = other._values
116
--> 117 result = op(self._data, maybe_unwrap_index(other))
118 return result
119
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\arrays\datetimes.py in wrapper(self, other)
146 except ValueError:
147 # string that cannot be parsed to Timestamp
--> 148 return ops.invalid_comparison(self, other, op)
149
150 result = op(self.asi8, other.view('i8'))
~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\ops.py in invalid_comparison(left, right, op)
1056 else:
1057 raise TypeError("Invalid comparison between dtype={dtype} and {typ}"
-> 1058 .format(dtype=left.dtype, typ=type(right).__name__))
1059 return res_values
1060
TypeError: Invalid comparison between dtype=datetime64[ns] and str
The error
TypeError: Invalid comparison between dtype=datetime64[ns] and str
tells that you try to compare datetime with string. To do it convert your string to datetime manually. In your case try:
from datetime import datetime
date = '2019-09-08'
date = datetime.strptime(date, '%Y-%m-%d')
To learn more information about date formatting see documentation
I am trying converting date string format to numeric, but I get some error,
my date column like this :
train['AVERAGE_ACCT_AGE'].head(6)
0 0yrs 0mon
1 1yrs 11mon
2 0yrs 0mon
3 0yrs 8mon
4 0yrs 0mon
5 1yrs 9mon
Name: AVERAGE_ACCT_AGE, dtype: object
I tried this code to add DateTime format to that variable.
train['AVERAGE_ACCT_AGE']=pd.to_datetime(train['AVERAGE.ACCT.AGE'], format='%Y%m')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~\Anaconda3\lib\site-packages\pandas\core\tools\datetimes.py in _convert_listlike(arg, box, format, name, tz)
376 try:
--> 377 values, tz = conversion.datetime_to_datetime64(arg)
378 return DatetimeIndex._simple_new(values, name=name, tz=tz)
pandas\_libs\tslibs\conversion.pyx in pandas._libs.tslibs.conversion.datetime_to_datetime64()
TypeError: Unrecognized value type: <class 'str'>
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-49-13f5c298f460> in <module>()
----> 1 train['AVERAGE_ACCT_AGE']=pd.to_datetime(train['AVERAGE.ACCT.AGE'], format='%Y-%m')
~\Anaconda3\lib\site-packages\pandas\core\tools\datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, box, format, exact, unit, infer_datetime_format, origin, cache)
449 else:
450 from pandas import Series
--> 451 values = _convert_listlike(arg._values, True, format)
452 result = Series(values, index=arg.index, name=arg.name)
453 elif isinstance(arg, (ABCDataFrame, MutableMapping)):
~\Anaconda3\lib\site-packages\pandas\core\tools\datetimes.py in _convert_listlike(arg, box, format, name, tz)
378 return DatetimeIndex._simple_new(values, name=name, tz=tz)
379 except (ValueError, TypeError):
--> 380 raise e
381
382 if arg is None:
~\Anaconda3\lib\site-packages\pandas\core\tools\datetimes.py in _convert_listlike(arg, box, format, name, tz)
366 dayfirst=dayfirst,
367 yearfirst=yearfirst,
--> 368 require_iso8601=require_iso8601
369 )
370
pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()
pandas\_libs\tslib.pyx in pandas._libs.tslib.array_to_datetime()
ValueError: time data 0yrs 0mon doesn't match format specified
After that, I tried this code to added error ignore to the column.
train['AVERAGE_ACCT_AGE']=pd.to_datetime(train['AVERAGE.ACCT.AGE'], format='%Y%m',errors='ignore',infer_datetime_format=True)
Its added datetime format then I this code
train['yrs']=train['AVERAGE_ACCT_AGE'].dt.year
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-50-39b8c6e07f77> in <module>()
----> 1 train['yrs']=train['AVERAGE_ACCT_AGE'].dt.year
~\Anaconda3\lib\site-packages\pandas\core\generic.py in __getattr__(self, name)
4366 if (name in self._internal_names_set or name in self._metadata or
4367 name in self._accessors):
-> 4368 return object.__getattribute__(self, name)
4369 else:
4370 if self._info_axis._can_hold_identifiers_and_holds_name(name):
~\Anaconda3\lib\site-packages\pandas\core\accessor.py in __get__(self, obj, cls)
130 # we're accessing the attribute of the class, i.e., Dataset.geo
131 return self._accessor
--> 132 accessor_obj = self._accessor(obj)
133 # Replace the property with the accessor object. Inspired by:
134 # http://www.pydanny.com/cached-property.html
~\Anaconda3\lib\site-packages\pandas\core\indexes\accessors.py in __new__(cls, data)
323 pass # we raise an attribute error anyway
324
--> 325 raise AttributeError("Can only use .dt accessor with datetimelike "
326 "values")
please help me how to convert object type to numeric type. I want years and months of columns separately.
AttributeError: Can only use .dt accessor with datetimelike values
The column is not of Datetime format.
Here is a quick way to get it to numeric.
I am using more lines than needed.
# doing this so we can have it in string format
train['AVERAGE_ACCT_AGE'] = train['AVERAGE_ACCT_AGE'].astype(str)
#Now remove the trailing or any such spaces
train['AVERAGE_ACCT_AGE'] = train['AVERAGE_ACCT_AGE'].map(lambda x: x.strip())
#Next we split and expand the column into 2 columns:
train[['yrs','months']] = train['AVERAGE_ACCT_AGE'].str.split(' ',n=1,expand=True)
#remove characters from new columns,
#I am assuming the characters remain the same
train['yrs'] = train['yrs'].str.replace('yrs','')
train['months'] = train['months'].str.replace('mon','')
# Convert yrs to float
train['yrs'] = train['yrs'].astype('float')
# Convert months to float
train['months'] = train['yrs'].astype('float')
Hope it helps.
So, I started on a new toy project and decided I'd use Python 3 for the first time...
In [1]: import plistlib
In [2]: with open("/Volumes/Thunderbay/CURRENT/Music/iTunes/iTunes Library.xml") as itl:
library = plistlib.load(itl)
...:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-6459a022cb71> in <module>()
1 with open("/Volumes/Thunderbay/CURRENT/Music/iTunes/iTunes Library.xml") as itl:
----> 2 library = plistlib.load(itl)
3
/usr/local/Cellar/python3/3.4.3_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plistlib.py in load(fp, fmt, use_builtin_types, dict_type)
984 fp.seek(0)
985 for info in _FORMATS.values():
--> 986 if info['detect'](header):
987 P = info['parser']
988 break
/usr/local/Cellar/python3/3.4.3_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plistlib.py in _is_fmt_xml(header)
556
557 for pfx in prefixes:
--> 558 if header.startswith(pfx):
559 return True
560
TypeError: startswith first arg must be str or a tuple of str, not bytes
hmm ok, let's give it a hint:
In [3]: with open("/Volumes/Thunderbay/CURRENT/Music/iTunes/iTunes Library.xml") as itl:
library = plistlib.load(itl, fmt=plistlib.FMT_XML)
...:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-4-ef5f06b44ec2> in <module>()
1 with open("/Volumes/Thunderbay/CURRENT/Music/iTunes/iTunes Library.xml") as itl:
----> 2 library = plistlib.load(itl, fmt=plistlib.FMT_XML)
3
/usr/local/Cellar/python3/3.4.3_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plistlib.py in load(fp, fmt, use_builtin_types, dict_type)
995
996 p = P(use_builtin_types=use_builtin_types, dict_type=dict_type)
--> 997 return p.parse(fp)
998
999
/usr/local/Cellar/python3/3.4.3_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plistlib.py in parse(self, fileobj)
323 self.parser.EndElementHandler = self.handle_end_element
324 self.parser.CharacterDataHandler = self.handle_data
--> 325 self.parser.ParseFile(fileobj)
326 return self.root
327
TypeError: read() did not return a bytes object (type=str)
plistlib is in the standard library, but from the problems above I have the feeling it has not actually been converted to Python 3?
Anyway, my actual question: is it possible to open an XML plist file with plistlib in Python 3.4.3?
surely I'm missing something obvious here perhaps... just noticed the Py2 version of plistlib (which works!) has a different interface, so someone has actually modified the code of the library for inclusion with Py3...
Thanks to #J Presper Eckert for giving me a clue about what to look for...
I then found this article:
http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#the-binary-option
which suggests the answer is simply to open the file in binary mode, tried it and it works!
with open("/Volumes/Thunderbay/CURRENT/Music/iTunes/iTunes Library.xml", 'rb') as itl:
library = plistlib.load(itl)
Got same error message using python 3.4.3 against .plist file ( mac property list, configuration file in xml ).
Try this (worked for me):
copy plist / xml file to new file.
change new file extension to '.txt'
open newfile.txt in textEdit (mac), notepad++(win) or similar
save as .txt file with UTF-8 encoding & plaintext only.
Now, when you read newfile.txt, you won't see "startswith first arg must be str or a tuple of str, not bytes".
Cheers