I have data in excel in form of timestamps s it looks like
30/11/12 12:42 AM
30/11/12 12:47 AM
30/11/12 12:56 AM
30/11/12 1:01 AM
I need to get it to matlab to look like this
dateStrings = {...
'30/11/12 12:42 AM' ...
'30/11/12 12:47 AM' ...
'30/11/12 12:56 AM' ...
'30/11/12 1:01 AM' ...
};
I have tried xlsread but it doesn't put in the strings.
The following works for me (in Octave, but should be the same in MATLAB):
>> [num,txt,raw]=xlsread('dates.xls','A1:A4')
num =
4.1243e+004
4.1243e+004
4.1243e+004
4.1243e+004
txt = {}(0x0)
raw =
{
[1,1] = 4.1243e+004
[2,1] = 4.1243e+004
[3,1] = 4.1243e+004
[4,1] = 4.1243e+004
}
>> datestr(num+datenum(1900,1,1,0,0,0)-2)
ans =
30-Nov-2012 00:42:00
30-Nov-2012 00:47:00
30-Nov-2012 00:56:00
30-Nov-2012 01:01:00
>> whos ans
Variables in the current scope:
Attr Name Size Bytes Class
==== ==== ==== ===== =====
ans 4x20 80 char
Total is 80 elements using 80 bytes
Check out the datestr function for the various output format options.
Arnaud
I manage to find a way how to solve it
1. Copy and paste your dates into Excel in dd-mm-yyyy format
2. In Excel, highlight the data and go Right Click, Format Cells/Number
3. In Matlab go a=xlsread(xlsfile);
4. Type datestr(a+693960)
Related
I want to import this kind of csv into Excel
Work Item Type,ID,State,Date Request,Created Date
"Task","4533","Closed","2-9-2020 14:26:00","3-9-2020 08:17:39"
"Task","4535","Closed","3-9-2020 12:26:44","3-9-2020 12:29:33"
"Task","4577","Closed","3-9-2020 15:56:00","4-9-2020 09:12:21"
"Task","4580","New","17-8-2020 09:47:00","4-9-2020 09:49:39"
"Task","4581","Resolved","28-8-2020 10:22:00","4-9-2020 10:24:46"
"Task","4582","Resolved","24-8-2020 10:05:00","4-9-2020 10:31:12"
"Task","4604","Resolved","8-9-2020 08:06:58","8-9-2020 08:07:23"
"Task","4605","Resolved","8-9-2020 09:18:32","8-9-2020 09:18:58"
All dates in this example must be seen with a format day-month-year hour:minute:second
I do the import like this:
Import-Csv -Path '.\Issues.csv' | ForEach-Object {
$sheet1.Cells.Item(1,1) = 'ID'
$sheet1.Cells.Item(1,2) = 'Status'
$sheet1.Cells.Item(1,3) = 'Date Request'
$sheet1.Cells.Item(1,4) = 'Date Created'
$DateRequest = ([datetime]::ParseExact(($($_."Date Request")),$fmtDate,$inv).ToString($fmtDate))
$sheet1.Cells.Item($row,1) = $($_.ID)
$sheet1.Cells.Item($row,2) = $($_.State)
$sheet1.Cells.Item($row,3) = $($_."Date Request")
$sheet1.Cells.Item($row,4) = $($_."Created Date")
$row = $row + 1
}
The result of my Import
ID Status Date Request Date Created
4533 Closed 9/02/2020 14:26 9/03/2020 8:17
4535 Closed 9/03/2020 12:26 9/03/2020 12:29
4577 Closed 9/03/2020 15:56 9/04/2020 9:12
4580 New 17-8-2020 09:47:00 9/04/2020 9:49
4581 Resolved 28-8-2020 10:22:00 9/04/2020 10:24
4582 Resolved 24-8-2020 10:05:00 9/04/2020 10:31
4604 Resolved 9/08/2020 8:06 9/08/2020 8:07
4605 Resolved 9/08/2020 9:18 9/08/2020 9:18
As you can see, some dates are red in the CSV with a month-day-year format,
other are red with a day-month-year format.
The date 3 september has become 9 march
I have tried using CultureInfo, but without any succes.
$inv = [System.Globalization.CultureInfo]::InvariantCulture<br>
$fmtDate = "dd/MM/YYYY HH:mm:ss"
$DateRequest = ([datetime]::ParseExact(($($_."Date Request")),$fmtDate,$inv).ToString($fmtDate))
Does anyone hove any suggestions to solve this?
First of all, the dates in your CSV file have this format d-M-yyyy HH:mm:ss (yyyy is in lowercase and the days and months in the fields do not have a leading zeroes).
Try
$fmtDate = "d-M-yyyy HH:mm:ss"
What puzzles me is why you want to first parse the date in the csv and then use ToString() to reformat it in the exact same string format.
Take off the .ToString($fmtDate) as in
$DateRequest = [datetime]::ParseExact($_."Date Request",$fmtDate, $inv)
and feed that DateTime object into the Excel cell
dd/MM/YYYY HH:mm:ss does NOT describe the input date format you have - dd and MM are for day and month numbers with leading zeros.
Use:
$fmtDateInput = 'd-M-yyyy HH:mm:ss'
$fmtDateOutput = "dd/MM/yyyy HH:mm:ss"
[datetime]::ParseExact($dateString, $fmtDateInput, $culture).ToString($fmtDateOutput)
I'm unable to print the result of re.findall in an Excel cell, which is the result of another re.findall which is captured from a text file with multi-line data.
Below is the section of code from the entire code where I am facing a problem.
import openpyxl,os,re # -- Import
pyro_asr920_dir = '{}\\'.format(os.getcwd()) # -- MARK DIRECTORIES
input_asr920_robot_pid_wb = openpyxl.load_workbook('Data_Workbook.xlsm', keep_vba = True) # -- OPEN EXCEL
input_asr920_robot_pid_tpd_sheet = input_asr920_robot_pid_wb['Topology-Ports_Details']
wduplinka = open(os.path.join(pyro_asr920_dir, 'DELSNACAG01C7606-logs.txt'),'r') # -- OPEN MULTILINE TEXT FILE
uplinkacontent = wduplinka.read()
PreBBa = re.findall( r'{}[\s\S]*?!\ni'.format('interface TenGigabitEthernet4/2'), uplinkacontent) # -- GET REQUIRED SUBSTRING WITH MATCH CRITERIA IN BETWEEN
print(PreBBa)
output01 = '''
['interface TenGigabitEthernet4/2\n isis metric 10 level-2\n!\ni']'''
for line in PreBBa: # - 01 > I CAN PRINT THIS ON EXCEL CELL
input_asr920_robot_pid_tpd_sheet['H27'] = line[:-1]
print(line[:-1])
print('-----------')
output02 = '''
interface TenGigabitEthernet4/2
isis metric 10 level-2
!'''
# ----------------------------------------------------------------- UNABLE TO GET VALUES IN CELL
for line in PreBBa: # - 02 > I CAN'T PRINT THIS ON EXCEL CELL {THIS IS WHERE I AM STUCK}
if 'ospf' in line:
theOSPF = re.findall(r'{}[\s\S]*?{}'.format(' ip ospf','\n c'), line)
input_asr920_robot_pid_tpd_sheet['C47'] = 'Yes'
else:
input_asr920_robot_pid_tpd_sheet['C47'] = 'No' # UNABLE TO GETRESULT IN EXCEL
output03 = '''No'''
# -----------------------------------------------------------------
metric = re.findall(r'{}[\s\S]*?{}'.format('metric ',' '), str(line))
metric = re.findall(r'\d+',str(metric))
input_asr920_robot_pid_tpd_sheet['C46'].value = metric[0] # UNABLE TO GETRESULT IN EXCEL
print(metric)
output04 = '''10'''
# -----------------------------------------------------------------
input_asr920_robot_pid_wb.save('Data_Workbook.xlsm')
wduplinka.close()
input_asr920_robot_pid_wb.close()
print('TEST COMPLETED')
Some content of Text file is as below:
DELSNACAG01C7606#sh running-config
Load for five secs: 18%/1%; one minute: 26%; five minutes: 26%
Time source is NTP, 13:43:23.718 IST Fri Aug 16 2019
Building configuration...
Current configuration : 228452 bytes
!
! Last configuration change at 21:15:56 IST Thu Aug 15 2019
! NVRAM config last updated at 06:42:52 IST Sat Aug 10 2019 by cor382499
!
interface TenGigabitEthernet4/2
isis metric 10 level-2
!
interface TenGigabitEthernet4/3
!
end
Is there a simple way to convert a start and end time input into a list of evenly separated times? the input can be string or integer with format 1000,"1000",or "10:00" in 2400hr format. I've managed to accomplish this in a messy looking way, is there a tighter more efficient way to create this list? As you'll notice I created an array first and then called .tolist() to make the time transformation iteration easier. The problem is that an input of 1030 or 1015 would need to be translated into 1050 or 1025 to create the right spacing but if there were a way I could call a datetime.timedelta or something and cleanly make the array?
start="1000"
end="1600"
total_minutes=(int(end[:2])*60)+int(end[2:])-(int(start[:2])*60)-
int(start[2:])
dog=list(range(0,int(total_minutes),25))
walk=dog_df["Walk Length"][dog_df.index[dog_df["Name"]==self.name][0]]
if walk=='half':
self.dogarr=np.array([(x-25,x,x+25,x+50) for x in dog])
elif walk=='full':
self.dogarr=np.array([(x-25,x,x+25,x+50,x+75,x+100) for x in dog])
else:
self.dogarr=np.array([(x,x+25,x+50) for x in dog])
if int(start[2])!=0:
start=start[:2]+str(int(int(start[2:])*1.667))
self.dogarr+=(int(start))
self.dogarr=self.dogarr.tolist()
z=0
while z<len(self.dogarr):
for timespot in self.dogarr[z].copy():
self.dogarr[z][self.dogarr[z].index(timespot)]=time.strftime('%H%M', time.gmtime(self.dogarr[z][self.dogarr[z].index(timespot)]*36))
z+=1
self.dogarr=np.array(self.dogarr)```
array([['1115', '1130', '1145', '1200'],
['1130', '1145', '1200', '1215'],
['1145', '1200', '1215', '1230'],
['1200', '1215', '1230', '1245'],
['1215', '1230', '1245', '1300']], dtype='<U4')
I'm sure you can figure out to parse times from any number of existing questions. The crux of your question seems to be how to create evenly separated times within a range. Here's a simple way:
start = datetime.datetime(2018,12,20,10) # or use strptime etc.
end = datetime.datetime(2018,12,24,18)
count = 10
interval = (end - start) / count
dt = start
while dt <= end:
print(dt)
dt += interval
The output is:
2018-12-20 10:00:00
2018-12-20 20:24:00
2018-12-21 06:48:00
2018-12-21 17:12:00
2018-12-22 03:36:00
2018-12-22 14:00:00
2018-12-23 00:24:00
2018-12-23 10:48:00
2018-12-23 21:12:00
2018-12-24 07:36:00
2018-12-24 18:00:00
I can't find a way to identify the date formats of a string in MATLAB and put all of them in the same format. I have the following cell array:
list = {'01-Sep-1882'; ...
'01-Aug-1895'; ...
'04/01/1912'; ...
'Tue, 05/28/46'; ...
'Tue, 03/10/53'; ...
'06/20/58'; ...
'Thu, 09/20/73'; ...
'Fri, 08/15/75'; ...
'Sun, 12/01/1996'};
If I do datenum(list) there's an error message because all the rows don't have the same date format. Can you think of a way to circumvent this?
You can do this by successively applying datetime to convert each format and isnat to identify those dates that didn't convert properly. In addition, you can specify days of the week in the date format string and what pivot year to use for years with only the last two numbers. Starting with the sample data and expected date formats in your question, here's the code to do it:
% Input:
list = {'01-Sep-1882'; ...
'01-Aug-1895'; ...
'04/01/1912'; ...
'Tue, 05/28/46'; ...
'Tue, 03/10/53'; ...
'06/20/58'; ...
'Thu, 09/20/73'; ...
'Fri, 08/15/75'; ...
'Sun, 12/01/1996'};
% Conversion code:
dt = datetime(list, 'Format', 'dd-MMM-yyyy');
index = isnat(dt);
dt(index) = datetime(list(index), 'Format', 'MM/dd/yy', 'PivotYear', 1900);
index = isnat(dt);
dt(index) = datetime(list(index), 'Format', 'eee, MM/dd/yy', 'PivotYear', 1900)
% Output:
dt =
9×1 datetime array
01-Sep-1882
01-Aug-1895
01-Apr-1912
28-May-1946
10-Mar-1953
20-Jun-1958
20-Sep-1973
15-Aug-1975
01-Dec-1996
Now you can convert these to numeric values with datenum:
dnum = datenum(dt);
Part of the issue here is that MATLAB's datenum cannot understand this format: Tue, 05/28/46. So let's clean the original list so that it can understand.
% original list
list = {'01-Sep-1882','01-Aug-1895','04/01/1912','Tue, 05/28/46','Tue, 03/10/53','06/20/58','Thu, 09/20/73','Fri, 08/15/75','Sun, 12/01/1996'}
% split each cell in the list
list_split = cellfun(#(x) strsplit(x,' '), list, 'UniformOutput', false);
% now detect where there is unusual format, this will give logical array
abnormal_idx = cellfun(#(x) length(x) == 2, list_split,'UniformOutput', true)
% make copy
clean_list = list;
% now at abnormal indices retain only the part that MATLAB understands
clean_list(abnormal_idx) = cellfun(#(x) x{2}, list_split(abnormal_idx), 'UniformOutput', false);
% now run datenum on clean list
date_num = cellfun(#datenum, clean_list, 'UniformOutput', true);
I have a list that has several days in it. Each day have several timestamps. What I want to do is to make a new list that only takes the start time and the end time in the list for each date.
I also want to delete the Character between the date and the time on each one, the char is always the same type of letter.
the time stamps can vary in how many they are on each date.
Since I'm new to python it would be preferred to use a lot of simple to understand codes. I've been using a lot of regex so pleas if there is a way with this one.
the list has been sorted with the command list.sort() so it's in the correct order.
code used to extract the information was the following.
file1 = open("test.txt", "r")
for f in file1:
list1 += re.findall('20\d\d-\d\d-\d\dA\d\d\:\d\d', f)
listX = (len(list1))
list2 = list1[0:listX - 2]
list2.sort()
here is a list of how it looks:
2015-12-28A09:30
2015-12-28A09:30
2015-12-28A09:35
2015-12-28A09:35
2015-12-28A12:00
2015-12-28A12:00
2015-12-28A12:15
2015-12-28A12:15
2015-12-28A14:30
2015-12-28A14:30
2015-12-28A15:15
2015-12-28A15:15
2015-12-28A16:45
2015-12-28A16:45
2015-12-28A17:00
2015-12-28A17:00
2015-12-28A18:15
2015-12-28A18:15
2015-12-29A08:30
2015-12-29A08:30
2015-12-29A08:35
2015-12-29A08:35
2015-12-29A10:45
2015-12-29A10:45
2015-12-29A11:00
2015-12-29A11:00
2015-12-29A13:15
2015-12-29A13:15
2015-12-29A14:00
2015-12-29A14:00
2015-12-29A15:30
2015-12-29A15:30
2015-12-29A15:45
2015-12-29A15:45
2015-12-29A17:15
2015-12-29A17:15
2015-12-30A08:30
2015-12-30A08:30
2015-12-30A08:35
2015-12-30A08:35
2015-12-30A10:45
2015-12-30A10:45
2015-12-30A11:00
2015-12-30A11:00
2015-12-30A13:00
2015-12-30A13:00
2015-12-30A13:45
2015-12-30A13:45
2015-12-30A15:15
2015-12-30A15:15
2015-12-30A15:30
2015-12-30A15:30
2015-12-30A17:15
2015-12-30A17:15
And this is how I want it to look like:
2015-12-28 09:30
2015-12-28 18:15
2015-12-29 08:30
2015-12-29 17:15
2015-12-30 08:30
2015-12-30 17:15
First of all, you should convert all your strings into proper dates, Python can work with. That way, you have a lot more control on it, also to change the formatting later. So let’s parse your dates using datetime.strptime in list2:
from datetime import datetime
dates = [datetime.strptime(item, '%Y-%m-%dA%H:%M') for item in list2]
This creates a new list dates that contains all your dates from list2 but as parsed datetime object.
Now, since you want to get the first and the last date of each day, we somehow have to group your dates by the date component. There are various ways to do that. I’ll be using itertools.groupby for it, with a key function that just looks at the date component of each entry:
from itertools import groupby
for day, times in groupby(dates, lambda x: x.date()):
first, *mid, last = times
print(first)
print(last)
If we run this, we already get your output (without date formatting):
2015-12-28 09:30:00
2015-12-28 18:15:00
2015-12-29 08:30:00
2015-12-29 17:15:00
2015-12-30 08:30:00
2015-12-30 17:15:00
Of course, you can also collect that first and last date in a list first to process the dates later:
filteredDates = []
for day, times in groupby(dates, lambda x: x.date()):
first, *mid, last = times
filteredDates.append(first)
filteredDates.append(last)
And you can also output your dates with a different format using datetime.strftime:
for date in filteredDates:
print(date.strftime('%Y-%m-%d %H:%M'))
That would give us the following output:
2015-12-28 09:30
2015-12-28 18:15
2015-12-29 08:30
2015-12-29 17:15
2015-12-30 08:30
2015-12-30 17:15
If you don’t want to go the route through parsing those dates, of course you could also do this simply by working on the strings. Since they are nicely formatted (i.e. they can be easily compared), you can do that as well. It would look like this then:
for day, times in groupby(list2, lambda x: x[:10]):
first, *mid, last = times
print(first)
print(last)
Producing the following output:
2015-12-28A09:30
2015-12-28A18:15
2015-12-29A08:30
2015-12-29A17:15
2015-12-30A08:30
2015-12-30A17:15
Because your data is ordered you just need to pull the first and last value from each group, you can use re.sub to remove the single letter replacing it with a space then split each date string just comparing the dates:
from re import sub
def grp(l):
it = iter(l)
prev = start = next(it).replace("A"," ")
for dte in it:
dte = dte.replace("A"," ")
# if we have a new date, yield that start and end
if dte.split(None, 1)[0] != prev.split(None,1)[0]:
yield start
yield prev
start = dte
prev = dte
yield start, prev
l=["2015-12-28A09:30", "2015-12-28A09:30", .....................
l[:] = grp(l)
This could also certainly be done as your process the file without sorting by using a dict to group:
from re import findall
from collections import OrderedDict
with open("dates.txt") as f:
od = defaultdict(lambda: {"min": "null", "max": ""})
for line in f:
for dte in findall('20\d\d-\d\d-\d\dA\d\d\:\d\d', line):
dte, tme = dte.split("A")
_dte = "{} {}".format(dte, tme)
if od[dte]["min"] > _dte:
od[dte]["min"] = _dte
if od[dte]["max"] < _dte:
od[dte]["max"] = _dt
print(list(od.values()))
Which will give you the start and end time for each date.
[{'min': '2016-01-03 23:59', 'max': '2016-01-03 23:59'},
{'min': '2015-12-28 00:00', 'max': '2015-12-28 18:15'},
{'min': '2015-12-30 08:30', 'max': '2015-12-30 17:15'},
{'min': '2015-12-29 08:30', 'max': '2015-12-29 17:15'},
{'min': '2015-12-15 08:41', 'max': '2015-12-15 08:41'}]
The start for 2015-12-28 is also 00:00 not 9:30.
if you dates are actually as posted one per line you don't need a regex either:
from collections import defaultdict
with open("dates.txt") as f:
od = defaultdict(lambda: {"min": "null", "max": ""})
for line in f:
dte, tme = line.rstrip().split("A")
_dte = "{} {}".format(dte, tme)
if od[dte]["min"] > _dte:
od[dte]["min"] = _dte
if od[dte]["max"] < _dte:
od[dte]["max"] = _dte
print(list(od.values()
Which would give you the same output.