When is the time_type set to NumericTimeIndex or DatetimeTimeIndex in the entityset? - featuretools

I have problems calculating feature_matrix using cutoff_times table because of a type mismatch in cutoff times in the cutoff table and the time_type of the entityset.
I am trying to understand the predict-next-purchase example using synthetic data.
I got to the point of having cutoff_labels with 'datetime64[ns]' or pandas._libs.tslibs.timestamps.Timestamp type of cutoff time entries.
The dfs procedure gave me an error message:
cutoff_time times must be numeric: try casting via pd.to_numeric(cutoff_time['time'])
I figured that the problems lies in the entity set because the time_type of the entityset is not DatetimeTimeIndex but NumericTimeIndex?!

found out...
I set a "wrong" time index defining an entity from a dataframe. So when one use a numeric variable as a time index then the time_type of the entity is numeric but if you have set the time index using a datetime variable the time_type of the entity is DatetimeTimeIndex!

Related

How truncate time while querying documents for date comparison in Cosmos Db

I have document contains properties like this
{
"id":"1bd13f8f-b56a-48cb-9b49-7fc4d88beeac",
"name":"Sam",
"createdOnDateTime": "2018-07-23T12:47:42.6407069Z"
}
I want to query a document on basis of createdOnDateTime which is stored as string.
query e.g. -
SELECT * FROM c where c.createdOnDateTime>='2018-07-23' AND c.createdOnDateTime<='2018-07-23'
This will return all documents which are created on that day.
I am providing date value from date selector which gives only date without time so, it gives me problem while comparing date.
Is there any way to remove time from createdOnDateTime property or is there any other way to achieve this?
CosmosDB clients are storing timestamps in ISO8601 format and one of the good reasons to do so is that its lexicographical order matches the flow of time. Meaning - you can sort and compare those strings and get them ordered by time they represent.
So in this case you don't need to remove time components just modify the passed in parameters to get the result you need. If you want all entries from entire date of 2018-07-23 then you can use query:
SELECT * FROM c
WHERE c.createdOnDateTime >= '2018-07-23'
AND c.createdOnDateTime < '2018-07-24'
Please note that this query can use a RANGE index on createdOnDateTime.
Please use User Defined Function to implement your requirement, no need to update createdOnDateTime property.
UDF:
function con(date){
var myDate = new Date(date);
var month = myDate.getMonth()+1;
if(month<10){
month = "0"+month;
}
return myDate.getFullYear()+"-"+month+"-"+myDate.getDate();
}
SQL:
SELECT c.id,c.createdOnDateTime FROM c where udf.con(c.createdOnDateTime)>='2018-07-23' AND udf.con(c.createdOnDateTime)<='2018-07-23'
Output :
Hope it helps you.

python3 psycopg SQL identifier must be string

I am trying to reference dynamic tables and fields in a tkinter GUI project using MySQLdb. Using psycopg2.sql to handle an insert statement.
The user select a code, size and color and inputs a quantity. The table names are made up of the size and the code (eg. size-small and code-1111, table_name=small1111). Then the color is the column name and the quantity is an integer entered into the field. The inputs are saved in a dictionary (tdict) when the user selects them. And the dictionary elements are called to be saved in the database table.
table_name = tdict['Size']+tdict['Code']
stmnt = ("INSERT INTO {} (%s, Date) VALUES(%s, %s)").format(sql.Identifier((table_name, tdict['Color'])))
c.execute(sql.SQL(stmnt, (tdict['Quantity'], date)))
The insert query is giving me a TypeError
TypeError("SQL identifiers must be strings")
Can anyone please help? What am I doing wrong? How should the Identifier be made to behave as a string?
Note: I've tried to pass the Identifier elements through a str class but it didn't work. ie
stmnt = ("INSERT INTO {} (%s, Date) VALUES(%s, %s)").format(sql.Identifier((str(table_name, tdict['Color']))))
You are asking sql.Identifier() to create an identifier out of a tuple, e.g. ('small1111','magenta'). Because format() only substitutes into braces {} (and not %s), I think what you actually had in mind was this:
stmnt = sql.SQL("INSERT INTO {} ({}, Date) VALUES(%s %s)").format( sql.Identifier(table_name), sql.Identifier(tdict['Color']) )
I'd suggest you rethink your database design, though --- you should probably have columns named size, code, and color rather than separate tables and columns for each. That will prevent you from having to add a new column each time a new color or a new table for each new size or code. SELECT count(*) FROM inventory WHERE size = 'small' AND code = '1111' GROUP BY color seems preferable to having to create queries dynamically.
This error message will also appear when you have a typo error where you should have used sql.Literal('someFixedNumber'), but instead using sq.Identifier('someFixedNumber')

Azure DocumentDB: order by and filter by DateTime

I have the following query:
SELECT * FROM c
WHERE c.DateTime >= "2017-03-20T10:07:17.9894476+01:00" AND c.DateTime <= "2017-03-22T10:07:17.9904464+01:00"
ORDER BY c.DateTime DESC
So as you can see I have a WHERE condition for a property with the type DateTimeand I want to sort my result by the same one.
The query ends with the following error:
Order-by item requires a range index to be defined on the corresponding index path.
I have absolutely no idea what this error message is about :(
Has anybody any idea?
You can also do one thing that don't require indexing explicitly. Azure documentBD is providing indexing on numbers field by default so you can store the date in long format. Because you are already converting date to string, you can also convert date to long an store, then you can implement range query.
I think I found a possible solution, thanks for pointing out the issue with the index.
As stated in the following article https://learn.microsoft.com/en-us/azure/documentdb/documentdb-working-with-dates#indexing-datetimes-for-range-queries I changed the index for the datatype string to RangeIndex, to allow range queries:
DocumentCollection collection = new DocumentCollection { Id = "orders" };
collection.IndexingPolicy = new IndexingPolicy(new RangeIndex(DataType.String) { Precision = -1 });
await client.CreateDocumentCollectionAsync("/dbs/orderdb", collection);
And it seems to work! If there are any undesired side effects I will let you know.

Using datetime in SharePoint

I have a date time item in a list that I want to interact with. I access the items in the list via SPListItem.
My code:
_relevantDate = (DateTime)_siteInfo["cimKeyDocumentDate"];
What the code means in terms of types etc:
[datetime property] = [datetime cast]SPListItem[field name];
I'm aware of the SPFieldDateTime type but I can't see a type it returns and I am confused at how I should do return the datetime.
Thanks.
Your first code would return datetime type after correct casting.
DateTime myDateTime = DateTime.Parse(myListItem["MyDateField"].ToString());
but if you want to use SPFieldDateTime then you will have to write it like
SPFieldDateTime myDateTime =(SPFieldDateTime)web.Fields[FieldNames.DateFieldName];
This will return a field that contains date and time values.See here.

Gremlin - How to filter while converting property to Integer?

I have the following segment of gremlin code:
vert.as('x').
both.or(
_().has("time").filter{ _()["time"] > startTime.toInteger() },
_().has("isRead"), _().has("isWrite")).dedup{}.gather.scatter.
store(y).loop('x'){c++ < limit.toInteger()}.iterate();
I would think that this would filter the items out whose time attribute value resolve to be greater than startTime. But this isn't the case. How do I get the time of the current object in the pipeline in order to compare it?
Actually, I found the answer really fast. I should have known, being as I've read basically all the gremlin documentation... :/
vert.as('x').
both.or(
_().has("time").filter{ it.time > startTime.toInteger() },
_().has("isRead"), _().has("isWrite")).dedup{}.gather.scatter.
store(y).loop('x'){c++ < limit.toInteger()}.iterate();

Resources