Something like this:
timestamp > now - and timestamp < now +
So if the message is not in the future or too much in the past.
I tried to do this with pipelines, but I couldn't get a timestamp and add there a few seconds and couldn't check the time against the timestamp in the message. The idea with pipelines was to check and write a new field if the time isn't right and create a alarm that searches for this field
Try this pipeline rule, which insert new field if timestamp is lower or higher than 1 hour from server's time:
rule "check wrong timestamp"
when
parse_date(value: to_string($message.timestamp), pattern: "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'") - seconds(3600) > now() ||
parse_date(value: to_string($message.timestamp), pattern: "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'") + seconds(3600) < now()
then
//set_field("timestamp", now()); // uncomment if want to fix timestamp
set_field("timestamp_problem", "true");
end
Related
I'm having problems with writing a simple KQL query in Log Analytics Workspace. Basically I want to filter out all Application Registrations that have secret expiry date longer than 3 years.
I have made a following query, but I think my custom made datetime is still considered a string, instead of datetime type.
Here is the query:
//create a new date which is 3 years later than today
let newyear = format_datetime(datetime_add('year',3,datetime(now)), 'M/d/y, h:m:s.fff tt');
//Query the AppRegs for SecretEndDate ending at least in 2025
AppRegExp_CL
|project AppRegName_s, AppID_g, SecretEndDate_t
|where SecretEndDate_t >= datetime(newyear)
I'm always getting the same error:
Query could not be parsed at 'datetime(newyear)'on line [6,0]
I've tried using makedatetime(), to_datetime() but the result was the same.
Normally I'm not working on KQL queries, so maybe I'm missing something very obvious. I hope that you can help me out. Thanks
Since SecretEndDate_t is of datetime type, no conversion is needed.
//create a new date which is 3 years later than today
let newyear = datetime_add('year', 3, now());
//Query the AppRegs for SecretEndDate ending at least in 2025
AppRegExp_CL
|project AppRegName_s, AppID_g, SecretEndDate_t
|where SecretEndDate_t >= newyear
I want to find events that are a on now and upcoming (next 30 days) but that are also not in the past.
When i run this as a cloud function, I get "Cannot have inequality filters on multiple properties". How am I meant to get this type of data.
(ignore the fact that the date stuff is a bit messy, am still playing around).
// Create date 30 days in future
const searchData: Date = new Date();
searchData.setDate(searchData.getDate() + 30);
// Load data and handle empty respoonse
const response: admin.firestore.QuerySnapshot = await admin
.firestore()
.collection(Collections.EVENTS)
.where("startDate", "<=", admin.firestore.Timestamp.fromMillis(searchData.valueOf()))
.where("endDate", ">=", admin.firestore.Timestamp.fromMillis(new Date().valueOf()))
.where("public", "==", true)
.limit(NUMBER_OF_EVENTS)
.get();
Edit:
I would like to know the data structure/query method that will allow me to return all events in the events collection that are on now or that will start in the next month. Additionally, I would like the query to exclude events that have already finished. Each document (event) has a startDate and endDate timestamp on it.
Since you have two fields to check with ranges, I'm not sure this is doable with a single query. What you can do instead is perform two queries, merge the results on the client, and perform a final filter to get the exact matches.
Make an assumption about the maximum duration for an event. Call that amount of time "X".
Query for all documents where startDate is greater than now - X, and also less than now + 30 days. Call this result set "A".
Query for all documents where endDate is greater than now, and also less than now + 30 days. Call this result set "B".
On the client, iterate all the results from A and B, checking to see if the start and end dates fit the criteria you want.
I can't think of a way to structure your data that will do this with a single query.
I know this is kind a old thread, but the answer might be good for others.
What you can do as I at the end ended up doing, is that you have a $start_date and a $target_date.
You then do like this:
<?php
$start = strtotime('2021-11-22');
$target = strtotime('2022-01-01 0:00:00');
$limit = (($target - $start) / 86400);
$query = $col_data->where('date.day_start', '>=', $start);
$query = $query->limit($limit);
?>
Not bad, eh? You welcome!
From the docs:
You can only perform range comparisons (<, <=, >, >=) on a single field, and you can include at most one array-contains or array-contains-any clause in a compound query:
citiesRef.where("state", ">=", "CA").where("state", "<=", "IN");
citiesRef.where("state", "==", "CA").where("population", ">", 1000000);
https://firebase.google.com/docs/firestore/query-data/queries
I have document contains properties like this
{
"id":"1bd13f8f-b56a-48cb-9b49-7fc4d88beeac",
"name":"Sam",
"createdOnDateTime": "2018-07-23T12:47:42.6407069Z"
}
I want to query a document on basis of createdOnDateTime which is stored as string.
query e.g. -
SELECT * FROM c where c.createdOnDateTime>='2018-07-23' AND c.createdOnDateTime<='2018-07-23'
This will return all documents which are created on that day.
I am providing date value from date selector which gives only date without time so, it gives me problem while comparing date.
Is there any way to remove time from createdOnDateTime property or is there any other way to achieve this?
CosmosDB clients are storing timestamps in ISO8601 format and one of the good reasons to do so is that its lexicographical order matches the flow of time. Meaning - you can sort and compare those strings and get them ordered by time they represent.
So in this case you don't need to remove time components just modify the passed in parameters to get the result you need. If you want all entries from entire date of 2018-07-23 then you can use query:
SELECT * FROM c
WHERE c.createdOnDateTime >= '2018-07-23'
AND c.createdOnDateTime < '2018-07-24'
Please note that this query can use a RANGE index on createdOnDateTime.
Please use User Defined Function to implement your requirement, no need to update createdOnDateTime property.
UDF：
function con(date){
var myDate = new Date(date);
var month = myDate.getMonth()+1;
if(month<10){
month = "0"+month;
}
return myDate.getFullYear()+"-"+month+"-"+myDate.getDate();
}
SQL：
SELECT c.id,c.createdOnDateTime FROM c where udf.con(c.createdOnDateTime)>='2018-07-23' AND udf.con(c.createdOnDateTime)<='2018-07-23'
Output :
Hope it helps you.
I am having a hard time getting a statement to render in SS2, I had found an answer is SS1 and hadn't used it sometime back but it doesn't seem to work in ss2. I am using the SS2 render.statement method but I am getting an "error.SuiteScriptError","name":"UNEXPECTED_ERROR","message":null,"stack":["renderStatement(N/render.js)" I am unclear as to what some of the arguments actually are. the entityid I think is the customer id, the printMode is the pdf enum, the formId I have no idea, it says it wants a number but I don't see internal ids next to the forms in the list like other objects in NetSuite, I have the InternalIds on. StartDate is a date before you want to start pulling in transactions from? Statement date? Todays Date? OpentransactionOnly true. Any help with this is appreciated.
thanks
Many of the parameters in this API call mimic the UI options on the 'Print Individual Statement' page. I've copied some of the field level help there to help explain each setting.
entityId - internal ID of the customer. NOTE: I have had errors in the past which were solved by using parseInt(customer).
printMode - the print mode to render this statement in (eitherrender.PrintMode.HTML or render.PrintMode.PDF)
formId - internal ID of the form to use to print the statement. Leave this option off to use the default statement form.
startDate - If you choose to enter a date, this is the date of the oldest transaction that appears on the statement. If you choose to note enter a date, all transactions in the customer's history appear on the statement.
statementDate - Statement date. The date in this field is the date used to calculate aging and the date that appears on the statement form.
openTransactionsOnly - Select this option to include only open transactions on statements. If you have entered a start date, open transactions from that start date appear. If you have not entered a start date, all open transactions appear. The Show Only Open Transaction option is most useful for statements printed as of your current today date. If you select Show Only Open Transactions and are using a date other that today’s date, you may have balance discrepancies.
Here is an code snippet. It uses the moment.js for calculating the dates of the first and last day of the previous month.
var startDate = format.format({
value: moment().subtract(1, 'months').startOf('month').toDate(),
type: format.Type.DATE
});
var statementDate = format.format({
value: moment().subtract(1, 'months').endOf('month').toDate(),
type: format.Type.DATE
});
var statement = render.statement({
entityId: parseInt(customer),
printMode: render.PrintMode.PDF,
startDate: startDate,
statementDate: statementDate
});
I am trying to query the WadPerformanceCountersTable generated by Azure Diagnostics which has a PartitionKey based on tick marks accurate up to the minute. This PartitionKey is stored as a string (which I do not have any control over).
I want to be able to query against this table to get data points for every minute, every hour, every day, etc. so I don't have to pull all of the data (I just want a sampling to approximate it). I was hoping to using the modulus operator to do this, but since the PartitionKey is stored as a string and this is an Azure Table, I am having issues.
Is there any way to do this?
Non-working example:
var query =
(from entity in ServiceContext.CreateQuery<PerformanceCountersEntity>("WADPerformanceCountersTable")
where
long.Parse(entity.PartitionKey) % interval == 0 && //bad for a variety of reasons
String.Compare(entity.PartitionKey, partitionKeyEnd, StringComparison.Ordinal) < 0 &&
String.Compare(entity.PartitionKey, partitionKeyStart, StringComparison.Ordinal) > 0
select entity)
.AsTableServiceQuery();
If you just want to get a single row based on two different time interval (now and N time back) you can use the following query which returns the single row as described here:
// 10 minutes span Partition Key
DateTime now = DateTime.UtcNow;
// Current Partition Key
string partitionKeyNow = string.Format("0{0}", now.Ticks.ToString());
DateTime tenMinutesSpan = now.AddMinutes(-10);
string partitionKeyTenMinutesBack = string.Format("0{0}", tenMinutesSpan.Ticks.ToString());
//Get single row sample created last 10 mminutes
CloudTableQuery<WadPerformanceCountersTable> cloudTableQuery =
(
from entity in ServiceContext.CreateQuery<PerformanceCountersEntity>("WADPerformanceCountersTable")
where
entity.PartitionKey.CompareTo(partitionKeyNow) < 0 &&
entity.PartitionKey.CompareTo(partitionKeyTenMinutesBack) > 0
select entity
).Take(1).AsTableServiceQuery();
The only way I can see to do this would be to create a process to keep the Azure table in sync with another version of itself. In this table, I would store the PartitionKey as a number instead of a string. Once done, I could use a method similar to what I wrote in my question to query the data.
However, this is a waste of resources, so I don't recommend it. (I'm not implementing it myself, either.)