Log Parser Query - iis

I am working on getting a specific report out of a bunch of IIS Logs using Log parser. At the moment when an unexpected error occurrs on the web application, we send the user on to system error page (SystemError.htm).
I want to know what page they were on just before they were redirected to the system error page. I know this is a silly way to do error reporting/logging, but this is what I have to work with.
At the moment I can retrieve the IP Address, date and time of the user that ended up on the system error page.
SELECT c-ip as IPAddress, date as Date, time as Time
FROM D:\IISLog\*.log
WHERE cs-uri-stem = '/SystemError.htm'
ORDER BY c-ip, date, time DESC
I am using the log parser as such:
LogParser.exe -i:IISW3C file:C:\IISLog.sql -q:off -recurse:1
This means I recurse through alot of IISLogs that exist in subfolders. What I would I like to do now is join the result from this query to the logs again on the ip address, and take the top 1 result that has a datetime before the system error page.
My issue is that I can't seem to find a way to do the join. As far as I can see this is not possible. Has anyone of you's come across something like this? My SQL isn't the greatest, maybe I'm missing another way to do it.
Maybe this is too much for the logparser, time to switch to powershell? Any help will be appreciated.
Thanks in advance!

When using the W3C log format you could try to track and evaluate cs(Referer):
SELECT c-ip as IPAddress, date as Date, time as Time, cs(Referer) as Referer
FROM D:\IISLog\*.log
WHERE cs-uri-stem = '/SystemError.htm'
ORDER BY c-ip, date, time DESC

Related

Dialogflow console returning different result to when sent through the API (Where the wrong timezone is used)

I have a DialogFlow agent set up where one intent is used to schedule reminders. I can say something like:
"At 5pm remind me to go for a run"
And it returns the sentence to remind (in this case 'go for a run') as well as the time to set the reminder using #sys.date-time.
This works as intended, I am able to get the correct time because it just sends the time without a timezone attached.
When I use a command such as:
"In 15 minutes remind me to go for a run"
it sends the result as the time using the timezone, in this case, the incorrect one.
So right now, a result for the date-time using the API was:
2020-11-09T14:20:33+01:00
which is an hour more than it should be.
I have checked the DialogFlow agent's default time zone where it is set to:
(GMT0:00) Africa/Casablanca
Which I am fairly certain is the correct one for London time. However moving to a different timezone changes it and actually gives the correct one for the timezone (Just not my timezone)
Leaving me to wonder if this time zone is broken?
Regardless though, the Dialogflow console on the webpage returns the correct date-time but in a different format using 'startDateTime' and 'endDateTime', something that the agent does not do when sent using the API.
I have checked all configurations within the program and cannot find any evidence of any code giving a new timezone and in fact have tried to add the London timezone when a query is sent but this does not resolve the issue.
Does anyone have any advice on how to solve this?
EDIT:
After receiving a good suggestion from a user I am reminded of the most puzzling part of this issue. Chaning the timezone to GMT -1:00 vs 0:00 actually having a difference of two hours.
Around 1pm I queried it to get the time in 15 minutes.
When it was set to GMT-1:00 Atlantiv/Cape_Verde the time returned is:
2020-11-10T12:21:15-01:00
When it was set to GMT0:00 Africa/Casablanca the time returned is:
2020-11-10T14:22:07+01:00
Neither is the correct time and despite the timezone suggesting a 1 hour difference, it is actually 2 hours apart.
For London the correct timezone should be GMT -1, Casablanca Africa is GMT+1, I used this web page to determine this.
If you are in London is recommended to configure your agent to sue the correct time zone (GMT-1).

The following code has been giving me the same output for a long time

I have been writing some code with PRAW. I want the following to give the top posts each day.But it has been returning the same ids even though new posts are submitted each day and different posts are at the top. What should I do?
import praw
reddit=praw.Reddit(client_id='id',
client_secret='secret',password='password',
user_agent='agent',username='username')
top_funny=subreddit.top(limit=1)
for submission in top_funny:
print(submission)
by default subreddit.top() returns the top of all time. You can change this by using the time_filter parameter and specify any of the following: hour, day, week, month, year, all.
ex:
subreddit.top("day", limit=1)
will return the top submission from today.
As LampToast said, you should use the time_filter.
Arguments for .top() are listed in the docs:
Parameters: time_filter – Can be one of: all, day, hour, month, week, year (default: all).
https://praw.readthedocs.io/en/latest/code_overview/models/multireddit.html#praw.models.Multireddit.top

TZD database missing transitions?

I'm using version 'TZDB: 2018c (mapping: 13290)' of the TZD db however when I ask for Pacific/Auckland timezone information using either
VTimeZone.FromTzId(tzdbId);
or
VTimeZone.FromTzId(tzdbId, earliestDateTimeToSupport, true);
If I use 2008-01-01 as earliestDateTimeToSupport I'll get the current daylight saving details and recurrence rules. If however I use any date after that or NO date I'll get UTC -11 (WHEN was that rule last in play??? not while I've been alive.) and the recurrence rule looks all wrong. Can anyone advise if this is the DB or something I'm likely to be doing?
Australia/Sydney seems to behave properly... they've had a timezone change during 2008 and that rule comes through... and I'm not going to test them all but I think this is just a fault with NZDST for dates post (ish) 2009-1-1.

Strange sybase behavior around daylight savings time (DST)

I've got a strange sybase behavior which I do not understand.
Situation
I have a table (MY_TABLE) with several columns of type smalldatetime. For illustration purposes let's assume the following table and data:
MY_TABLE||ID |TS_INIT |TS_LASTCHANGE |MY_TEXT |
||4711|3/31/2013 12:00:00 AM|3/31/2013 3:00:00 AM|someText|
TS_INIT and TS_LASTCHANGE are of type smalldatetime.
When executing the following statement I get the above result:
SELECT ID, TS_INIT, TS_LASTCHANGE MY_TEXT
FROM MY_TABLE
WHERE ID = 4711
go
My client is running in UTC+1 (Berlin) and has daylight savings time (DST) enabled.
I am not sure in what time zone the server is running and whether or not DST is enabled.
Problem
When I execute this (note that it is 03:00h):
SELECT ID, TS_INIT, TS_LASTCHANGE MY_TEXT
FROM MY_TABLE
WHERE ID = 4711 AND TS_LASTCHANGE = "2013-03-31 03:00:00:000"
go
I get NO results but when I execute this (note that it is 02:00h this time):
SELECT ID, TS_INIT, TS_LASTCHANGE MY_TEXT
FROM MY_TABLE
WHERE ID = 4711 AND TS_LASTCHANGE = "2013-03-31 02:00:00:000"
go
I do again get the above result which is saying TS_LASTCHANGE is
3/31/2013 3:00:00 AM
Note that the result prints 03:00h, even though I queried for 02:00h.
Why Is the first query returning no results even though there should be a match and why is the second query returning a result even though there should be no match?!
Note also that 3/31/2013 3:00:00 AM is the first moment in DST (at least in the year 2013) and 3/31/2013 2:00:00 AM should never ever exist at all because when transitioning from winter to summer time, the clock switches from 01:59:59 to 03:00:00 (as per this site).
Database: Adaptive Server Enterprise V15.0.3
Client: Aqua Data Studio V16.0.5
EDIT:
When querying whit the TS_INIT everything works as one would expect (only a result for 3/31/2013 12:00:00 AM)
Aqua Data Studio is written in Java.
The problem you are having has to do with the fact that Java is aware of timezones and databases don't have a concept of timezone when they store date and times. When the time comes back from the database, the database's JDBC driver puts it in a Java date and just assumes the timezone is irrelevant. The problem happens when you try to display a time which the JVM thinks is invalid, so a valid date is presented, which basically pushes the time by an hour. Daylight savings for 2015 started on March 08 2.00 AM and one of your rows contains a date which is invalid according to JVM.
This has been a known design issue with Java, and they are trying to fix this with JSR-310 for inclusion in Java SE 8. With this, they will have LocalDate, OffsetDate and ZonedDate. You can read more about it here ...
https://today.java.net/pub/a/today/2008/09/18/jsr-310-new-java-date-time-api.html#jsr-310-datetime-concepts
https://jcp.org/en/jsr/detail?id=310
http://docs.google.com/View?id=dfn5297z_8d27fnf
Workaround
The only workaround, is to probably trick the JVM by setting the timezone in the JVM to GMT. If you are running ADS 16 on Windows, and you launch ADS with the shortcut icon on the desktop (which runs datastudio.exe), then you need to modify the datastudio.ini file in your folder. Add a new entry for vmarg.5=-Duser.timezone=gmt
This link explains the location of where to find the data studio.ini
https://www.aquaclusters.com/app/home/project/public/aquadatastudio/wikibook/Documentation14/page/50/Launcher-Memory-Configuration#windows
Once you have made the change, Restart ADS. Then go to Help->About->System: and double check your user.timezone setting and make sure it is GMT. Then give it a try.
With the above change there might be side effects in the application where timezone are involved, For e.g. in the Table Data Editor->Insert Current Date&Time, which would display a GMT time ... so there would be an offset.

Search formula not working on Linux machine

I have a strange behavior: an agent called via an AJAX request should search documents to display in a calendar. For that reason I compute a search formula and then run the search method of my database in Lotusscript. This is the formula:
form="mholiday" | form="mserviceevent" | (form="mereignis" & co_status!="9") & #texttotime(#text(startdatetime)) >= [29.09.2014] & #texttotime(#text(enddatetime)) =< [10.11.2014]
Everything's fine on Domino on Windows but fails with "formula error" on a Linux machine. Am I missing something?
If I omit the term with the dates everything is fine, so this is the part that causes the error.
Try it with #ToTime() and #Date() instead. That might help to get away from local settings' influence on server:
... & #ToTime(startdatetime) >= #Date(2014; 9; 29) & ...
#ToTime() doesn't convert the field if it's a date time value already.
#Date doesn't depend on local settings whereas [29.09.2014] probably does.
I don't think it's a Linux problem, I think it's a data problem. It sounds like either a date format problem or a problem with the UNK table, used by full text search.
If the first document created on that server that had a field called "startdatetime" had a text value, then any search expects "startdatetime" to be a text value, even if there is another field in the database called startdatetime that is a date or the startdatetime field is subsequently changed to be a date. To confirm this, you can use the search bar and select the field. The operators it offers will confirm if it's expecting a date or a text value. See this answer for details on how to resolve "Query is not understandable" - Full text searching where field types have changed.
Alternatively, it may be a problem with the date format, as Knut says. In which case a test for 9/9/2014 would work but 29/9/2014 wouldn't.

Resources