Working with files based on filename calulated from time - linux

I have a remote Linux computer, a raspberry pi, that snaps two pictures a minute and uploads them to a Linux server. The photos are named like this: SITE-03-22-16-091543.jpeg. With the filename being formatted like: Sitename-month-day-year-hourminutesecond.jepg. Before the photo is sent, via scp, I embed some local weather date into each photo using exiv2. That way the weather conditions are stored within each photo. All of that is working fine. I hope to have about 15 of these all sending back two snaps a minute to the server.
On the server side, these photos are stored within their own SITE folder. The idea is to make time-lapse videos from each site. There are four types of time-lapses we are interested in:
1) A 24 hour loop, from 12:00am to 11:59pm.
2) A sunrise loop, from 30 minutes before sunrise to 2 hours past sunrise
3) A sunset loop, from 2 hours before sun set to 30 minutes past sunset
4) A daylight loop, from 30 minutes before sunrise to 30 minutes past sunset
The 24 hour loop is simple.
The sunrise and sunset loops are a little trickier. I downloaded and complied the “sunwait” program from Ian Craig on SourceForge (https://sourceforge.net/projects/sunwait4windows/). Using the command “sunwait list rise 35.1174N 89.9711W | gawk -F: '{ print $1$2 }'” produces the output 0700, sun rise at my location. And using the 'set' option, produces 1913, sunset at my location. Since I don't live at the equator, the sunrise and sunset vary from 5:30am to 7:30am. Depending on season. Of course.
I have the code to compile a list of images into the move, add on overlay, and add the embedded weather data. The question is how to create a list of the 30 minutes of pictures before the sunrise + 2 hours. Then 2 hours before sunset + 30 minutes past. Then finally, 30 minutes before sunrise all the way through sunset + 30 minutes.
I'm sure the answer is MATH! Can someone start me on the yellow brick road?

awk to the rescue!
substituting your script to generate time by echo here
$ echo 07:10 |
awk -F: -v offset=30 -v path="$filepath" '{
h=$1-int(offset/60);
m=$2-offset%60;
if(m<0) {m=m+60; h--}
for(i=0;i<=150;i++)
{m++;
if(m>59) {m=m%60; h++};
printf path"%02d%02d.jpeg\n",h,m}}'
creates a 151 step counter that starts from offset (in minutes) given hours minutes. For the other case enter offset as 120. Assumes start/end times doesn't change the date. May not be true around North Pole!
I think some of the variables can be simplified, but can be a working base for further improvements.
update: int() was missing, fixed, also you can pass the path as another variable

Related

Open trip planner script slower on days other than today

I'm making use of open trip planner using the jython scripting method explained here: http://docs.opentripplanner.org/en/latest/Scripting/
(specifically 'Using OTP as a library') and am using a script very similar to their example script
For testing purposes I have two csv files containing 40 locations each. The locations are inside the Netherlands and I have loaded both the dutch gtfs and map. The strange thing is that the code that calculates the public transport trip times (line 32 in the example script: res = spt.eval(colleges), using modes WALK,TRANSIT) takes longer when I specify a day other than today.
An example:
req.setDateTime(2018, 12, 8, 16, 00, 00) # today
spt.eval(my_data) # -> takes ~7 - 10 seconds
req.setDateTime(2018, 12, 7, 16, 00, 00) # yesterday
spt.eval(my_data) # -> takes ~30 - 40 seconds
When not setting req.setDateTime(), spt.eval() is even faster. Note that I ran the script on the 6th, for the 6th, as well, and it was fast then too, so it's certainly related to "today" and not specifically the 8th.
Of course my primary question is, how do I make it fast for days other than today? (my main interest is actually tomorrow)
Is it related to when the OTP instance is started or is it some internal optimization? I don't think it's related to the building of the graph because that was built a couple of days ago. I was looking into providing a day or datetime setting when initializing OTP but am unable to find that in the docs.
(I haven't tried messing with my system time yet, but that's also an option I'm not very fond of). Any ideas or comments are welcome. If necessary I will provide a reproducible sample tomorrow.
This problem was actually caused because of how I used req.setDateTime() in combination with req.setMaxTimeSec().
Basically, setMaxTimeSec() uses the date set by setDateTime() as a starting point, and defines a worstTime (aka the last possible time) to that date time + the maxTimeSec. However, if setDateTime() was not yet set when calling setMaxTimeSec(), the current date time is used instead. This will consequently cause problems when you happen to call setDateTime() AFTERWARDS. Example:
setMaxTimeSec(60*60) # Sets worst time to now + 1 hour
setDateTime(yesterday) # Sets departure time to yesterday
This example has a very long time window to search for solutions! Instead of only looking within an hour time, we are now looking in a window of 25 hours!
Anyway, a simple solution is to first call setDateTime(), and then setMaxTimeSec():
setDateTime(yesterday) # Sets departure time to yesterday
setMaxTimeSec(60*60) # Sets worst time to yesterday + 1 hour
Alternatively, if, for some reason, you can't switch these methods, you can always correct the setMaxTimeSec() with the time difference between now and your setDateTime()-value:
date = datetime.strptime('2019-01-08 21:00', '%Y-%m-%d %H:%M')
date_seconds = time.mktime(date.timetuple())
now_seconds = time.mktime(datetime.now().timetuple())
date_diff_seconds = int(round(date_seconds - now_seconds))
req.setMaxTimeSec(60*60 + date_diff_seconds)
req.setDateTime(date.year, date.month, date.day, date.hour, date.minute, 00)

When does linux set the computer's correct time?

In a Linux CentOS 5 machine, I am running process.sh using a cronjob at #reboot, every day (the machine gets shut off every night and turned on every morning).
process.sh takes the 'date' of the computer, and writes it to a log file, then exits.
When I check the logfile, the timestamp in it says "Tue Jan 1 13:14:38 GMT 2008"
When I go to the console of the computer and give it the 'date' command, it prints the correct date.
My best guess is that my cronjob is running BEFORE the computer sets its correct time.
Is there a way to fix this?
The battery that powers the CMOS memory on your motherboard may have run out. Try replacing it by a fresh one. It should look something like this.
This advice is based on the fact that the date of your log entry is "Jan 1 2008" which looks conspicuously like an epoch your motherboard may use. However, the time-of-day 13:14:38 is a little off for this; while the 13 hour shift can be explained if you are in the correct time zone, the nearly 15 minute offset seems odd. Unless your computer takes that long to boot to the point where cron executes your job. And of course, the reason why you eventually see the correct time is probably that ntp fixed the system time, as others have noted.

NodaTime supports relative time display?

I have a situation where the relative time is more important to a user than an absolute time. So it's more important to be able to quickly say "event happened 5 days and 5 hours ago" than "event happened at 1 PM CDT and it's 5 PM CST 5 days later now."
We store our dates in UTC and convert to display for the user:
pDateTime = DateTime.SpecifyKind(pDateTime, DateTimeKind.Utc);
DateTimeZone dateTimeZone = DateTimeZoneProviders.Tzdb[pCurrentUser.PreferredTimezone];
return Instant.FromDateTimeUtc(pDateTime).InZone(dateTimeZone).ToString("HH':'mm':'ss' 'MM'/'dd'/'yy' 'x", CultureInfo.InvariantCulture);
We'll be using NodaTime 1.2 when it's fully out and just used vanilla ToString before.
However, times using this pattern end up using the daylight status of the time as opposed to the current daylight status. This means that times look something like: 16:15:32 10/25/13 CDT even though we have now transitioned to CST.
It is an absolute measure of the time. This forces the user to do the logic: "How long ago was that? Is it daylight saving time now? If so, the difference is x. If not, I have to add or subtract an hour? That makes the difference y."
Meanwhile, a relative measure of the time would display 15:15:32 10/25/13 CST in the absence of DST. This forces the user to do no conversions and allows them to compute what that time means in context much easier.
In a display that has numerous dates, it can get tricky to do the absolute time logic over the entire set. Doing it once is tricky to get right. However, a friendly relative string like "posted 5 hours ago" also forces them to resolve both the date and time themselves - that information is still important.
A compromise might be to do the posted blank hours/minutes ago for the first 24 hours or to include both the friendly string and absolute time - these are both patterns I've seen done.
But ignoring those, is there a way in NodaTime to imbue a time with a specific daylight status in order to get times displaying in a relative context?
However, times using this pattern end up using the daylight status of the time as opposed to the current daylight status. This means that times look something like: 16:15:32 10/25/13 CDT even though we have now transitioned to CST.
Yes, and it should. Displaying a date/time with CST despite that date/time occurring in CDT would be very odd, IMO.
So it's more important to be able to quickly say "event happened 5 days and 5 hours ago" than "event happened at 1 PM CDT and it's 5 PM CST 5 days later now."
In that case you shouldn't be displaying a date/time at all, in my view. Convert both ZonedDateTime values to Instant, take the Duration between them, and then you can see that it's 5 days and 5 hours ago. (I can't remember how much help we provide with that - you may need to manually take the number of ticks and divide by NodaConstants.TicksPerStandardDay etc. Look at DurationPattern to see if it helps though.)
Alternatively, if you really want to display a date and time, but still easily be able to extract the difference between them mentally, two options suggest themselves:
Use OffsetDateTime instead; there you could force the offsets to be the same, although I still think it would be odd to display an offset which wasn't actually the current offset in the zone you were observing the time in. Or you could just display the relevant offset at the time, so -5 for CST and -4 for CDT.
Just display everything in UTC, so that daylight saving transitions are irrelevant.
Note that you can't get months between the two ZonedDateTime values, as we're dealing with an elapsed time (a duration) rather than calendar-logical arithmetic (a period).

Shell Scripting - Hour and Minute Computation

I am working on a shell script computation program wherein I want a script that computes for the total number of hours and minutes a particular user has been logged on to the system. The script accepts one parameter, the login name. Information like the complete name of the user, the current month and the dates the user logged on earliest during the month and the latest during the month. In the pseudo, I wanted somewhat like this:
[prompt]$ <your familyname>user_login mycroft<enter>
For the Month of June :
User : mycroft
In Real life : Mycroft Holmes
period : Mon Jun 16 to Sat Jun 28
total login time : 2 hours and 2 minutes
Is there a possible way to do this in Shell? I have been on blanks and errors until now.
Look at who or finger program. Probably it's all you need.

How should timestamps in last output be interpreted on linux?

$ last -aid
foouser pts/12 Sun Feb 15 07:30 - 15:23 (2+07:52) XXX.YYY.ZZZ.QQQ
The Sun Feb 15 07:30 - 15:23 part is pretty obvious. The user logged in at 7:30 on Sunday. But (2+07:52) is unclear. 7:52 is the difference between the two times, but what's the 2+ part?
Use the source, Luke:
sprintf(length, "(%d+%02d:%02d)", days, hours, mins);
This is from the sysvinit source package on Debian.
Hint for those without knowledge in C: it's the number of days.
The string between the parentheses is the duration of the session. As you note, the time is the difference between the end and start times. The 2+ is the number of days as hop noted.
At some point, this question probably should be moved over to Server Fault since it's most likely to be of interest to System Administrators.
It's a fairly readable format once you know what you are looking at. If I were designing the output, I think I'd make the day of the week optional and put the entire end date in the string instead. Even better would be to let the date stamp be configurable. That way, the output could be more easily used by another program.
The actual duration, if the user has logged out of the session is fairly easy to pick out with a regular expression:
$ last | perl -lne 'print "$2 days, $3 hours, $4 minutes" if /\(((\d+)\+)(\d{2}):(\d{2})\)$/'
17 days, 05 hours, 23 minutes
3 days, 23 hours, 16 minutes
14 days, 06 hours, 09 minutes
23 days, 04 hours, 54 minutes
16 days, 06 hours, 57 minutes
...
Those might be days but I'm not sure; also this is not a programming question.

Resources