I have an application that automatically updates itself each time it is run. It does this by comparing the "modified" timestamp on each file in the program directory, copying files that are newer from a network file share.
I packaged this program with Inno Setup Compiler and ran the resulting installer on a test system. All of the installed files have timestamps one second older compared to the files I used when compiling the installer. This causes every file in the program directory to be recopied from the file share on first run. This is an issue for our remote users that have limited bandwidth.
I've tried this numerous times, with and without the "TimeStampsInUTC" directive set in my Inno Setup script and on four different test systems. The timestamps are consistently one second off.
Could this be a problem in Inno Setup Compiler? Are there any suggestions to work around this issue? I'm hoping to use this installer at two new offices soon.
There's TimeStampRounding directive, which is by default set to 2:
By default, time stamps on files referenced by non external [Files] section entries are rounded down to the nearest 2-second boundary. FAT partitions have only a 2-second time stamp resolution, so this ensures that time stamps are set the same way on both FAT and NTFS partitions.
The rounding can be altered or disabled by setting the TimeStampRounding directive. Setting it to 0 will disable the rounding. Setting it to a number between 1 and 60 will cause time stamps to be rounded down to the nearest TimeStampRounding-second boundary.
(emphasis mine)
Related
In a custom-developed NodeJS web server (running on Linux) that can dynamically generate thumbnail images, I want to cache these thumbnails on the filesystem and keep track of when they are actually used. If they haven't been used for a certain period of time (say, one year), I'd consider them "orphans" and delete them.
To this end, I considered to touch them each time they're requested from a client, so that I can use the modification time to check when they were last used.
I assume this would incur a significant performance hit on the web server in high-load situations, as it is an "unnecessary" filesystem write, while, apart from logging, most requests will only consist of reads.
Has anyone performed any benchmarks on how big an impact this might have and if it's worthwhile?
It's probably not great, and probably worth avoiding updating every time you open a file. That's the reason the relatime / noatime mount options were invented, to prevent the existing Unix access-time timestamp from being updated every time a file was opened.
Is your filesystem mounted with relatime? That updates atime at most once per day, when the file is opened (even for reading). The other mount option that's common on Linux is noatime: never update atime.
If you can't let the kernel do this for you without needing extra system calls, you might be better off making an fstat system call after opening the file and only touching it to update the mod time if the mod time is older than a day or week. (You're concerned about intervals of a year, so a week is fine.) i.e. manually implement the relatime logic, but for mod time.
Frequently accessed files will not need updates (and you're still making a total of one system call for them, plus a date-compare). Rarely accessed files will need another system call and a metadata write. If most of the accesses in your access pattern are to a smallish set of files repeatedly, this should be excellent.
Possible reasons for not being able to use atime could include:
The filesystem is mounted with noatime and it's not worth changing that
The files sometimes get read by something other than your web server / CGI setup. (e.g. a backup job that does more than compare size / timestamps)
Of course, the other option is to not update timestamps on use, and simply let a thumbnail be regenerated once a year after your weekly cron job deleted it. That might be ok depending on your workload.
If you manually touch some of the "hottest" thumbnails so you stagger their deletion, instead of having a big load spike this time next year, you could be ok. And/or have your deleter walk your filesystem very slowly, again so you don't have a big batch of frequently-needed thumbnails deleted at once.
You could come up with schemes like enabling mod-time updates in the week before the bi-annual cleanup, so thumbnails that should stay hot in cache get their modtimes updated. But probably better to just fstat / check / update all the time since that shouldn't be too much extra load.
I have a project packaged using electron-builder with NSIS target, producing as artifacts a 40 MB .exe file and a .exe.blockmap file (which is gzipped JSON, I know). The problem is that even if something as simple as version number changes, blockmaps start differing vastly (for example only 1756 chunks of 2032 matched) and each update ends up downloading multiple megabytes.
I understand it may not be easy to make a detailed file-by-file map of an NSIS .exe containing app-64.7z containing app.asar finally containing files, but does electron-builder even try? Can it be overridden to use some binary-diff as basis for the block splitting, to ensure minimum differences between consecutive versions? I can't seem to find any documentation on app-builder.exe's routines for blockmap creation.
The largest part of the solution was introducing shortVersion and shortVersionWindows configuration strings in electron-updater v22.3.1 (20 Jan 2020), letting the 50 MB application not be re-stamped with full versioning at every single version update.
Also, space-padding all -bundle.js files (if present) helps to keep any app.asar file length fields to stay the same, again limiting the scope of changes in the final file.
I have a class that does some parsing of two large (~90K rows, 11 columns in the first and around ~20K, 5 columns in the second) CSV files. According to the specification I'm working with the CSV files can be externally changed (removing/adding of new rows; columns remain constant as well as the paths). Such updates can happen at any time (though highly unlikely that an update will be launched in time intervals shorter than a couple of minutes) and an update of any of the two files has to terminate the current processing of all that data (CSV, XML from an HTTP GET request, UDP telegrams), followed by re-parsing the content of each of the two (or just one if only one has changed).
I keep the CSV data (quite reduced since I apply multiple filters to remove unwanted entries) in memory to speed working with it and also to avoid unnecessary IO operations (opening, reading, closing file).
Right now I'm looking into the QFileSystemWatcher, which seems to be exactly what I need. However I'm unable to find any information on how it actually works internally.
Since all I need is to monitor 2 files for changes the number of files shouldn't be an issue. Do I need to run it in a separate thread (since the watcher is part of the same class where the CSV parsing happens) or is it safe to say that it can run without too much fuss (that is it works asynchronously like the QNetworkAccessManager)? My dev environment for now is a 64bit Ubuntu VM (VirtualBox) on a relatively powerful host (a HP Z240 workstation) however the target system is an embedded one. While the whole parsing of the CSV files takes just 2-3 seconds at the most I don't know how much performance impact there will be once the application gets deployed so additional overhead is something of a concern of mine.
I would like to configure log4j to write only files up to a maximum specified size, e.g. 5MB. When the log file hits 5MB I want it to start writing to a new file. I'd like to put some kind of meaningful timestamp into the logfile name to distinguish one file from the next.
I do not need it to rename or manipulate the old files in any way when a new one is written (compression would be a boon, but is not a must).
I absolutely do not want it to start deleting old files after a certain period of time or number of rolled files.
Timestamped chunks of N MB logfiles seems like the absolute basic minimum simple strategy I can imagine. It's so basic that I almost refuse to believe that it's not provided out of the box, yet I have been incapable of figuring out how to do this!
In particular, I have tried all incarnations of DailyRollingFileAppender, RollingFileAppender or the 'Extras' RollingFileAppender. All of these delete old files when the backupcount is exceeded. I don't want this, I want all my log files!
TimeBasedRollingPolicy from Extras does not fit because it doesn't have a max file size option and you cannot associate a secondary SizeBasedTriggeringPolicy (as it already implements TriggeringPolicy).
Is there an out of the box solution - preferably one that's in maven central?
I gave up trying to get this to work out of the box. It turns out that uk.org.simonsite.log4j.appender.TimeAndSizeRollingAppender is, as the young folk apparently say, the bomb.
I tried getting in touch some time back to get the author to stick this into maven central, but no luck so far.
I'm writing a program that needs to be able to read in the time zone files on Linux. And that means that I need to be able to consistently find them across distros. As far as I know, they are always located in /usr/share/zoneinfo. The question is, are they in fact always located in /usr/share/zoneinfo? Or are there distros which put them elsewhere? And if so, where do they put them?
A quote from tzset(3):
The system timezone directory used depends on the (g)libc version.
Libc4 and libc5 use /usr/lib/zoneinfo,
and, since libc-5.4.6,
when this doesn't work, will try /usr/share/zoneinfo.
Glibc2 will use the environment
variable TZDIR, when that exists. Its
default depends on how it was installed, but normally is
/usr/share/zoneinfo.
Note, however, that nothing prevents some perverse distro from patching libc and placing the files wherever they want.
The public-domain time zone database contains the code and data to handle time zones on Linux.
The public-domain time zone database
contains code and data that represent
the history of local time for many
representative locations around the
globe. It is updated periodically to
reflect changes made by political
bodies to time zone boundaries, UTC
offsets, and daylight-saving rules.
This database (often called tz or
zoneinfo) is used by several
implementations, including the GNU C
Library used in GNU/Linux, FreeBSD,
NetBSD, OpenBSD, Cygwin, DJGPP, AIX,
Mac OS X, OpenVMS, Oracle Database,
Solaris, Tru64, and UnixWare.
That covers a lot of system but I can only agree with Roman that nobody can be prevented from creating a distribution that differs for whatever reasons. The existence and location of a zonezinfo file is not covered by any official standard as far as I know. The standards (e.g. POSIX and XPG4) only establish the API.