Is there a way of flushing Windows.Storage.ApplicationData.Current.LocalSettings to disk? I cannot find any documentation about when the value is persisted after setting it.
The root problem is that is in some abrupt shutdown scenarios (e.g. power off, system restart) my colleagues claim that we intermittently experience lost (as in: not persisted) values.
Related
Linux exploits remaining memory for its file cache
In my application (written in C++), I'd like to flush the dirty pages to disks explicitly from time to time
(Using O_DIRECT is not appropriate for me)
I tried fflush(), but it seems not what I wanted
Is there any way to flush the dirty pages of OS file cache to disks?
Thanks
You can use sync_file_range() to encourage flushing on Linux but confusingly you can't use sync_file_range() to guarantee file durability/data integrity - it is simply a hint that might help get flushing underway (see this Linux Plumbers Conference 2019 video of Postgres developer Andres Freund complaining about the sync_file_range()'s manpage and the reply from filesystem developer Jan Kara). In short: it can help trigger flushing but you'll need to add/use something else to know durability.
I believe all the usual file descriptor sync style calls (fsync(), fdatasync() etc.) also hint that you want writeback to start occurring but in a more heavy handed fashion compared to sync_file_range() (because they also force flushing of device non-volatile caches too)...
I am trying to understand the difference between File system consistent and crash consistent backups provided by Azure. The majority of the information that I find is from this link. I see Application consistent backup is to ensure that all memory data and pending I/O are accounted for perhaps by using a quiescing process so proper snapshot can be taken. However bit confused between the other two. I see Crash consistent is one which doesn't consider the in-memory, pending I/Os and only considers backing up what has been written. But then what exactly would be meant by file-consistent backup? I don't find any definition. As a result when the docs mention that by default Linux VM backups are File system consistent if not using pre/post scripts, I am not understanding the implications. Any help much appreciated.
Simple example to demark the difference is : when a recovery point is file-system consistent, there won't be any file system check performed to make sure that file system is not corrupted. In case of crash consistency, after a VM boots up, a file-system check may be performed and based on that there can be potentially a data loss because of corruption of file system. So, it is always better to strive for file system consistency.
Recently, January 3rd, we observed interesting behavior with Redis Cache in Azure. It happened just once, and I'm trying to make sense of it.
We got alert that CPU went above 80% on Redis Cache service. Looking closely we discovered that used memory was dropped from typical 100MB to almost 0. Then it was quickly populated back to normal, I assume by normal usage of the application. While it was being populated, there was this CPU spike.
It looked like if cache was reset. However, this is production environment with very limited people having access to it, and we sure 100% that nobody reset it. There were no any deployment around that time. I couldn't find anything in diagnostic logs.
Questions:
1. Any ideas what could happen?
2. Where can I look, what to look for?
Update: We are on standard (C1) tier
No customers reported any problems, I just hate when I don't understand what is going on.
It depends on which cache tier you are using.
The basic tier only has one node with the cache data stored in memory. Any loss of memory in that node will cause the cache data to be lost.
If you are using the Standard tier then there are 2 nodes, a primary and secondary, with cached data being asynchronously replicated from primary to secondary. If the primary is offline then client requests are sent to the secondary. In this scenario the chance of cache data loss is low since it basically requires both nodes to be offline at the same time, which should only happen during scenarios of hardware failure (Azure ensures that normal updates maintenance such as OS updates are not done at the same time).
If you are using the premium tier then the cache data is backed by persistent storage so you should not experience cache data loss.
https://azure.microsoft.com/en-us/documentation/articles/cache-faq/#what-redis-cache-offering-and-size-should-i-use has some more information about this.
We have a couple of production couchdb databases that have blown out to 30GB and need to be compacted. These are used by a 24/7 operations website and are replicated with another server using continuous replication.
From tests I've done it'll take about 3 mins to compact these databases.
Is it safe to compact one side of the replication while the production site and replication are still running?
Yes, this is perfectly safe.
Compaction works by constructing the new compacted state in memory, then writing that new state to a new database file and updating pointers. This is because CouchDB has a very firm rule that the internals of the database file never gets updated, only appended to with an fsync. This is why you can rudely kill CouchDB's processes and it doesn't have to recover or rebuild the database like you would in other solutions.
This means that you need extra disk space available to re-write the file. So, trying to compact a CouchDB database to prevent full disk warnings is usually a non-starter.
Also, replication uses the internal representation of sequence trees (b+trees). The replicator is not streaming the entire database file from disk onto the network pipe.
Lastly, there will of course be an increase in system resource utilization. However, your tests should have shown you roughly how much this costs on your system vs an idle CouchDB, which you can use to determine how closely you're pushing your system to the breaking point.
I have been working with CouchDB since a while; replicating databases and writing Views to fetch data.
I have seen its replication behavior and observed this, which can answer your question:
In the replication process previous revisions of the documents are not replicated to the destination, only current revision is replicated.
Compacting database only removes the previous revisions. So it will not cause any problem.
Compaction will be done on the database on which you are currently logged in. So it should not affect its replica which is continuously listening for changes in it. Because it listens for the current revision changes not the previous revisions. To verify it you can see this:
Firing this query will show you changes of all the sequences of database. It only works on the basis of latest revision changes not the previous ones(So I think compaction will not make any harm):
curl -X GET $HOST/db/_changes
The result is simple:
{"results":[
],
"last_seq":0}
More info can be found here: CouchDB Replication Basics
This might help you to understand it. In short answer of your question is YES, It is safe to compact database in continuous replication.
We have a really strange problem. One of the servers in the server farm becomes really slow. We see a number of timeouts in the logs and overall response time is not where it should be (and is on other servers in the farm).
What is also strange is that it is not just the web app - Just logging into the server takes up to 1.5 min to show you the desktop. Once you are in, the system is as responsive as ever - unless you try to launch something, i.e. notepad - it takes another minute to launch and after launch it works fine.
I checked a number of things - memory utilization is reasonable, CPU is below 15%, windows handles, event logs do not show anything.
Recycling the aps.net process does not fix it - it still takes over a minute to log in. Rebooting the server helped, but now it started to slow down again.
After a closer look we found out that Windows Temp directory is full of temp files - over 65k files. This is certainly something to take care of. But my question is could it be the root cause of the sluggishness, or there is still something else lurking in the shadows?
Edit
After more digging I am zeroing in on the issue related to the size of temp directories. This article: describes something very similar. I am still not too sure because the fact that the server is slow to open even Notepad remains unexplained.
Is it possible that under such conditions creating a new temp file takes over a minute?
You might want to check how many threads your using in the ASP.NET thread pool when the timeouts occur. Another idea might be to look at the GC information in perfmon and see if the GC is running a gen2 collection?
Ok, It is official, all of this was grief caused by this issue. When one of our servers was again behaving badly we cleaned the temp directory and it fixed the problem, including the slow login.
This last part still baffles me - I do not understand how excessive number of files in a temp directory can cause login to take over 1 min, leave alone launching a program, but whatever it is clearing the directory fixed it and I can live with it.
Did you check virtual memory as well ? paging ? does you app logs a lot of data in different files ? also - check - maybe the utilization happens in kernel mode and not user mode.