I am creating a sqlite database in temp folder. Now I want to copy that file to another folder. Is there any sqlite command to rename the sqlite database file?
I tried using rename function in c++ but it returns error 18. Error no 18 means: "The directory containing the name newname must be on the same file system as the file (as indicated by the name oldname)".
Can someone suggest a better way to do this.
Use a temporary directory on the correct filesystem!
First, sqlite database is just a file. It can be moved or copied around whatever you wish, provided that:
It was correctly closed last time, so there would be no stuff to roll-back in the journal
If it uses write-ahead log type of journal, it is fully checkpointed.
So moving it as a file is correct. Now there are two ways to move a file:
Using the rename system call.
By copying the file and deleting the old one.
The former has many advantages: it can't leave partially written file around, it can't leave both files around , it is very fast and if you use it to rename over old file, there is no period where the target name wouldn't exist (the last is POSIX semantics and Windows can do it on NTFS, but not FAT filesystem). It also has one important disadvantage: it only works within the filesystem. So you have to:
If you are renaming it to ensure that a partially written file is not left behind in case the process fails, you need to use rename and thus have to use a temporary location on the same filesystem. Often this is done by using different name in the same directory instead of using temporary directory.
If you are renaming it because the destination might be slow, e.g. because you want to put it on a network share, you obviously want to use temporary directory on different filesystem. That means you have to read the file and write it under the new name. There is no function for this in the standard C or C++ library, but many libraries will have one (and high-level languages like python will have one and you can always just execute /bin/mv to do it for you).
Now I want to copy that file to another folder. Is there any sqlite command to rename the sqlite database file?
Close the database. Copy the database to the new path using the shell.
Also see Distinctive Features Of SQLite:
Stable Cross-Platform Database File
The SQLite file format is cross-platform. A database file written on
one machine can be copied to and used on a different machine with a
different architecture. Big-endian or little-endian, 32-bit or 64-bit
does not matter. All machines use the same file format. Furthermore,
the developers have pledged to keep the file format stable and
backwards compatible, so newer versions of SQLite can read and write
older database files.
Most other SQL database engines require you to dump and restore the
database when moving from one platform to another and often when
upgrading to a newer version of the software.
Related
I have a hash-value database with tags and I want to implement a FUSE interface for it. Because values are indexed by their hashes they must be read-only.
Native interface for this database is very simple:
You can download, upload or tag a file.
You can get the set of all defined tags.
You can search for files tagged in accordance to a boolean combination of tags.
FUSE interface semantics are simple:
Database is viewed as a big synthetic directory hierarchy where values are files named by its hash and tags are directories.
cd-ing inside a directory is semantically equivalent to search for a given tag (naming conventions on paths can be used to implement boolean operations).
read-ing a file is semantically equivalent to download (part of) a value (FUSE allows an stateless read so open and close can be no-ops).
Copying/moving an inexistent file into a given path is equivalent to upload and tag it. Copying/moving an existent file into a given path is equivalent to add new tags.
Any other operation throws an error.
This FUSE interface is quite usable and allows you to easily embed a tag file system inside a hierarchical one without the need of external tools like TagSpaces or Evernote.
My problem arises identifying a file copy or move from any other forbidden operation with FUSE interface: there are endless possible combination of operations with equivalent semantics.
What is the most reliable way to identify a file copy or move with FUSE interface?
Hooking rename of a file should be straightforward by implementing rename() fuse call. In this call, you will get path of both old and new location, so that you can check if the file comes from outside or not. That said, this would work only if user space tool renames a file by invoking rename(2) kernel call.
On the other hand, hooking file copy operation would be harder: it can't be done directly as there is no such fuse call - copying happens in user space completely and so it's not directly detectable in kernel space.
You could try to do some heuristics and process incoming fuse operations to detect rename of already stored file (eg. by hashing content of new file and comparing that with already existing files), but I'm not sure how much it makes sense in your case or if it would be actually practical.
I wonder if there is any way i could retrieve all changes i made to my various configuration files since install(residing in /etc and so on) in one shot?
I imagine some kind of loop, that uses 'diff' to compare all those files to a 'standard installation' of ubuntu. Output should be a single file with information regarding the changes that were made and a timestamp.
Perhaps there is even a way to put all that in a script and let it run regularly to automatically keep track of future config file changes.
If the files are already modified, I guess your only option is to diff your files with a fresh install. Keep in mind some files might be specific to you computer, I'm thinking of files that can hold device-specific values like your mac address udev/rules.d/70-persistent-net.rules, your drives uuid /etc/fstab, etc.
If you're planning this ahead, there are at least two options you can consider:
use a VCS such as git.
use a filesystem that keeps a complete history of the changes made.
The thing is, I want to track if a user tries to open a file on a shared account. I'm looking for any record/technique that helps me know if the concerned file is opened, at run time.
I want to create a script which monitors if the file is open, and if it is, I want it to send an alert to a particular email address. The file I'm thinking of is a regular file.
I tried using lsof | grep filename for checking if a file is open in gedit, but the command doesn't return anything.
Actually, I'm trying this for a pet project, and thus the question.
The command lsof -t filename shows the IDs of all processes that have the particular file opened. lsof -t filename | wc -w gives you the number of processes currently accessing the file.
The fact that a file has been read into an editor like gedit does not mean that the file is still open. The editor most likely opens the file, reads its contents and then closes the file. After you have edited the file you have the choice to overwrite the existing file or save as another file.
You could (in addition of other answers) use the Linux-specific inotify(7) facilities.
I am understanding that you want to track one (or a few) particular given file, with a fixed file path (actually a given i-node). E.g. you would want to track when /var/run/foobar is accessed or modified, and do something when that happens
In particular, you might want to install and use incrond(8) and configure it thru incrontab(5)
If you want to run a script when some given file (on a native local, e.g. Ext4, BTRS, ... but not NFS file system) is accessed or modified, use inotify incrond is exactly done for that purpose.
PS. AFAIK, inotify don't work well for remote network files, e.g. NFS filesystems (in particular when another NFS client machine is modifying a file).
If the files you are fond of are somehow source files, you might be interested by revision control systems (like git) or builder systems (like GNU make); in a certain way these tools are related to file modification.
You could also have the particular file system sits in some FUSE filesystem, and write your own FUSE daemon.
If you can restrict and modify the programs accessing the file, you might want to use advisory locking, e.g. flock(2), lockf(3).
Perhaps the data sitting in the file should be in some database (e.g. sqlite or a real DBMS like PostGreSQL ou MongoDB). ACID properties are important ....
Notice that the filesystem and the mount options may matter a lot.
You might want to use the stat(1) command.
It is difficult to help more without understanding the real use case and the motivation. You should avoid some XY problem
Probably, the workflow is wrong (having a shared file between several users able to write it), and you should approach the overall issue in some other way. For a pet project I would at least recommend using some advisory lock, and access & modify the information only thru your own programs (perhaps setuid) using flock (this excludes ordinary editors like gedit or commands like cat ...). However, your implicit use case seems to be well suited for a DBMS approach (a database does not have to contain a lot of data, it might be tiny), or some index locked file like GDBM library is handling.
Remember that on POSIX systems and Linux, several processes can access (and even modify) the same file simultaneously (unless you use some locking or synchronization).
Reading the Advanced Linux Programming book (freely available) would give you a broader picture (but it does not mention inotify which appeared aften the book was written).
You can use ls -lrt, it displays the last RW operations in the shell. Then you can conclude whether the file is opened or not. Make sure that you are in the exact directory.
POSIX famously lets processes rename and unlink file entries with no regard as to the effects on others using them, whilst Windows by default raises an error if you even try to touch the timestamps of a directory which has a file handle open somewhere deep inside inside.
However Windows doesn't need to be so conservative. If you open all your file handles with FILE_FLAG_BACKUP_SEMANTICS and FILE_SHARE_DELETE and take care to rename files to random names just before flagging deletion, you get POSIX semantics including lack of restriction on manipulating file paths containing open file handles.
One very nifty thing Windows can do is to perform renames and deletes and hard links only using an open file descriptor, and therefore you can delete a file without having to worry about whether another process has renamed it or any of the directories in the path preceding the file's location. This facility lets you perform completely race free file deletions - once you have an open file handle to the right file, you can stop caring about what other processes are doing to the filing system, at least for deletion (which is the most important as it implicitly involves destroying data).
This raises the question of what about POSIX? On POSIX unlink() takes a path, and between retrieving the current path of a file descriptor using /proc/self/fd/x or F_GETPATH and calling unlink() someone may have changed that path, thus potentially leading to the wrong file being unlinked and data lost.
A considerably safer solution is this:
Get one of the current paths of the open file descriptor using /proc/self/fd/x or F_GETPATH etc.
Open its containing directory.
Do a statat() on the containing directory for the leafname of the open file descriptor, checking if the device ids and inodes match.
If they match, do an unlinkat() to remove the leafname.
This is race safe from the parent directory upwards, though the hard link you delete may not be the one expected. However, it is not race safe if within the containing directory a third party process were to rename your file to something else and rename another file to your leafname between you checking for inode equivalence and calling the unlinkat(). Here the wrong file could be deleted, and data lost.
I therefore ask the question: can POSIX, or any specific POSIX implementation such as Linux, allow programs to unlink file entries completely race free? One solution could be to unlink a file entry by open file descriptor, another could be to unlink a file entry by inode, however google has not turned up solutions for either of those. Interestingly, NTFS does let you delete by a choice of inode or GUID (yes NTFS does provide inodes, you can fetch them from the NT kernel) in addition to deletion via open file handle, but that isn't much help here.
In case this seems like too esoteric a question, this problem affects proposed Boost.AFIO where I need to determine what filing system races I can mitigate and what I cannot as part of its documented hard behaviour guarantees.
Edit: Clarified that there is no canonical current path of an open file descriptor, and that in this use case we don't care - we just want to unlink some one of the links for the file.
No replies to this question, and I have spent several days trawling through Linux source code. I believe the answer is "currently you can't unlink a file race free", so I have opened a feature request at https://bugzilla.kernel.org/show_bug.cgi?id=93441 to have Linux extend unlinkat() with the AT_EMPTY_PATH Linux extension flag. If they accept that idea, I'll mark this answer as the correct one.
Up to now when I have to move files across shares I use code like below:
Copy file to target using File.Copy method
Check if file is successfully copied by using File.Exists
Delete the source
I have used the above process because I thought that you could not use File.Move to move files across shares or volumes. After digging a bit today I realised that you can do that so my next concern was how are failures handled i.e. does the underlying .NET code actually copy and then delete the files?
I checked for the File.Move code at reference source and see that it actually calls the windows MoveFile API.
So my question is, can I safely used File.Move to move files across UNC paths?
I could be wrong, but it seems that you do get delete-after-copy semantics.
MoveFile says that
To specify how to move the file, use the MoveFileEx or
MoveFileWithProgress function. To perform this operation as a
transacted operation, use the MoveFileTransacted function.
This implies that MoveFile does not provide either transaction semantics or guarantees as to "how the file is moved" (and there is nothing else in the documentation that refers to such).
MoveFileEx, on the other hand, states that
When moving a file, the destination can be on a different file system
or volume. If the destination is on another drive, you must set the
MOVEFILE_COPY_ALLOWED flag in dwFlags.
The effect of MOVEFILE_COPY_ALLOWED is
If the file is to be moved to a different volume, the function
simulates the move by using the CopyFile and DeleteFile functions. If
the file is successfully copied to a different volume and the original
file is unable to be deleted, the function succeeds leaving the source
file intact.
The way I interpret this is that since MoveFileEx does not give you the option of doing things otherwise when moving across volumes, it stands to reason that MoveFile also works in that mode as well.
In addition, moving across volumes means possibly moving across filesystems and even machines (especially when using UNC paths). In this situation there is no way to place the file in its destination path other than incrementally copying its contents, which seems to pretty much guarantee that the operation will be a delete-after-copy.