After looking all over the Internet, I've come to this.
Let's say I have already made a text file that reads:
Hello World
Well, I want to remove the very last character (in this case d) from this text file.
So now the text file should look like this: Hello Worl
But I have no idea how to do this.
All I want, more or less, is a single backspace function for text files on my HDD.
This needs to work on Linux as that's what I'm using.
Use fileobject.seek() to seek 1 position from the end, then use file.truncate() to remove the remainder of the file:
import os
with open(filename, 'rb+') as filehandle:
filehandle.seek(-1, os.SEEK_END)
filehandle.truncate()
This works fine for single-byte encodings. If you have a multi-byte encoding (such as UTF-16 or UTF-32) you need to seek back enough bytes from the end to account for a single codepoint.
For variable-byte encodings, it depends on the codec if you can use this technique at all. For UTF-8, you need to find the first byte (from the end) where bytevalue & 0xC0 != 0x80 is true, and truncate from that point on. That ensures you don't truncate in the middle of a multi-byte UTF-8 codepoint:
with open(filename, 'rb+') as filehandle:
# move to end, then scan forward until a non-continuation byte is found
filehandle.seek(-1, os.SEEK_END)
while filehandle.read(1) & 0xC0 == 0x80:
# we just read 1 byte, which moved the file position forward,
# skip back 2 bytes to move to the byte before the current.
filehandle.seek(-2, os.SEEK_CUR)
# last read byte is our truncation point, move back to it.
filehandle.seek(-1, os.SEEK_CUR)
filehandle.truncate()
Note that UTF-8 is a superset of ASCII, so the above works for ASCII-encoded files too.
Accepted answer of Martijn is simple and kind of works, but does not account for text files with:
UTF-8 encoding containing non-English characters (which is the default encoding for text files in Python 3)
one newline character at the end of the file (which is the default in Linux editors like vim or gedit)
If the text file contains non-English characters, neither of the answers provided so far would work.
What follows is an example, that solves both problems, which also allows removing more than one character from the end of the file:
import os
def truncate_utf8_chars(filename, count, ignore_newlines=True):
"""
Truncates last `count` characters of a text file encoded in UTF-8.
:param filename: The path to the text file to read
:param count: Number of UTF-8 characters to remove from the end of the file
:param ignore_newlines: Set to true, if the newline character at the end of the file should be ignored
"""
with open(filename, 'rb+') as f:
last_char = None
size = os.fstat(f.fileno()).st_size
offset = 1
chars = 0
while offset <= size:
f.seek(-offset, os.SEEK_END)
b = ord(f.read(1))
if ignore_newlines:
if b == 0x0D or b == 0x0A:
offset += 1
continue
if b & 0b10000000 == 0 or b & 0b11000000 == 0b11000000:
# This is the first byte of a UTF8 character
chars += 1
if chars == count:
# When `count` number of characters have been found, move current position back
# with one byte (to include the byte just checked) and truncate the file
f.seek(-1, os.SEEK_CUR)
f.truncate()
return
offset += 1
How it works:
Reads only the last few bytes of a UTF-8 encoded text file in binary mode
Iterates the bytes backwards, looking for the start of a UTF-8 character
Once a character (different from a newline) is found, return that as the last character in the text file
Sample text file - bg.txt:
Здравей свят
How to use:
filename = 'bg.txt'
print('Before truncate:', open(filename).read())
truncate_utf8_chars(filename, 1)
print('After truncate:', open(filename).read())
Outputs:
Before truncate: Здравей свят
After truncate: Здравей свя
This works with both UTF-8 and ASCII encoded files.
In case you are not reading the file in binary mode, where you have only 'w' permissions, I can suggest the following.
f.seek(f.tell() - 1, os.SEEK_SET)
f.write('')
In this code above, f.seek() will only accept f.tell() b/c you do not have 'b' access. then you can set the cursor to the starting of the last element. Then you can delete the last element by an empty string.
with open(urfile, 'rb+') as f:
f.seek(0,2) # end of file
size=f.tell() # the size...
f.truncate(size-1) # truncate at that size - how ever many characters
Be sure to use binary mode on windows since Unix file line ending many return an illegal or incorrect character count.
with open('file.txt', 'w') as f:
f.seek(0, 2) # seek to end of file; f.seek(0, os.SEEK_END) is legal
f.seek(f.tell() - 2, 0) # seek to the second last char of file; f.seek(f.tell()-2, os.SEEK_SET) is legal
f.truncate()
subject to what last character of the file is, could be newline (\n) or anything else.
This may not be optimal, but if the above approaches don't work out, you could do:
with open('myfile.txt', 'r') as file:
data = file.read()[:-1]
with open('myfile.txt', 'w') as file:
file.write(data)
The code first opens the file, and then copies its content (with the exception of the last character) to the string data. Afterwards, the file is truncated to zero length (i.e. emptied), and the content of data is saved to the file, with the same name.
This is basically the same as vins ms's answer, except that it doesn't use the os package, and that is used the safer 'with open' syntax. This may not be recommended if the text file is huge. (I wrote this since none of the above approaches worked out too well for me in python 3.8).
here is a dirty way (erase & recreate)...
i don't advice to use this, but, it's possible to do like this ..
x = open("file").read()
os.remove("file")
open("file").write(x[:-1])
On a Linux system or (Cygwin under Windows). You can use the standard truncate command. You can reduce or increase the size of your file with this command.
In order to reduce a file by 1G the command would be truncate -s 1G filename. In the following example I reduce a file called update.iso by 1G.
Note that this operation took less than five seconds.
chris#SR-ENG-P18 /cygdrive/c/Projects
$ stat update.iso
File: update.iso
Size: 30802968576 Blocks: 30081024 IO Block: 65536 regular file
Device: ee6ddbceh/4000177102d Inode: 19421773395035112 Links: 1
Access: (0664/-rw-rw-r--) Uid: (1052727/ chris) Gid: (1049089/Domain Users)
Access: 2020-06-12 07:39:00.572940600 -0400
Modify: 2020-06-12 07:39:00.572940600 -0400
Change: 2020-06-12 07:39:00.572940600 -0400
Birth: 2020-06-11 13:31:21.170568000 -0400
chris#SR-ENG-P18 /cygdrive/c/Projects
$ truncate -s -1G update.iso
chris#SR-ENG-P18 /cygdrive/c/Projects
$ stat update.iso
File: update.iso
Size: 29729226752 Blocks: 29032448 IO Block: 65536 regular file
Device: ee6ddbceh/4000177102d Inode: 19421773395035112 Links: 1
Access: (0664/-rw-rw-r--) Uid: (1052727/ chris) Gid: (1049089/Domain Users)
Access: 2020-06-12 07:42:38.335782800 -0400
Modify: 2020-06-12 07:42:38.335782800 -0400
Change: 2020-06-12 07:42:38.335782800 -0400
Birth: 2020-06-11 13:31:21.170568000 -0400
The stat command tells you lots of info about a file including its size.
I use gforth running on linux boxes.
For one of my mini-applications I want to register a formatted text output from a few different user inputs.
Here is the INPUT$ I use:
: INPUT$
pad swap accept pad swap ;
I think this is correct. I tested it this way:
cr ." enter something : " 4 INPUT$ CR
enter something : toto
ok
cr ." enter something : " 8 INPUT$ CR
enter something : titi
ok
.S <4> 140296186274576 4 140296186274576 4 ok
My file definition:
256 Constant max-line
Create line-buffer max-line 2 + allot
//prepare file for Write permissions :
s" foo.out" w/o create-file throw Value fd-out
: close-output ( -- ) fd-out close-file throw ;
The end goal is to build very small files as:
data1;data2;data3
data4;data5;data6
where each data is the user input (asked 3times to insert text & a second wave of 3 inputs)
I did not find documentation about how I can use text inputs to build my file.
How can I call my stack data to copy them to the text file format? (using type will only echo texts to my terminal)
I think you are looking for the Forth write-file and write-line words, which are documented here: https://www.complang.tuwien.ac.at/forth/gforth/Docs-html/General-files.html
write-file ( c-addr u fileid -– ior )
write-line ( c-addr u fileid –- ior )
Pass the address and length of your text buffer, and the file ID (fd-out in your example) to write text to the file. The ior result will be zero on success.
i am re-writing a scull device driver.
I have written the open/read/write fns in driver code.
echo "hello" > /dev/myscull0 shows the data being written successfully to my device driver, and open() -> write() ->release() has been called in driver successfully.
Similarly, other operations using standard bash commands are successfull when operated on driver device file. Eg: cat /dev/myscull0 execute drivers read() call.
Now, i am writing a user space program to operate on my device file.
void
scull_open(){
char device_name[DEV_NAME_LENGTH];
int fd = 0;
memset(device_name, 0, DEV_NAME_LENGTH);
if((fgets(device_name, DEV_NAME_LENGTH -1, stdin) == NULL)){
printf("error in reading from stdin\n");
exit(EXIT_SUCCESS);
}
device_name[DEV_NAME_LENGTH -1] = '\0';
if ((fd = open(device_name, O_RDWR)) == -1) {
perror("open failed");
exit(EXIT_SUCCESS);
}
printf("%s() : Success\n", __FUNCTION__);
}
But i am seeing, drivers open() call is not being executed, confirmed from dmesg. I am running the program with sudo privileges, yet no succsess. I supply the input as /dev/myscull0
Infact, after executing the user program, i am seeing two entries in /dev dir
vm#vm:/dev$ ls -l | grep scull
crw-r--r-- 1 root root 247, 1 Feb 27 14:38 myscull0
---Sr-S--- 1 root root 0 Feb 27 14:38 myscull0
vm#vm:/dev$
The first entry was created by me using mknod command, however second entry is created with strange set of permissions after executing the user program.
Output :
/dev/myscull0
scull_open() : Success
Can anyone pls help what wrong i am doing here ?
This one is really doing my head in! please help!
So I have a little raspberry pi 2 running debian and node 0.12.6.
The node script listens to a home automation bus for events and saves these to a json list. This list is then saved to cvs using a cron job hourly (every min for testing) to a usb drive that is mounted at boot on the system.
The issue is that every time fs.writeFile is called it throws an error.
This only happens if I try and save to the usb drive, and not to local folders, which work fine.
I have altered the /etc/fstab file to include the following
/dev/sda1 /home/pi/knx/usb vfat user,auto,nodev,gid=pi,uid=pi,fmask=0111,dmask=0000 0 0
This appears to mount the usb at boot and I can as pi user make dirs and files no problem from the console.
The permissions of the folder (with the usb drive mounted at usb) look like this....
pi#raspberrypi ~/knx $ ls -l
total 24
-rw-r--r-- 1 pi pi 965 Nov 20 16:40 app.js
drwxr-xr-x 2 pi pi 4096 Nov 20 16:00 data_files
drwxr-xr-x 2 pi pi 4096 Nov 19 12:27 install_scripts
drwxr-xr-x 7 pi pi 4096 Nov 20 16:21 node_modules
drwxrwxrwx 8 pi pi 8192 Jan 1 1970 usb
The drive looks to be mounted ok....
pi#raspberrypi ~/knx $ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 15G 0 disk
└─sda1 8:1 1 15G 0 part /home/pi/knx/usb
The node script is ...
//set up chron job to save list
var job = new CronJob({
cronTime: '00 0-59 * * * *',
onTick: function() {
console.log('saving file');
json2csv({ data: listToSave, fields: fields }, function(err, csv) {
if (err) console.log(err);
tempfilename = new Date();
fname = './usb/data_files/'+ tempfilename.toString() +'.csv';
console.log(fname);
fs.writeFile(fname, csv, function(err) {
if (err) throw err;
console.log('file saved');
listToSave=[];
});
});
},
start: true,
});
The fname looks like this....
'./usb/data_files/Fri Nov 20 2015 16:42:00 GMT+0000 (UTC).csv'
If I had any hair - I would have well and truly pulled it out by now!
Any ideas on a solution?
It looks like the filename you are creating with "new Date()" is an invalid format for the filesystem.
I'm able to make your code work by formatting the raw date string into a friendlier file name:
var utils = require('util')
Above adds the util module for string formatting.
var today = new Date();
var fname = utils.format('./usb/%s-%s-%s.csv', today.getMonth(), today.getDay(), today.getHours());
This writes a "MM-dd.csv" filename which appears to work on my debian machine. You can extend this by adding the time, year , etc.
We do profiling kernel modules with Oprofile and there is a warning in the opreport as follow
warning: could not check that the binary file /lib/modules/2.6.32-191.el6.x86_64/kernel/fs/ext4/ext4.ko has not been modified since the profile was taken. Results may be inaccurate.
1591 samples % symbol name
1592 1622 9.8381 ext4_iget
1593 1591 9.6500 ext4_find_entry
1594 1231 7.4665 __ext4_get_inode_loc
1595 783 4.7492 ext4_ext_get_blocks
1596 752 4.5612 ext4_check_dir_entry
1597 644 3.9061 ext4_mark_iloc_dirty
1598 583 3.5361 ext4_get_blocks
1599 583 3.5361 ext4_xattr_get
May anyone please explain what is the warning, does it impact the accuracy of the oprofile output and is there anyway to avoid this warning?
Any suggestions are appreciated. Thank a lot!
Add more information:
In daemon/opd_mangling.c
if (!sf->kernel)
binary = find_cookie(sf->cookie);
else
binary = sf->kernel->name;
...
fill_header(odb_get_data(file), counter,
sf->anon ? sf->anon->start : 0, last_start,
!!sf->kernel, last ? !!last->kernel : 0,
spu_profile, sf->embedded_offset,
binary ? op_get_mtime(binary) : 0);
For kernel module file, the sf->kernel->name is the kernel module name, so the fill header will always fill mtime with 0 and generate the unwanted warning
This failure indicates that a stat of the file in question failed. Do an strace -e stat to see the specific failure mode.
time_t op_get_mtime(char const * file)
{
struct stat st;
if (stat(file, &st))
return 0;
return st.st_mtime;
}
...
if (!header.mtime) {
// FIXME: header.mtime for JIT sample files is 0. The problem could be that
// in opd_mangling.c:opd_open_sample_file() the call of fill_header()
// think that the JIT sample file is not a binary file.
if (is_jit_sample(file)) {
cverb << vbfd << "warning: could not check that the binary file "
<< file << " has not been modified since "
"the profile was taken. Results may be inaccurate.\n";
does it impact the accuracy of the oprofile output and is there anyway to avoid this warning?
Yes, it impacts the output in that it has no opportunity to warn you whether "the last modified time of the binary file does not match that of the sample file...". As long as you're certain what you measured then matches which binary is installed now, the warning you're seeing is harmless.