How do I rename multiple files beginning with a Unix timestamp - imapsync issue - linux

I didn't got the script from imapsync to rename maildir filenames to work. :-/
So what I need is:
I have a mail folder with thousands of mails. After importing those emails to my new server, the filename of the emails got the creation date as a Unix timestamp in the filename, but the creation date flag of the file is the correct receive date from the email.
ls -l for one file looks like this:
-rw-r--r-- 1 popuser popuser 1350432 2013-03-16 07:22 1363563215.M562903P29332V0000000000000802I0000000000AEA46B_527.my-domain.org,S=1350432:2,S
So what the script has to do is:
1) read the creation date/time of the file (I found the command
stat -c %y filename
does this)
2) convert the date/time from 1) to a Unix timestamp
date -d "2013-03-17 11:19:01.000000000 +0100" "+%s"
3) delete the first 10 digits (wrong timestamp) of the filename and us the the timestamp from 2) instead
4) do this for all files in a specific directory
I'm a newby in Linux scripts, can anyone help me with this script?
Thank you!

Try doing this with rename :
$ rename -n 's/^\d+/(stat($_))[9]/e' [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]*
from the shell prompt. It's very useful, you can put some perl code like I does in a substitution for stat with the e modifier.
You can remove the -n (dry-run mode switch) when your tests become valids.
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (linux)
$ file $(readlink -f $(type -p rename))
and you have a result like
.../rename: Perl script, ASCII text executable
and not containing:
ELF
then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo update-alternatives --set rename /path/to/rename
(replace /path/to/rename to the path of your perl's rename command.
If you don't have this command, search your package manager to install it or do it manually
Last but not least, this tool was originally written by Larry Wall, the Perl's dad.
Edit
As stated here, if you have the following error :
Argument list too long
Then use find like this :
find -type f -name '[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]*' -print0|
xargs -0 -n1 rename -n 's/^\d+/(stat($_))[9]/e'
(try it without -n1, that should works too)

Related

Shell script does not run completely when run by cron

The script in file modBackup.sh does not run completely when started by cron, the result is a corrupted tar.gz file that is half the size of this one if I run manually. In any case, its size is many times smaller than the one started manually, but still creates some content that can not be opened normally, archive is damaged
file modBackup.sh:
#!/bin/sh
find /home/share/ -mmin -720 -type f -exec tar -rvf /mnt/archives/`date +%F`-modified.tar.gz "{}" +
Тhe behavior of the automatic one seems to be interrupted and does not end.
When I run it manualy, the script creates a genuine archive as [current date]-modified.tar.gz
Here is the crontab -e:
00 18 * * 1-5 /home/myScripts/modBackup.sh
Edit:
There is no information in the logs except that crond has started
neither in the mail log, nor in the cron, nor in the messages
(I use very old CentOS :( but I don't think this is the reason for the error).
For testing only: I added %H%M of the file name in the script and did the following:
I ran it manually: sh /home/myScripts/modBackup.sh
and set with crontab -e to run a two minutes later the same command
After a few minutes, two files appeared that grew at the same time, but then the one created by cronjob
stopped growing
(two files).
I use the same GUI tool (Archive Manager) to open in both cases.
Тhe file, created by manually starting the script, opens (manually started), but the other one, from cronjob cannot, even after I changed the extension, the error is 'unexpected EOF in archive' (auto started)
Suggesting to include the users's environment context with $PATH and other critical environment variables for the application to work.:
modBackup.sh:
#!/bin/sh
source ~/.profile
find /home/share/ -mmin -720 -type f -exec tar -rvf /mnt/archives/`date +%F`-modified.tar.gz "{}" +
I found that in the cron environment the "find" command misinterprets filenames containing specific characters, even with the explicit change of the encoding with add at the beginning of the script line "export LANG = en_US.UTF-8; LC_CTYPE=...". With many other combinations and attempts I had no success.
That's why I left the "find" command and use the tar command with an option to archive modified files. This way works perfect now:
fromDate = $(date --date = '15 hours ago')
/bin/tar -N "$fromDate" -zcf /mnt/archives/`date +% F-% H% M`-share.modified.tar.gz /home/share/

How can I change the order of some string in a filename

I have lots of files like these:
tf_CVBV6Z_CVSA1Z_pws2_pcc1.sac
tf_CVBV5Z_CVSA2Z_pws2_pcc1.sac
tf_CVBV4Z_CVSA3Z_pws2_pcc1.sac
tf_CVBV3Z_CVSA4Z_pws2_pcc1.sac
tf_CVBV2Z_CVSA5Z_pws2_pcc1.sac
tf_CVBV1Z_CVSA6Z_pws2_pcc1.sac
and I want to change the order to end up like this:
tf_CVSA1Z_CVBV6Z_pws2_pcc1.sac
tf_CVSA2Z_CVBV5Z_pws2_pcc1.sac
tf_CVSA3Z_CVBV4Z_pws2_pcc1.sac
tf_CVSA4Z_CVBV3Z_pws2_pcc1.sac
tf_CVSA5Z_CVBV2Z_pws2_pcc1.sac
tf_CVSA6Z_CVBV1Z_pws2_pcc1.sac
I tried the rename option but it does not work.
Any thoughts?
Thanks
With perl's rename :
$ rename -n 's/(CV[^_]+)_(CV[^_]+)/$2_$1/' tf_CVSA1Z_CVBV6Z_pws2_pcc1.sac
tf_CVSA1Z_CVBV6Z_pws2_pcc1.sac -> tf_CVBV6Z_CVSA1Z_pws2_pcc1.sac
Remove -n switch when the output looks good.
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (GNU)
$ file "$(readlink -f "$(type -p rename)")"
and you have a result that contains Perl script, ASCII text executable and not containing ELF, then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo update-alternatives --set rename /path/to/rename
Replace /path/to/rename to the path of your perl rename executable.
If you don't have this command, search your package manager to install it or do it manually (no deps...)
This tool was originally written by Larry Wall, the Perl's dad.

How to replace double spaces with one space in filenames (also subdirectories) (CloudLinux Server release 6.10)

I want to replace double spaces with one space in the filenames of a lot of photos. These photos are located in directory /foto and it's subfolders. How to do this? For example "photo 1.jpg" needs to become "photo 1.jpg"
The best way is to use commandline, because it's on CloudLinux server. (and it is over 50GB of photos). I searched here on Stackoverflow, also Google to find the command I need. I guess rename is the one to use, or mv.
The only things I found were commands about replacing space and replacing other symbols, but not about double (multiple) spaces.
find -iname \*.* | rename -v "s/\s{2}/ /g"
This is the final command which helped me out. I used perl rename, see answer by Gilles
Use this, using Perl's rename :
rename 's/\s{2}/ /g' files*
Remove -n switch when the output looks good.
There are other tools with the same name which may or may not be able to do this, so be careful.
If you run the following command (GNU)
$ file "$(readlink -f "$(type -p rename)")"
and you have a result that contains Perl script, ASCII text executable and not containing ELF, then this seems to be the right tool =)
If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :
$ sudo update-alternatives --set rename /path/to/rename
Replace /path/to/rename to the path of your perl rename executable.
If you don't have this command, search your package manager to install it or do it manually (no deps...)
This tool was originally written by Larry Wall, the Perl's dad.

In Bash, how can I access the date of when a file was last edited?

Basically, I can see that
ls -l
displays the date that all the files in a directory were last edited. What I want to do is to access the date and then store it into a variable using a .sh script. However, not exactly sure what I can do to do so, and I've looked up the man pages for ls as well as searched up the matter to no avail.
If your date command supports the -r option, you can use it:
variable=$(date -r file)
The advantage of the date command is that you'll be able to format the date in many ways. For example, in seconds since Epoch:
variable=$(date -r file +%s)
Use stat with the correct format option:
stat -c%y file

Adding timestamp to a filename with mv in BASH

Well, I'm a linux newbie, and I'm having an issue with a simple bash script.
I've got a program that adds to a log file while it's running. Over time that log file gets huge. I'd like to create a startup script which will rename and move the log file before each run, effectively creating separate log files for each run of the program. Here's what I've got so far:
pastebin
DATE=$(date +"%Y%m%d%H%M")
mv server.log logs/$DATE.log
echo program
When run, I see this:
: command not found
program
When I cd to the logs directory and run dir, I see this:
201111211437\r.log\r
What's going on? I'm assuming there's some syntax issue I'm missing, but I can't seem to figure it out.
UPDATE: Thanks to shellter's comment below, I've found the problem to be due to the fact that I'm editing the .sh file in Notepad++ in windows, and then sending via ftp to the server, where I run the file via ssh. After running dos2unix on the file, it works.
New question: How can I save the file correctly in the first place, to avoid having to perform this fix every time I resend the file?
mv server.log logs/$(date -d "today" +"%Y%m%d%H%M").log
The few lines you posted from your script look okay to me. It's probably something a bit deeper.
You need to find which line is giving you this error. Add set -xv to the top of your script. This will print out the line number and the command that's being executed to STDERR. This will help you identify where in your script you're getting this particular error.
BTW, do you have a shebang at the top of your script? When I see something like this, I normally expect its an issue with the Shebang. For example, if you had #! /bin/bash on top, but your bash interpreter is located in /usr/bin/bash, you'll see this error.
EDIT
New question: How can I save the file correctly in the first place, to avoid having to perform this fix every time I resend the file?
Two ways:
Select the Edit->EOL Conversion->Unix Format menu item when you edit a file. Once it has the correct line endings, Notepad++ will keep them.
To make sure all new files have the correct line endings, go to the Settings->Preferences menu item, and pull up the Preferences dialog box. Select the New Document/Default Directory tab. Under New Document and Format, select the Unix radio button. Click the Close button.
A single line method within bash works like this.
[some out put] >$(date "+%Y.%m.%d-%H.%M.%S").ver
will create a file with a timestamp name with ver extension.
A working file listing snap shot to a date stamp file name as follows can show it working.
find . -type f -exec ls -la {} \; | cut -d ' ' -f 6- >$(date "+%Y.%m.%d-%H.%M.%S").ver
Of course
cat somefile.log > $(date "+%Y.%m.%d-%H.%M.%S").ver
or even simpler
ls > $(date "+%Y.%m.%d-%H.%M.%S").ver
I use this command for simple rotate a file:
mv output.log `date +%F`-output.log
In local folder I have 2019-09-25-output.log
Well, it's not a direct answer to your question, but there's a tool in GNU/Linux whose job is to rotate log files on regular basis, keeping old ones zipped up to a certain limit. It's logrotate
You can write your scripts in notepad but just make sure you convert them
using this ->
$ sed -i 's/\r$//' yourscripthere
I use it all they time when I'm working in cygwin and it works. Hope this helps
First, thanks for the answers above! They lead to my solution.
I added this alias to my .bashrc file:
alias now='date +%Y-%m-%d-%H.%M.%S'
Now when I want to put a time stamp on a file such as a build log I can do this:
mvn clean install | tee build-$(now).log
and I get a file name like:
build-2021-02-04-03.12.12.log

Resources