I have the original images in a directory structure that looks like this:
./Alabama/1.jpg
./Alabama/2.jpg
./Alabama/3.jpg
./Alaska/1.jpg
...the rest of the states...
I wanted to convert all of the original images into thumbnails so I can display them on a website. After a bit of digging / experimenting, I came up with the following Linux command:
find . -type f -iname '*.jpg' | sed -e 's/\.jpg$//' | xargs -I Y convert Y.jpg -thumbnail x100\> Y-small.jpg
It recursively finds all the jpg images in my subdirectories, removes the file type (.jpg) from them so I can rename them later, then makes them into a thumbnail and renames them with '-small' appended before the file type.
It worked for my purposes, but its a tad complicated and it isn't very robust. For example, I'm not sure how I would insert 'small-' at the beginning of the file's name (so ./Alabama/small-1.jpg).
Questions:
Is there a better, more robust way of creating thumbnails from images that are located in multiple subdirectories?
Can I make the existing command more robust (for example, but using sed to rename the outputted thumbnail before it is saved- basically modify the Y-small.jpg part).
No need to build up such a complicated construct. Make small blocks first, then connect.
I have decided to not insert -small in the middle of the filename. it makes everything more complicated. Better to use a prefix (simpler code, and also easier to derive the original filename from the thumb) which can either be a thumbsDir/ or a thumbsPrefix-.
#!/bin/sh
make_thumbnail() {
pic=$1
thumb=$(dirname "$1")/thumb-$(basename "$1")
convert "$pic" -thumbnail x100 "$thumb"
}
# Now we need a way to call make_thumbnail on each file.
# The easiest robust way is to restrict on files at level 2, and glob those
# with */*.jpg
# (we could also glob levels 2 and 3 with two globs: */*.jpg */*/*.jpg)
for pic in */*.jpg
do
make_thumbnail "$pic"
done
Related
How do I use file command for copying the files in a directory according to their type? I know I can use file to find the type of the file, but I don't know how to use it it the if condition.
What I want to achieve is this. I need to tidy up my downloads folder. When I run the specific script, I want the files in the mentioned folder to be moved into a dedicated folder, according to its type. For eg, image files should be moved to a folder named "Images", video files to "Videos", executables to "Programs" and so on.
Something like this?
for filename in ./*; do
case $(file -b -i "$filename") in
inode/directory* | inode/symlink*)
echo "$0: skip $filename" >&2
continue;;
application/*) dest=Random;;
image/*) dest=Images;;
text/html*) dest=Webpages;;
text/plain*) dest=Documents;;
video/*) dest=Videos;;
*) dest=Unknown;;
esac
mkdir -p "$dest"
mv "$filename" "$dest/"
done
The mapping of MIME types (-i option) to your hierarchy of directories isn't entirely straightforward. The application MIME type hierarchy in particular corresponds to a vast number of document types (PDF, Excel, etc) - some of which also have designated types - as well as the completely unspecified generic application/octet-stream. Using something else than MIME types is often even more problematic, as the labels that file prints are free-form human-readable text which can be essentially random (for example, different versions of the same file format may correspond to different detections with different labels, which are not systematically formatted, and so you might get Evil Empire Insult (tm) format 1997 from one file and Insult 2000 from another with the same extension).
Probably do a test run with file -i ./* and examine the results you get, then update the code above with cases which actually make sense for your specific files.
I need to find hardlink files under specific folder in Solaris. Tried this below command which lists the files based on inode count.
find . -type f -links +1
The above command list both source and target files. But i need to list only the target_file.
For Eg: Under Test folder, there is source.txt
Test
->source.txt
Created hardlink:
ln source.txt target.txt
The above find command return both source.txt and target.txt. But I need a command to fetch only target.txt. Is it possible?
No. After the hardlink both names of the file are equal in all ways, there is no original or copy.
Since they share the underlying inode, both files have the same attributes -- change one you change all of them.
Either switch to symbolic links or find a heuristic to choose which one you don't want to see, like it has an extension, or sorts later.
So I want to backup all my music to an external hard drive. This worked well for the most part using Grsync, but some didn't copy over because of encoding issues with the file name.
I would like to compare my two music directories (current and backup) to see what files were missed, so I can copy these over manually.
What is a good solution for this? Note there are many many files, so ideally I don't want a tool that wastes time comparing the file contents. I just need to know if a file is missing from the backup that is in the source.
Are there good command line or gui solutions that can do this in good time?
Go to the top level directory in each set.
find . -type f -print | sort > /tmp/listfile.txt
Set up a sorted list for each directory, and diff should help you spot the differences.
I'm interested in simply adding a comment next to my files in Linux (Ubuntu). An example would be:
info user ... my_data.csv Raw data which was sent to me.
info user ... my_data_cleaned.csv Raw data with duplicates filtered.
info user ... my_data_top10.csv Cleaned data with only top 10 values selected for each ID.
So sort of the way you can comment commits in Git. I don't particularly care about searching on these tags, filtering them etc. Just seeings them when I list files in a directory. Bonus if the comments/tags follow the document around as I copy or move it.
Most filesystem types support extended attributes where you could store comments.
So for example to create a comment on "foo.file":
xattr -w user.comment "This is a comment" foo.file
The attributes can be copied/moved with the file just be aware that many utilities require special options to copy the extended attributes.
Then to list files with comments use a script or program that grabs the extended attribute. Here is a simple example to use as a starting point, it just lists the files in the current directory:
#!/bin/sh
ls -1 | while read -r FILE; do
comment=`xattr -p user.comment "$FILE" 2>/dev/null`
if [ -n "$comment" ]; then
echo "$FILE Comment: $comment"
else
echo "$FILE"
fi
done
The xattr command is really slow and poorly written (it doesn't even return error status) so I suggest something else if possible. Use setfattr and getfattr in a more complex script than what I have provided. Or maybe a custom ls command that is aware of the user.comment attribute.
This is a moderately serious challenge. Basically, you want to add attributes to files, keep the attributes when the file is copied or moved, and then modify ls to display the values of these attributes.
So, here's how I would attack the problem.
1) Store the information in a sqlLite database. You can probably get away with one table. The table should contain the complete path to the file, and your comment. I'd name the database something like ~/.dirinfo/dirinfo.db. I'd store it in a subfolder, because you may find later on that you need other information in this folder. It'd be nice to use inodes rather than pathnames, but they change too frequently. Still, you might be able to do something where you store both the inode and the pathname, and retrieve by pathname only if the retrieval by inode fails, in which case you'd then update the inode information.
2) write a bash script to create/read/update/delete the comment for a given file.
3) Write another bash function or script that works with ls. I wouldn't call it "ls" though, because you don't want to mess with all the command line options that are available to ls. You're going to be calling ls always as ls -1 in your script, possibly with some sort options, such as -t and/or -r. Anyway, your script will call ls -1 and loop through the output, displaying the file name, and the comment, which you'll look up using the script from 2). You may also want to add file size, but that's up to you.
4) write functions to replace mv and cp (and ln??). These would be wrapper functions that would update the information in your table, and then call the regular Unix versions of these commands, passing along any arguments received by the functions (i.e. "$#"). If you're really paranoid, you'd also do it for things like scp, which can be used (inefficiently) to copy files locally. Still, it's unlikely you'll catch all the possibilities. What if someone else does a mv on your file, who doesn't have the function you have? What if some script moves the file by calling /bin/mv? You can't easily get around these kinds of issues.
Or if you really wanted to get adventurous, you'd write some C/C++ code to do this. It'd be faster, and honestly not all that much more challenging, provided you understand fork() and exec(). I can't recall whether sqlite has a C API. I assume it does. You'd have to tangle with that, too, but since you only have one database, and one table, that shouldn't be too challenging.
You could do it in perl, too, but I'm not sure that it would be that much easier in perl, than in bash. Your actual code isn't that complex, and you're not likely to be doing any crazy regex stuff or string manipulations. There are just lots of small pieces to fit together.
Doing all of this is much more work than should be expected for a person answering a question here, but I've given you the overall design. Implementing it should be relatively easy if you follow the design above and can live with the constraints.
My web application creates multiple image thumbnail files when users upload images.
I want to separate original images and thumbnail images. Thumbnail images contain 'crop-smart' in their file name.
For example, original image is watermelon.jpg, then thumbnail's name is watermelon_jpg_120x120_crop-smart.jpg.
How do I find by say 'crop-smart' and either move them to the different folder or delete them?
Standard file globbing will do this, the exact details may vary depending out which shell you are running but for your exact problem, it should be the same:
mv -- *_crop-smart.jpg /path/to/new/folder/
(This will also work if you have spaces in the filename)
Note the -- signals to mv that no more option switches will follow, so even if filenames look like options, mv won't get confused.