Currently I try to use:
"wget --user=xxx --password=xxx -r ftp://www.domain.com/htdocs/"
But this saves output files to current directory in this fashion:
curdir/www.domain.com/htdocs/*
I need it to be:
curdir/*
Is there a way to do this, I only see a way to use output prefix, but i think this will just allow me to define directory outside current dir?
You can combine --no-directories if you want all your files inside one directory or --no-host-directories to have subdirectories but no subdirectories per host with your --directory-prefix option.
2.6 Directory Options
‘-nd’
‘--no-directories’
Do not create a hierarchy of directories when retrieving recursively. With this option turned on, all files will get saved to the current directory, without clobbering (if a name shows up more than once, the filenames will get extensions ‘.n’).
‘-nH’
‘--no-host-directories’
Disable generation of host-prefixed directories. By default, invoking Wget with ‘-r http://fly.srk.fer.hr/’ will create a structure of directories beginning with fly.srk.fer.hr/. This option disables such behavior.
‘-P prefix’
‘--directory-prefix=prefix’
Set directory prefix to prefix. The directory prefix is the directory where all other files and subdirectories will be saved to, i.e. the top of the retrieval tree. The default is ‘.’ (the current directory).
(From the wget manual.)
Related
When creating filepaths and URLs, I noticed that many times the path starts with ./ or ~/.
What is the difference between filepaths that start with ./ and ~/?
What do each of them mean?
For the sake of completeness ...
Just path is a file or directory named path in the current directory.
./path is a file or directory named path in the current directory, with the directory spelled out. The dot directory . represents the current directory, and path is the name of the file or directory within this directory.
~/path is a shorthand for $HOME/path where $HOME is a variable which refers to your home directory. Typically your home directory will be somewhere like /home/you or /Users/you where you is your account name. (The command echo "$HOME" will display your home directory.) The expanded value is an absolute path (unless you have messed up the value of $HOME thoroughly), as indicated by the initial slash.
/path is an absolute path which refers to a file or directory named path which is in the root directory /. Every file on Unix is ultimately somewhere in the directory tree which starts with the root directory.
A file name which begins with $ includes the value of a shell variable in its name (like for example $HOME above); you have to know the value of that variable to determine whether it ends up containing a relative or an absolute path. Similarly, ~ at the beginning of a file name gets replaced ("expanded") by the shell to a different string, as outlined above.
(Technically, it's possible for a file name to begin with a literal dollar sign or tilde, too; you would then have to quote or backslash-escape that character to avoid having the shell expand it to something else. This is rather inconvenient, so these file names tend to be rare in practice.)
In the following exposition, we refer to the result of any such replacements, and ignore the complication of possible quoting.
Every file name which begins with / is an absolute path (aka full path) which explains how to reach a particular node starting from the root directory. For example, /var/tmp/you/reminder.txt refers to a file or directory reminder.txt (probably a file, judging from the name; but Unix doesn't care what you call your files or directories) which is in the directory you which is in the directory tmp which is in the directory var which is in the root directory.
Every file name which doesn't begin with / is a relative path which indicates how to reach a particular file or directory starting from the current directory. The special directory .. is the parent directory (that is, the directory which contains this directory) and the special directory . is the current directory. So path/there refers to the file or directory there inside the directory path in the current directory; and (hover the mouse over the gray area to display the spoiler)
there/.././and/back/.. is a (wicked complicated) way to refer to the directory and in the current directory, where we traverse the there directory and then move back to the current directory; then stay in the current directory; then refer to the directory back inside the directory and, but then move back to the parent directory of that, ending up with ./and.
In addition to ~/ for the current user's home directory, some shells and applications allow the notation ~them/ to refer to the home directory of the user account them. Also, some web server configurations allow each user to have a public web site in their directory ~/public_html and the URL notation http://server/~them/ would serve up the site of the user account them for outside visitors.
The current directory is a convenience which the shell provides so you don't have to type long paths all the time. You can, if you want to.
/bin/ls /home/you/Documents/unix-101/directories.txt
is a longwinded but perfectly valid way to say (assuming you are in your home directory),
ls Documents/unix-101/directories.txt
You could also say
cd Documents/unix-101
ls directories.txt
and until you cd again, all your commands will run in this directory.
See What exactly is current working directory? for a longer exposition of this related concept.
A "directory" is sometimes called a "folder" by people who are not yet old enough to prefer the former.
Tangentially, don't confuse the directory name . with the Bourne shell command which comprises a single dot (also known by its Bash alias source). The command
. ./scriptname
runs the commands from the file ./scriptname in the context of the current shell instance, as opposed to in a separate subshell (which is what just ./scriptname does). In other words, this command line invokes the dot command on a file scriptname in the dot directory.
The Bourne shell (and derivatives like Bash, Zsh, etc) use single quotes to prevent variable expansion and wildcard expansion, and double quotes to permit variable expansion, but inhibit wildcard expansion in a string. The quoting rules on Windows are different, and generally use double quotes to keep whitespace-separated values as a single string (and % instead of $ for variable substitutions).
./ means "starting from the current directory". . refers to the current working directory, so something like ./foo.bar would be looking for a file called foo.bar in the current directory. (As a side note, .. means refers to the parent directory of the current directory. So ../foo.bar would be looking for that file one directory above.)
~/ means "starting from the home directory". This could have different meanings in different scenarios. For example, in a Unix environment ~/foo.bar would be looking for a file called foo.bar in your home directory, something like /home/totzam/foo.bar. In many web applications, ~/foo.bar would be looking for a file called foo.bar in the web application root, something like /var/http/mywebapp/foo.bar.
./ is the current directory
~/ is the home directory of the current user
./ means that path is relative to your current position.
~/ means that path is relative to your home directory.
I will explain a simple example of it. As developers mentioned:
./ is current directory.
~/ is the home directory of the current user.
How both of the file path expressions can help us? Suppose you want to execute a script (.sh) and you're in the same directory where file exists then you can simply do it ./filename.sh
I mostly use ~/ to access my home directory files like .bashrc when I want to add any config in it. It's easier since the file path expression (for home directory) feels much easier and makes accessibility to the file from anywhere, without worrying about the path or changing the path.
. represents current directory
.. represents the parent directory
~ represents the home directory for the current user. Home directory is also represented by HOME env variable. you can do echo $HOME on the shell to see it.
These are generally used to specify relative paths. The / in the end of each notation is a separator that you can use when using these notations together.
Ex:
$ cd ../.. # Go 2 directories backwards
$ cd ~ # Takes you to $HOME directory
$ cd . # Does nothing :) As it literally means go to the directory that you are already present in.
$ cd ~/dir1 $ go to `$HOME/dir1`
On Unix, in any directory if you do ls -a you would see that . and .. will be mentioned (even for empty directory). Like mentioned, these have special meaning and are generated by default in Unix systems and are generally helpful to specify relative paths (i.e, path to a different directory relative to your current directory)
cd command is harmless. So, just play around by combining notations with cd command. You will eventually get a grip of them.
I know there has been a huge discussion about this but I have not found something this specific.
Im trying to copy all .key files in /home// directory
This does not work
/usr/bin/rsync -auPA --include="*/*.key" --exclude="*" /home/* /tmp/test
This works but it copies over unwanted empty directories like /home/uname/Documents
/usr/bin/rsync -auPA --include="*/" --include="*.key" --exclude="*" /home /tmp/test
Basically what i need for rsync to do is to copy only files with .key extension and only create necessarily folders that contain .key files
I think you are looking for the -m option. From the man page:
-m, --prune-empty-dirs
This option tells the receiving rsync to get rid of empty directories from the file-list, including nested directories that
have no non-directory children. This is useful for avoiding the creation of a bunch of useless directories when the sending
rsync is recursively scanning a hierarchy of files using include/exclude/filter rules.
Note that the use of transfer rules, such as the --min-size option, does not affect what goes into the file list, and thus
does not leave directories empty, even if none of the files in a directory match the transfer rule.
Because the file-list is actually being pruned, this option also affects what directories get deleted when a delete is active.
However, keep in mind that excluded files and directories can prevent existing items from being deleted due to an exclude both
hiding source files and protecting destination files. See the perishable filter-rule option for how to avoid this.
You can prevent the pruning of certain empty directories from the file-list by using a global "protect" filter. For instance,
this option would ensure that the directory "emptydir" was kept in the file-list:
--filter ’protect emptydir/’
Here’s an example that copies all .pdf files in a hierarchy, only creating the necessary destination directories to hold the
.pdf files, and ensures that any superfluous files and directories in the destination are removed (note the hide filter of
non-directories being used instead of an exclude):
rsync -avm --del --include=’*.pdf’ -f ’hide,! */’ src/ dest
If you didn’t want to remove superfluous destination files, the more time-honored options of "--include='*/' --exclude='*'"
would work fine in place of the hide-filter (if that is more natural to you).
I'm trying to create a file structure in unix using the relative path, however the directories are being created under my current directory? What am i doing wrong.
mkdir ../folder1 -p folder2/folder3 folder2/folder4 folder5
.. goes back to my home and /folder should go into folder1, but all of this is being done my current directory?
All the filename arguments to mkdir are simply directories to create, they're not interpreted relative to each other. Putting ../folder1 first doesn't make it use that as the starting directory for all the rest.
You can use brace expansion to concatenate all the subdirectory paths to a single directory prefix.
mkdir -p ../folder1/{folder2/{folder3,folder4},folder5}
Also, remember that options usually go before filename arguments.
When I try the command ls with -aF option inside any directory whether it's empty or not, I always got the following:
./ ../
so what does the output mean when I have these two options together with ls command?
When you use ls, you are reading a directory file, not actually looking in a directory. Every directory listing contains an entry for the present/current directory, as well as its parent directory, just as you would expect to also see listings for sub/child directories in any directory listing.
The -A option for ls merely tells ls to display ALL files, which includes the entries of ./ & ../ for present and parent. Note that these dots are merely a shorthand that the shell (bash) uses to represent file paths for those files. In other words, what "./" really means is say ~/Desktop if you were currently in the Desktop directory doing an ls. And "../" would mean "~/" which is just another symbolic shorthand to represent your user home directory, which is probably something like /Users/your_username on macOS (OS X), or /usr/your_username for various Linux distributions. Note that those paths could also be written with the forward slash appended at the end and would mean the same thing (e.g., /Users/your_username/ is the same as /Users/your_username because they are both references to other directories (directory files).
Use the -a option for ls if you don't want to see ./ & ../, but still want to see (other) hidden files.
Using the -F option causes ls to display appended characters to the file types based on the file type. This is why directories are displayed with the forward slash appended at the end, and executables are displayed as executable* (with the asterisk appended), and regular files have no appendage (e.g., .txt, .png, .dmg).
Is there a way not to symlink one or two files within a symlinked directory in CentOS?
I've got the entire directory symlinked but there are two css files that I would like to use the current copy for the website
In short: no.
Another way to do this would be to symlink all the files in that directory, except those you want local copy of.
Still another way to go might be using unionfs or aufs to union-mount the original directory and a directory containing the files you need local, with the directory containing local files being "on top".
Say, your original directory is orig, the directory with files that should be local is local, the union directory is union, and you want files from both directories to be writable. Then you can union-mount them like this:
unionfs-fuse local=RW:orig=RW union
And unmount like this:
fusermount -u union
See unionfs manpage (unionfs-fuse(8) at least on Debian) for details.