I have a request and a problem.
I have archived files
tar xvpf /to_arch |gzip - c | split -b10000m - /arch/to_arch.gz_
I use this comand. this is archive got my system and i need move it on other server.
on nev server i havent space for put arhive and extract it then i have idea.
Can someone help me write script in bash who can remuve extracted files.
like to_arch.gz_aa to_arch.gz_abto_arch.gz_acto_arch.gz_ad etc.
if finish extract aa file then script delete it.
cat *.gz* | tar zxvf - -i
Normaly i extract that but havent space on disk.
Related
I have a tar file which is 3.1 TB(TeraByte)
File name - Testfile.tar
I would like to split this tar file into 2 parts - Testfil1.tar and Testfile2.tar
I tried the following so far
split -b 1T Testfile.tar "Testfile.tar"
What i get is Testfile.taraa(what is "aa")
And i just stopped my command. I also noticed that the output Testfile.taraa doesn't seem to be a tar file when I do ls in the directory. It seems like it is a text file. May be once the full split is completed it will look like a tar file?
The behavior from split is correct, from man page online: http://man7.org/linux/man-pages/man1/split.1.html
Output pieces of FILE to PREFIXaa, PREFIXab, ...
Don't stop the command let it run and then you can use cat to concatenate (join) them all back again.
Examples can be seen here: https://unix.stackexchange.com/questions/24630/whats-the-best-way-to-join-files-again-after-splitting-them
split -b 100m myImage.iso
# later
cat x* > myImage.iso
UPDATE
Just as clarification since I believe you have not understood the approach. You split a big file like this to transport it for example, files are not usable this way. To use it again you need to concatenate (join) pieces back. If you want usable parts, then you need to decompress the file, split it in parts and compress them. With split you basically split the binary file. I don't think you can use those parts.
You are doing the compression first and the partition later.
If you want each part to be a tar file, you should use 'split' first with de original file, and then 'tar' with each part.
Recently MaxMind changed their download policy, and the old simple format is no longer available. The new file format looks like this: GeoLite2-Country_20191231.tar.gz, and inside we have a folder with the same name containing two additional files.
Although there is an option to delete the date parameter from the link, it seems that the downloaded file will still contain the date.
Now, the problem is to extract that GeoLite2-Country.mmdb from the gzip file having that variable name programmatically.
The unzip part existing in my old script was this:
gunzip -c "$1"GeoLite2-Country.mmdb.gz > "$1"GeoLite2-Country.mmdb
The question is how to modify the above part for the new situation. Or, maybe someone knows another way to solve the same problem. Thanks in advance.
The folder structure:
-+ Geolite2-Country_YYYYMMDD.tar.gz:
|-+ Geolite2-Country_YYYYMMDD
|- licence.txt
|- copyright.txt
|- Geolite2-Country.mmdb
What I need is Geolite2-Country.mmdb in the current folder of gzip file.
tar -tf /GeoLite2-City.tar.gz | grep mmdb | xargs tar -xf /GeoLite2-City.tar.gz --strip-components 1 -C /
Just fix source and destination paths
I have a ZIP file, not succeeded to unzip for some-reason, saying "invalid or incomplete multibytes or wide character". So I unzip -p myfile.zip > Messed.data , I want to separate them, with script.
unzip -l to get the files-size.
dd ibs=1 skip=$((sum of front file-size)) count=$((this-file-size))
I tried and found the speed was unbearable slow.
So I ask for any help to this. Thank you.
I'm trying to decompress ~8GB .zip file piped from curl command. Everything I have tried is being interrupted at <1GB and returns a message:
... has more than one entry--rest ignored
I've tried: funzip, gunzip, gzip -d, zcat, ... also with different arguments - all end up in the above message.
The datafile is public, so it's easy to repro the issue:
curl -L https://archive.org/download/nycTaxiTripData2013/faredata2013.zip | funzip > datafile
Are you sure the mentioned file deflates to a single file? If it extracts to multiple files you unfortunately cannot unzip on the fly.
Zip is a container as well as compression format and it doesn't know where the new file begins. You'll have to download the whole file and unzip it.
So here is what i do already,
i have a abc.txt which contains list of files.Am using abc.txt to
move those files to a folder , tar that folder and finally i download the tar to local pc from server(linux).
it goes like
1.abc.txt
2.abc.txt(files) -> folder
3.Folder -> folder.tar
4.folder.tar -> local pc.
Now i need to change this like below,
if abc.txt contains 2 files namely,
example1.css
example2.css
i need to download those files from abc.txt seperately and directly to local pc ,
since ftp or sftp need the file name to download it how can i read that
from abc.txt.
Please help.
I think the hub of your problem is how to extract the correct files from your list for your subsequent two logic paths.
egrep 'example1.css|example2.css' abc.txt
will give you all lines that match the exceptions, and
egrep -v 'example1.css|example2.css' abc.txt
will give you all lines that don't match the exceptions