In broad terms, what I'd like is a direct tar-to-tar transformation where the result's root contains only a particular directory-subtree of the original.
To illustrate with an example, say I want only the gitweb directory from git's repository. Running
$ git archive --prefix=git-gitweb/ master gitweb | tar tf -
gives
git-gitweb/
git-gitweb/gitweb/
git-gitweb/gitweb/INSTALL
git-gitweb/gitweb/Makefile
git-gitweb/gitweb/README
git-gitweb/gitweb/gitweb.perl
git-gitweb/gitweb/static/
git-gitweb/gitweb/static/git-favicon.png
git-gitweb/gitweb/static/git-logo.png
git-gitweb/gitweb/static/gitweb.css
git-gitweb/gitweb/static/gitweb.js
but I want
git-gitweb/
git-gitweb/INSTALL
git-gitweb/Makefile
git-gitweb/...
The manual provides for extra backend-specific options, but attempting to pass --strip-components produces an error:
$ git archive --prefix=git-gitweb/ --strip-components=1 master gitweb | \
tar tf -
error: unknown option `strip-components=1'
usage: git archive [options] [...] ...
The tar backend isn't GNU tar anyway.
Someone in freenode's #gnu channel suggested Tardy, but it doesn't understand the output of git-archive:
$ git archive master > old.tar
$ tardy old.tar new.tar
tardy: old.tar: file type ``g'' unknown
Yes, I could extract the output of git-archive with --strip-components and create a new archive of the result, but I'm trying to avoid using the filesystem as a temporary variable.
Archive that directory as the CWD and you won't have the extra path component:
(cd gitweb; git archive --prefix=git-gitweb/ master .) | tar tf -
Or use this syntax as suggested by the git-archive man page:
git archive --prefix=git-gitweb/ master:gitweb | tar tf -
Related
I tried to include my custom helloword patch to build root.
in
make menuconfig
I have added global patch directory /home/Downloads/buildroot/buildroot-2017.11/patches
and I place my patch files inside the below directory
(/home/Downloads/buildroot/buildroot-2017.11/patches/packagename/version/patch).
I referred this link and this link
After make command the patch is not getting applied in my source directory, the source is getting extracted to the output/build directory from the .tar fle.
please suggest a solution..
Config.in
config BR2_PACKAGE_HELLOWORLD
bool "helloworld"
help
Hello World package says hello world to you
see http://helloworld.com for more on this software
helloworld.mk
HELLOWORLD_VERSION = 1.0.0
HELLOWORLD_SOURCE = helloworld-1.1.tar.gz
HELLOWORLD_PATCH = 18-helloworld-testing.patch
HELLOWORLD_SITE_METHOD = local
define HELLOWORLD_BUILD_CMDS
$(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(#D)
endef
define HELLOWORLD_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0755 $(#D)/helloworld $(TARGET_DIR)/usr/bin/helloworld
$(INSTALL) -D -m 0755 $(#D)/helloworld-init $(TARGET_DIR)/etc/init.d/S90helloworld
endef
$(eval $(generic-package))
With the local _SITE_METHOD, patches are not applied. local is when you want to use the files directly from their source directory. With the local _SITE_METHOD, the directory specified in HELLOWORLD_SITE will be copied to the build directory, no patches are applied, the _SOURCE is not used.
However, since you don't specify HELLOWORLD_SITE, you trigger a corner case that causes it to behave like the file _SITE_METHOD (which is the one you actually want). Buildroot should give an error for this case. A patch is pending for this.
Unfortunately, that doesn't explain why the patches don't get applied.
I'm on a Linux system with limited resources and BusyBox -- this version of tar does not support --append, -r. Is there a workaround that will allow me to [1] append files from directory B to an existing tar of files from directory A after [2] making the B-files appear to have come from directory A? (Later, when someone extracts the files, they should all end up in the same directory A.)
Situation: I have a list of files that I want to tar, but I must process some of these files first. The files might be used by other processes so I don't want to edit them in-place. I want to be conservative when using disk space so my script only copies those files which it needs to change (vs copying them all and then processing some and finally archiving them all with tar -- if I copied them all I might run into disk space issues).
This means the files I want to archive end up in two separate locations. But I want the resulting tar file to appear as if they were all in the same location. Near the end of my script, I end up with two text files listing the A and B files by name.
I think this is straightforward with a full-blown version of tar, but I have to work with the BusyBox version (usage below). Thanks in advance for any ideas!
Usage: tar -[cxtzjaZmvO] [-X FILE] [-f TARFILE] [-C DIR] [FILE]...
Create, extract, or list files from a tar file
Operation:
c Create
x Extract
t List
Options:
f Name of TARFILE ('-' for stdin/out)
C Change to DIR before operation
v Verbose
z (De)compress using gzip
j (De)compress using bzip2
a (De)compress using lzma
Z (De)compress using compress
O Extract to stdout
h Follow symlinks
m Don't restore mtime
exclude File to exclude
X File with names to exclude
T File with names to include
In principle, you just need to append a tar repository containing the additional files to the end of the tar file. It is only slightly more difficult than that.
A tar file consists of any number of repetitions of header + file. The header is always a single 512-byte block, and the file is padded to a multiple of 512 bytes, so you can think of these units as being a variable number of 512-byte blocks. Each block is independent; it's header starts with the full pathname to the file. So there is no requirement that files in a directory be tarred together.
There is one complication. At the end of the tar file, there are at least two 512-byte blocks completely filled with 0s. When tar is reading a tar file, it will ignore a single zero-filled header, but the second one will cause it to stop reading the file. If it hits EOF, it will complain, so the terminating empty headers are required.
There might be more than two headers, because tar actually writes in blocks which are a multiple of 512 bytes. Gnu tar, for example, by default writes in multiples of 20 512-byte chunks, so the smallest tar file is normally 10240 bytes.
In order to append new data, you need to first truncate the existing file to eliminate the empty blocks.
I believe that if the tar file was produced by busybox, there will only be two empty blocks, but I haven't inspected the code. That would be easy; you only need to truncate the last 1024 bytes of the file before appending the additional files.
For general tar files, it is trickier. If you knew that the files themselves didn't have NUL bytes in them (i.e. they were all simple text files), you could remove empty headers until you found a block with a non-0 byte in it, which wouldn't be too difficult.
What I would do is:
Truncate the last 1024 bytes of the tar file.
Remember the current size of the tar file.
Append a test tar file consisting of the tar of a file with a simple short message
Verify that tar tf correctly shows the test file
Truncate the file back to the remembered length,
If the tar tf found the test file's name, succeed
If the last 512 bytes of the tar file are all 0s, truncate the last 512 bytes of the file, and return to step 2.
Otherwise fail
If the above procedure succeeds, you can proceed to append the tar repository with the new files.
I don't know if you have a trunc command. If not, you can use dd copy a file over top of an old file at a specified offset (see the seek= option). dd will truncate the file automatically at the end of the copy. You can also use dd to read a 512 byte block (see the skip and count options).
The best solution is to cut the last 1024 bytes and concatenate a new tar after it. In order to append a tar to an existing tar file, they must be uncompressed.
For files like:
$ find a b
a
a/file1
b
b/file2
You can:
$ tar -C a -czvf a.tar.gz .
$ gunzip -c a.tar.gz | { head -c -$((512*2)); tar -C b -c .; } | gzip > a+b.tar.gz
With the result:
$ tar -tzvf a+b.tar.gz
drwxr-xr-x 0/0 0 2018-04-20 16:11:00 ./
-rw-r--r-- 0/0 0 2018-04-20 16:11:00 ./file1
drwxr-xr-x 0/0 0 2018-04-20 16:11:07 ./
-rw-r--r-- 0/0 0 2018-04-20 16:11:07 ./file2
Or you can create both tar in the same command:
$ tar -C a -c . | { head -c -$((512*2)); tar -C b -c .; } | gzip > a+b.tar.gz
Although this is for tar generated by busybox tar. As mentioned in previous answer, GNU tar add multiple of 20 blocks. You need to force the number of blocks to be 1 (--blocking-factor=1) in order to know in advance how many blocks to cut:
$ tar --blocking-factor=1 -C a -c . | { head -c -$((512*2)); tar -C b -c .; } | gzip | tar --blocking-factor=1 -tzv
Anyway, GNU tar do have --append. The last --blocking-factor=1 is only needed if you indent do append the resulting tar again.
I have been using ns2 and nam for a long time on my ubuntu. Now I have to apply a patch (OSPF) to it. I have been searching for how to apply a patch in linux (as I'm kind of new in it) and all I got was:
1) $ tar xvf ns-allinone-2.35.tar.gz
2) $ cd ns-allinone-2.35/
3) $ ln -s ns-2.35/ ns-2.34
... Then the patch will work, except for one line. (commom/packet.h).
4) $ patch -p0 < 10-ospf4ns2.34-base.patch
http://sourceforge.net/projects/ospf4ns/
5) Replace commom/packet.h with the attached "packet.h" : Edited by hand.
6) export CC=gcc44 CXX=g++44 && ./install
I untared the tar file, copied the .patch file in the ns-allinone-2.35 folder called 10-ospf4ns2.34-base.patch, gone to the directory in a terminal, used ln -s ns-2.35/ ns-2.34 and then patched using line 4. After that I have these messages:
The next patch would create the file ns-2.34/classifier/classifier-mtopology.cc,
which already exists! Assume -R? [n] y
patching file ns-2.34/classifier/classifier-mtopology.cc
The next patch would create the file ns-2.34/classifier/classifier-mtopology.h,
which already exists! Assume -R? [n] y
patching file ns-2.34/classifier/classifier-mtopology.h
can't find file to patch at input line 92
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- ns-2.34.old/common/agent.cc 2009-06-14 18:35:45.000000000 +0100
|+++ ns-2.34/common/agent.cc 2010-02-26 23:34:33.161063590 +0000
--------------------------
File to patch:
After this I don't know what to do. What file should I patch? There is no classifier-mtopology.h in ns-2.34.
And after that, which packet.h should I replace with the specified new packet.h? Because there are several common folders. Any help would be much appreciated cause I'm a novice in linux.
It looks like you already applied the patch before (at least partially) and that you are not in the correct directory.
Start again, also you can avoid the symlink with the following instructions:
Extract ns-allinone-2.35.tar.gz in a fresh directory.
cd ns-allinone-2.35/ns-2.35
Apply the patch with:
patch -p1 < 10-ospf4ns2.34-base.patch
I want to name a tar file with the date, but what i try isn't working, see the code below for better explanation, and what a i have to do?
date=$( date +%Y-%m-%d_%H:%M:%S )
tar -czf ${imageName-$date}.tar.gz ${imageBasename} 2>>$ERRORLOG
The image name is a variable that stores a name of an image, like: win-ser-rdp
I had tried too:
tar -czf ${imageName}:${date}.tar.gz ${imageBasename} 2>>$ERRORLOG
Didn't work out too :(
Thanks to all the help :D
The problem is the : in the file name, which is treated specially by GNU tar.
On my own system, where I have a directory named foo, I get this:
$ tar zcf foo:42.tar.gz foo
tar (child): Cannot connect to foo: resolve failed
tar: Child returned status 128
tar: Error is not recoverable: exiting now
$
There doesn't seem to be any way to escape the : character -- but you can inhibit the special interpretation with the --force-local option (thanks to Daniel Serodio for pointing this out).
Colon characters in file names, though they're legal, are typically a bad idea anyway. For example, you probably wouldn't be able to access the file via scp.
Here's the relevant section from the GNU tar manual (info tar and go to section 6.1):
To specify an archive file on a device attached to a remote machine,
use the following:
--file=hostname:/dev/file-name
tar will set up the remote connection, if possible, and prompt you for
a username and password. If you use --file=#hostname:/dev/file-name,
tar will attempt to set up the remote connection using your username
as the username on the remote machine.
When I make file names that include a timestamp, I usually use YYYY-MM-DD-HHMMSS
$(date +%F-%H%M%S)
or you can use YYYY-MM-DD_HH_MM_SS if you prefer
$(date +%F-%H_%M_%S)
Your filenames and variables are incorrect:
${imageName-$date}
Should be
${imageName}${date}
and
${imageName}:${date}.tar.gz
is going to be interpreted as an NFS location because of the : in the name, e.g. server:filename. Replace the : with a - or other non-special character.
try avoiding all colons in the tarfilename, as this will make tar attempt to output to a remote file:
date=$( date +%Y%m%d_%H%M%S )
tar -czf ${imageName}-${date}.tar.gz ${imageBasename} 2>>${ERRORLOG}
tl;dr version:
Is it possible with CMake (>= 2.8) to generate zip files from some files and put the packed zip file in a specific location?
longer version:
I have a CMakeLists.txt that builds my project into a .exe file, and this exe file will read data from a zip file. The content to be packed in the zip file is in my git repository so that it can be edited, too. But, the program needs this data in a zip file. So it would be good if the CMake script could take the data, put it in a zip file, and place it next to the exe. I already heard of CPack, but I did not find any easy examples and am not sure if this is even the right tool for my task.
Is this possible? If yes, how?
Since version 3.2 CMake has the functionality to generate a zip file built-in. The CMake command-line mode sub-command tar supports both the creation of zip and 7zip archives.
For example, if the current CMake source directory contains the file testfile.txt and the directory testdir, you can use the following CMake commands to create a zip file containing both items:
add_custom_target(create_zip COMMAND
${CMAKE_COMMAND} -E tar "cfv" "archive.zip" --format=zip
"${CMAKE_CURRENT_SOURCE_DIR}/testfile.txt"
"${CMAKE_CURRENT_SOURCE_DIR}/testdir")
As a work-around for earlier CMake versions, you can use the jar command that is part of a standard Java JRE installation.
find_package(Java)
execute_process(
COMMAND
"${Java_JAR_EXECUTABLE}" "cfM" "archive.zip"
"-C" "${CMAKE_CURRENT_SOURCE_DIR}" "testfile.txt"
"-C" "${CMAKE_CURRENT_SOURCE_DIR}" "testdir"
RESULT_VARIABLE _result
)
The zip file will be generated in the current CMake binary dir (CMAKE_CURRENT_BINARY_DIR).
It's never late to show real answer:
function(create_zip output_file input_files working_dir)
add_custom_command(
COMMAND ${CMAKE_COMMAND} -E tar "cf" "${output_file}" --format=zip -- ${input_files}
WORKING_DIRECTORY "${working_dir}"
OUTPUT "${output_file}"
DEPENDS ${input_files}
COMMENT "Zipping to ${output_file}."
)
endfunction()
Use like
file(GLOB ZIP_FILES "${CMAKE_CURRENT_SOURCE_DIR}/zip/*")
create_zip("${CMAKE_CURRENT_BINARY_DIR}/native_data.zip" "${ZIP_FILES}" "${CMAKE_CURRENT_SOURCE_DIR}/zip")
This will pack all files from zip/ subdirectory into native_data.zip (in build directory). Then either include your archive (path will differ in different CMakeLists.txt!) as source file or add it as target:
add_custom_target("project-data" ALL DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/native_data.zip")
Install will not differ a lot from usual:
install(FILES "${CMAKE_CURRENT_BINARY_DIR}/native_data.zip" DESTINATION ${DATADIR} RENAME "data000.zip") # Install our zip (as data000.zip)
I assume you already have a zip-tool installed (WinZip or 7z, etc.). You could write a find_zip-tool script which will search for WinZip, or 7Z, etc...
Snippet for WinZip:
FIND_PROGRAM(ZIP_EXECUTABLE wzzip PATHS "$ENV{ProgramFiles}/WinZip")
IF(ZIP_EXECUTABLE)
SET(ZIP_COMMAND "\"${ZIP_EXECUTABLE}\" -P \"<ARCHIVE>\" #<FILELIST>")
ENDIF(ZIP_EXECUTABLE)
Snippet for 7-zip:
FIND_PROGRAM(ZIP_EXECUTABLE 7z PATHS "$ENV{ProgramFiles}/7-Zip")
IF(ZIP_EXECUTABLE)
SET(ZIP_COMMAND "\"${ZIP_EXECUTABLE}\" a -tzip \"<ARCHIVE>\" #<FILELIST>")
ENDIF(ZIP_EXECUTABLE)
Take a look at the file
<cmake-install-dir>\share\cmake-2.8\Modules\CPackZIP.cmake
it shows how CPack searches for a Zip_Executable and prepares some "useful" default flags.
After that, I would suggest to execute_process, similar to sakra's answer
As of version 3.18, CMake now directly supports creating zip or archive files using the file() command with ARCHIVE_CREATE:
file(ARCHIVE_CREATE OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/MyData.zip
PATHS ${CMAKE_CURRENT_SOURCE_DIR}/data
FORMAT zip
)
Be sure to specify a full path for the OUTPUT zipped filename, or the file may not be generated. Also, the PATHS option accepts files or directories to be placed in the zip file, but it does not accept wildcards at the time of writing.
This command supports several archive formats and compression flavors. So, you can use the same command to create tarballs as well:
file(ARCHIVE_CREATE OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/MyData.tar.gz
PATHS ${CMAKE_CURRENT_SOURCE_DIR}/data
FORMAT gnutar
COMPRESSION GZip
)
Since this is the top search result for creating zip files with CMake, here is a CPack solution for completeness. The basic idea is that you make calls to install() and then tell it what to name the resulting zip file. It will be placed in the build directory, though there may be a way to change that. Then you can create the zip file with make package or cpack.
# Version 1: Subtractive
# Include everything in the project source directory.
# Put it at the top level of the zip via `DESTINATION .`
# Subtract things we don't want.
# The trailing slash after "${PROJECT_SOURCE_DIR}/" prevents
# an extra layer of directories.
install(DIRECTORY "${PROJECT_SOURCE_DIR}/"
DESTINATION .
PATTERN ".git*" EXCLUDE
PATTERN ".DS_Store" EXCLUDE
PATTERN "examples" EXCLUDE
PATTERN "docs" EXCLUDE
PATTERN "README.md" EXCLUDE
)
# Version 2: Additive
# Include only the list of things we specify.
# Put it at the top level of the zip via `DESTINATION .`
# install(FILES
# ${SRCS}
# "Notes.txt"
# DESTINATION .
# )
# Tell CPack to create a zip file.
set(CPACK_GENERATOR "ZIP")
# Tell CPack what to name the zip file. It will append `.zip`.
set(CPACK_PACKAGE_FILE_NAME "${CMAKE_PROJECT_NAME}")
# Tell CPack not to put everything inside an enclosing directory.
set(CPACK_INCLUDE_TOPLEVEL_DIRECTORY OFF)
# Apparently this should be always on but isn't for backwards compatibility.
set(CPACK_VERBATIM_VARIABLES YES)
include(CPack)
Essentially what I did was create custom target
add_custom_target(STAGE_FILES)
With this target I copy the files and directories to the CMAKE_CURRENT_BINARY_DIR
add_custom_command(
TARGET STAGE_FILES
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_SOURCE_DIR}/assets/video ${CMAKE_CURRENT_BINARY_DIR}/video
COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_SOURCE_DIR}/assets/data ${CMAKE_CURRENT_BINARY_DIR}/data
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_SOURCE_DIR}/assets/strings_en.csv ${CMAKE_CURRENT_BINARY_DIR}/
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_SOURCE_DIR}/assets/strings_rules_en.csv ${CMAKE_CURRENT_BINARY_DIR}/
COMMAND ${CMAKE_COMMAND} -E tar "cfv" "data.zip" --format=zip --files-from=${CMAKE_SOURCE_DIR}/assets/to_zip.txt
COMMAND ${CMAKE_COMMAND} -E remove_directory ${CMAKE_CURRENT_BINARY_DIR}/data
COMMAND ${CMAKE_COMMAND} -E rename ${CMAKE_CURRENT_BINARY_DIR}/data.zip ${CMAKE_CURRENT_BINARY_DIR}/data
)
The important line
COMMAND ${CMAKE_COMMAND} -E tar "cfv" "data.zip" --format=zip --files-from=${CMAKE_SOURCE_DIR}/assets/to_zip.txt
inside my
to_zip.txt
I specify all the files I want to include in my zip
data/
video/
...
I can now execute the command
make STAGE_FILES
which will copy and zip everything i need