Golang tarring directory - linux

I am having trouble figuring out how to write a directory into tarball's headerinfo. For example, I have this directory structure:
test [dir]
-- test1.txt
-- subDirA [sub dir]
-- test2.txt
If I tar up test using my Go tar executable, it will untar and have the original structure. If I do a tar -tvf on the tarball, the listing will be:
test/test1.txt
test/subDirA/test2.txt
However if I manually do a tar -cvf of test and then do a tar -tvf, the listing will be:
test/
test/test1.txt
test/subDirA/
test/subDirA/test2.txt
This is what I want my Go tarring program to do as well. In my Go program, I tried to add a new method writeDirToTar to deal with adding directories into the tarWriter. I call either writeFileToTar or writeDirToTar from a fileWalk function. I am getting a "test/subDirA is a directory" error when I call io.Copy(tarWriter, file) in writeDirToTar, which kind of makes sense. Is there a workaround for this?
func writeToTar(fileDir string,
sourceBase string,
tarWriter *tar.Writer,
f os.FileInfo) error {
file, err := os.Open(fileDir)
if err != nil {
log.Print("error opening file")
return err
}
defer file.Close()
// relative paths are used to preserve the directory paths in each file path
relativePath, err := filepath.Rel(sourceBase, fileDir)
tarheader := new(tar.Header)
tarheader.Name = relativePath
tarheader.Size = f.Size()
tarheader.Mode = int64(f.Mode())
tarheader.ModTime = f.ModTime()
err = tarWriter.WriteHeader(tarheader)
if err != nil {
log.Print("error writing tarHeader")
return err
}
_, err = io.Copy(tarWriter, file)
if err != nil {
log.Print("error writing to tarWriter")
return err
}
return nil
}
func writeDirToTar(fileDir string,
sourceBase string,
tarWriter *tar.Writer,
f os.FileInfo) error {
file, err := os.Open(fileDir)
if err != nil {
log.Print("error opening file")
return err
}
defer file.Close()
// relative paths are used to preserve the directory paths in each file path
relativePath, err := filepath.Rel(sourceBase, fileDir)
if tarHeader, err := tar.FileInfoHeader(f, relativePath); err != nil {
log.Print("error writing tarHeader")
return err
}
tarHeader.Name = relativePath
tarheader.Mode = int64(f.Mode())
tarheader.ModTime = f.ModTime()
_, err = io.Copy(tarWriter, file)
if err != nil {
log.Print("error writing to tarWriter")
return err
}
return nil
}

You're missing the Typeflag field in the tar.Header.
For the directory you'll want at least:
tarheader := &tar.Header{
Name: relativePath,
Mode: int64(f.Mode()),
ModTime: f.ModTime(),
Typeflag: tar.TypeDir,
}
You don't want a size here, nor do you want to copy the directory data, since the internal directory data structures mean nothing to tar.
You may also want to add a / to the filename to visually indicate it's a directory and match the POSIX tar behavior.
Alternatively you could just use the tar.FileInfoHeader function just like you did with the files to create the proper header structure.

Related

Golang chmod files and directories recursively [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 months ago.
Improve this question
I'm trying to use WalkDir with Chmod to recursively change the permission of a directory and its files and subdirectories. Somehow it does just change the first file or directory of the given path and stops afterwards. Can someone spot the mistake? getFileMode just converts the string "755" in os.FileMode(0755) and returns it.
func ChmodRec(path string, di fs.DirEntry, err error) error {
fileMode, err2 := getFileMode(os.Getenv("CHMOD_MODE"))
if err2 != nil {
log.Fatal("Could not set file mode for chmodding", path)
panic(err)
}
err2 = os.Chmod(path, fileMode)
if err2 != nil {
fmt.Println("Could not chmod", path)
panic(err)
}
fmt.Println("Changing mode of", path)
return nil
}
func ChmodRecursive(path string, mode string) {
os.Setenv("CHMOD_MODE", mode)
err := filepath.WalkDir(path, ChmodRec)
if err != nil {
log.Fatal("Could not chmod recursively ", path)
panic(err)
}
}
func main() {
path := "bla/test/"
mode := "755"
ChmodRecursive(path, mode)
}
Your code does not check err argument in ChmodRec. This is an extract from official documentation:
WalkDir calls the function with a non-nil err argument in two cases.
First, if the initial fs.Stat on the root directory fails, WalkDir
calls the function with path set to root, d set to nil, and err set to
the error from fs.Stat.
Second, if a directory's ReadDir method fails, WalkDir calls the
function with path set to the directory's path, d set to an
fs.DirEntry describing the directory, and err set to the error from
ReadDir. In this second case, the function is called twice with the
path of the directory: the first call is before the directory read is
attempted and has err set to nil, giving the function a chance to
return SkipDir and avoid the ReadDir entirely. The second call is
after a failed ReadDir and reports the error from ReadDir. (If ReadDir
succeeds, there is no second call.)
Add this code to the beginning of the function. It can give you a hint:
func ChmodRec(path string, di fs.DirEntry, err error) error {
if err != nil {
log.Fatal(err)
}

Go "Permission Denied" when trying to create a new backup config file inside /etc

I am writing APIs with functionalities to save backups of config files inside /etc.
backupContents, openErr := os.ReadFile(path)
if openErr == nil {
t := time.Now()
backupPath := path + "." + t.Format("2006-01-02") + ".bk"
err := os.WriteFile(backupPath, backupContents, 0777)
if err != nil {
return err
}
}
if openErr == nil || errors.Is(openErr, os.ErrNotExist) {
file, err := os.Create(path)
if err != nil {
return err
}
defer file.Close()
if _, err := file.Write(updated); err != nil { //update file
return err
}
} else if openErr != nil {
return openErr
}
return nil
}
However, I get the error open /etc/dhcpcd.conf.2022-03-24.bk: permission denied"
I can write to /etc/dhcpcd.conf successfully in the same function as my API binaries run with root access, how come creating a new file in /etc has permission errors? I thought of umask though I think it is not the issue, my default umask was 0022 but I set it to 0000 instead to try, but got the same permission errors. I also tried to replace os.WriteFile with os.OpenFile (with os.O_RDWR|os.O_CREATE flags) or os.Create but get the same permission denied errors.
Here is the permission of the /etc folder:
drwxr-xr-x 115 root root 4096 Mar 24 10:56
Please help, thanks a lot~~
you are probably running the script with a user that is not root user.
try running the script with root user or with sudo.
/etc dir requires root permissions for creating/removing/editing files
You need root permission, run with sudo before the command to run the script or sudo su in bash to get root access.

fork/exec ./node_modules/.bin/solcjs: no such file or directory

I have a small issue here when I try to run this code
package main
import (
"fmt"
"os/exec"
)
func main() {
out, err := exec.Command("./node_modules/.bin/solcjs", "--version").Output()
if err != nil {
panic(err)
}
fmt.Println(out)
}
This code will get solcjs version from ./node_modules/.bin/solcjs.
But, the code return an error telling me that the file/folder doesn't exist, and I try the command ./node_modules/.bin/solcjs --version my self and it work perfectly. Why when i use go it show error?
You probably need to mention the full path of the solcjs file.
Use snippet below to take the current working directory and then add this path before /node_modules/.bin/solcjs:
mydir, _ := os.Getwd()
file_full_path := mydir + "/node_modules/.bin/solcjs"
out, err := exec.Command(file_full_path, "--version").Output()

Go - is it possible to encrypt excel workbook

I have checked the excelize package, there's the ProtectSheet function. However this only protects the sheet from changes etc not protecting from the unauthorised access.
I have look for several other options/packages and they don't seem to offer such capabilities.
I know that ultimately I could still zip-password protect the excel file, but being able to protect the workbook itself is much preferable.
After some research it seems there's no Go packages that can achieve this. However I've come across a npm package - secure-spreadsheet that is able to password protect an excel file.
As I needed to program the password protect operation in Go, what i did was to trigger an os command in the Go code to invoke the npm package to password protect the file.
The Go program is deployed & executed in an alpine container, which i can install the node-npm and have access to npx command.
code sample:
func passProtectExcelWorkbook(filename, outFilename string) error {
passwd := "password"
cat := exec.Command("cat", filename)
excel := exec.Command("npx", "secure-spreadsheet", "--password", passwd, "--input-format", excelExt)
file, err := os.Create(outFilename)
if err != nil {
return fmt.Errorf("error when creating excel file: %v err: %v", outFilename, err)
}
defer file.Close()
excel.Stdin, err = cat.StdoutPipe()
if err != nil {
return fmt.Errorf("error when reading from cat command output: %v", err)
}
excel.Stdout = file
if err := excel.Start(); err != nil {
return fmt.Errorf("error when starting npx command: %v", err)
}
if err := cat.Run(); err != nil {
return fmt.Errorf("error when running cat command: %v", err)
}
return excel.Wait()
}
Go language Excelize library support encrypt workbook with password since v2.6.1. Reference the issue, #199, you can specify password like this:
workbook.SaveAs("Encryption.xlsx", excelize.Options{Password: "your_password"})

Go: strange results when using strings with exec.Command

I have a Go function that processes Linux CLI commands and their arguments:
func cmd(cmd string, args ...string) ([]byte, error) {
path, err := exec.Command("/usr/bin/which", cmd).Output()
if err != nil {
return []byte(""), err
}
response, err := exec.Command(string(path), args...).Output()
if err != nil {
response = []byte("Unknown")
}
return response, err
}
Which is called by the following:
func main() {
uname, err := cmd("uname", "-a")
fmt.Println(string(uname))
}
The "which" command returns the correct path to the binary but when it tries to run the second exec command with a dynamic path the return is always:
fork/exec /usr/bin/uname
: no such file or directory
exit status 1
Yet if the second exec command is hardcoded, everything works as expected and prints the uname:
response, err := exec.Command("/usr/bin/uname", args...).Output()
Am I missing something about how exec and strings behave?
Thanks
The which command prints a newline following the name of the executable. The path variable is set to "/usr/bin/uname\n". There is no executable with this path. The extra newline is visible in the error message (the newline just before the ":").
Trim the newline suffix to get the correct name of the executable:
response, err := exec.Command(strings.TrimSuffix(string(path), "\n"), args...).Output()

Resources