I'm trying to upload certificates(just created) to some storage.
So I can read all certificates in my folder and want to use content of each of this file to a variable in a loop.
#!/bin/bash
dir="${0%/*}"
#for f in $(cat $dir"/"*.crt)
# do
# data='{"certificate_data":'"$f"'}'
#done
url="localhost:50183/api/v0.1/Certificates"
data='{"certificate_data":'$(cat $dir"/"*.crt)'}'
echo "$data"
So I got all certificates in one time but I need to get in $data each of content of files in a loop with correct form something like:
{"certificate_data":"<certificate_data_from_file>"}
{"certificate_data":"<certificate_data_from_file>"}
......
and so on
I know that I should use another one loop but don't know how.
Be grateful for any tips!
This should do the work:
#!/bin/bash
for f in ./dir/*.crt
do
data='{"certificate_data":"'"$(< "${f}")"'"}'
echo "${data}"
done
Test:
$ ls ./dir/*
./dir/cert1.crt ./dir/cert2.crt
$ cat ./dir/*
I am certificate1.
I am certificate2.
$ ./cert.sh
{"certificate_data":"I am certificate1."}
{"certificate_data":"I am certificate2."}
Related
I are creating a .sh using bash for validate the api sub folders versions
The objective is validate this strings into APIS_BUILD var and find all .proto files into ./proto folder to compile into protobuffer Go file
# define subfolder apis to build
APIS_BUILD=(
prototests/v1/
prototests2/v2
testfolder
)
# the "testfolder" are a invalid folder
Test cases:
prototestes/v1 # valid
prototestes/v1/cobranca # valid
prototestes/v1/cobrnaca/faturamento # valid
outrapastacomarquivosproto/v1 # valid
prototests # invalid
/prototests # invalid
Then, I created this script for validate the APIS_BUILD string array
#!/usr/bin/env bash
# text color
RED='\033[0;31m' # RED
BLUE='\033[0;34m' # Blue
NC='\033[0m' # No Color
# Underline color
UCyan='\033[4;36m' # Cyan
# define subfolder apis to build
APIS_BUILD=(
prototests/v1
cobrancas/v1
)
DST_DIR="." # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
# Compile proto file
# $1 = Filename to compile
function compile() {
protoc --go_out=$DST_DIR --proto_path=proto --go_opt=M$1=services \
--go_opt=paths=import --go-grpc_out=. \
$1
}
# Validate api_build's
function validateApiBuilds() {
for t in ${APIS_BUILD[#]}; do
IFS="/"
read -a SUBSTR <<<"$t"
if [ ${#SUBSTR[#]} -lt 2 ]; then
printf "${RED}The API_BUILD value ${UCyan}\"${t}\"${RED} are declare wrong, please declare [api_folder]/[version_folder] (example: prototest/v1)${NC}\n" 1>&2
exit 1
fi
done
}
validateApiBuilds
for filename in $(find $SRC_DIR -name '*.proto'); do
[ -e "$filename" ] || continue
echo $filename
done
The subfolder:
But I getting a strange behavior:
If run the .sh file with the validateApiBuilds function the return for $filename is always .
If run the .sh file without the validateApiBuilds function the return for $filename are getting the testservice.proto file
Pictures:
With validateApiBuilds function:
Without validateApiBuilds function:
All the variables:
# define subfolder apis to build
APIS_BUILD=(
prototests/v1
cobrancas/v1
)
DST_DIR="." # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
Bash version:
$ bash --version
$ GNU bash, versão 4.4.19(1)-release (x86_64-pc-linux-gnu)
Obs.: I changed the validateApiBuilds function to use a regex validation for strings into API_BUILDS variable. But I really wanted to know the reason for this behavior.
edit 2: The make-proto.config file
# define subfolder apis to build
APIS_BUILD=(
prototests/v1
cobrancas/v1
)
DST_DIR="." # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
Use find better
for filename in $(anything) is always an antipattern -- it splits values on characters in IFS, and then expands each result as a glob. To make find emit completely unambiguous strings, use -print0:
while IFS= read -r -d '' filename; do
[ -e "$filename" ] || continue
echo "$filename"
done < <(find "$SRC_DIR" -name '*.proto' -print0)
Don't change IFS unnecessarily
Change your code to make the assignment to IFS be on the same line as the read, which will make IFS only be modified for that one command.
That is to say, instead of:
IFS=/
read -a SUBSTR <<<"$t"
...you should write:
IFS=/ read -a SUBSTR <<<"$t"
I've some files in a folder A which are named like that:
001_file.xyz
002_file.xyz
003_file.xyz
in a separate folder B I've files like this:
001_FILE_somerandomtext.zyx
002_FILE_somerandomtext.zyx
003_FILE_somerandomtext.zyx
Now I want to rename, if possible, with just a command line in the bash all the files in folder B with the file names in folder A. The file extension must stay different.
There is exactly the same amount of files in each folder A and B and they both have the same order due to numbering.
I'm a total noob, but I hope some easy answer for the problem will show up.
Thanks in advance!
ZVLKX
*Example edited for clarification
An implementation might look a bit like this:
renameFromDir() {
useNamesFromDir=$1
forFilesFromDir=$2
for f in "$forFilesFromDir"/*; do
# Put original extension in $f_ext
f_ext=${f##*.}
# Put number in $f_num
f_num=${f##*/}; f_num=${f_num%%_*}
# look for a file in directory B with same number
set -- "$useNamesFromDir"/"${f_num}"_*.*
[[ $1 && -e $1 ]] || {
echo "Could not find file number $f_num in $dirB" >&2
continue
}
(( $# > 1 )) && {
# there's more than one file with the same number; write an error
echo "Found more than one file with number $f_num in $dirB" >&2
printf ' - %q\n' "$#" >&2
continue
}
# extract the parts of our destination filename we want to keep
destName=${1##*/} # remove everything up to the last /
destName=${destName%.*} # and past the last .
# write the command we would run to stdout
printf '%q ' mv "$f" "$forFilesFromDir/$destName.$f_ext"; printf '\n'
## or uncomment this to actually run the command
# mv "$f" "$forFilesFromDir/$destName.$f_ext"
done
}
Now, how would we test this?
mkdir -p A B
touch A/00{1,2,3}_file.xyz B/00{1,2,3}_FILE_somerandomtext.zyx
renameFromDir A B
Given that, the output is:
mv B/001_FILE_somerandomtext.zyx B/001_file.zyx
mv B/002_FILE_somerandomtext.zyx B/002_file.zyx
mv B/003_FILE_somerandomtext.zyx B/003_file.zyx
Sorry if this isn't helpful, but I had fun writing it.
This renames items in folder B to the names in folder A, preserving the extension of B.
A_DIR="./A"
A_FILE_EXT=".xyz"
B_DIR="./B"
B_FILE_EXT=".zyx"
FILES_IN_A=`find $A_DIR -type f -name *$A_FILE_EXT`
FILES_IN_B=`find $B_DIR -type f -name *$B_FILE_EXT`
for A_FILE in $FILES_IN_A
do
A_BASE_FILE=`basename $A_FILE`
A_FILE_NUMBER=(${A_BASE_FILE//_/ })
A_FILE_WITHOUT_EXTENSION=(${A_BASE_FILE//./ })
for B_FILE in $FILES_IN_B
do
B_BASE_FILE=`basename $B_FILE`
B_FILE_NUMBER=(${B_BASE_FILE//_/ })
if [ ${A_FILE_NUMBER[0]} == ${B_FILE_NUMBER[0]} ]; then
mv $B_FILE $B_DIR/$A_FILE_WITHOUT_EXTENSION$B_FILE_EXT
break
fi
done
done
I have directories a1..a5, b1..b5 and c1..c5. Inside each directory I have two files a1, b1 and c1.
do mkdir /tmp/{a,b}$d; touch /tmp/{a,b,c}$d/{a,b,c}1; done;
I want to get all the files starting with 'a' or 'b' inside the directories starting with an 'a'. I can do it with:
DIRS=`ls -1 -d /tmp/{a,b}*/a*`
echo ${DIRS}
and obtain:
/tmp/a1/a1 /tmp/a2/a1 /tmp/a3/a1 /tmp/a4/a1 /tmp/a5/a1
/tmp/b1/a1 /tmp/b2/a1 /tmp/b3/a1 /tmp/b4/a1 /tmp/b5/a1
Now, I will use a variable called DATA to store the directories and later get the files:
DATA="/tmp/{a,b}*"
echo ${DATA}
DIRS=`ls -1 -d ${DATA}/a*`
echo ${DIRS}
In the output, the DATA contents is OK (/tmp/{a,b}*), but I receive the following error:
ls: cannot access /tmp/{a,b}*/a*: No such file or directory
Any idea why this happens?
I solved the problem, but I can't find any reference about why my previous attempts failed.
DATA="/tmp/{a,b}*"
echo ${DATA}
DIRS=`eval "ls -1 -d ${DATA}/a*"`
echo ${DIRS}
Output:
/tmp/a1/a1 /tmp/a2/a1 /tmp/a3/a1 /tmp/a4/a1 /tmp/a5/a1 /tmp/b1/a1
/tmp/b2/a1 /tmp/b3/a1 /tmp/b4/a1 /tmp/b5/a1
I am currently writing a script that mounts a samba share, rsyncs the data to a local machine and archives into a directory structure (say /home/archive/). Currently when new pdfs are added, archiving done manually which seems like inefficient use of time
The files have the following structure
ABC140003.pdf
ABC140124.pdf
.
.
ABC144201.pdf
.
ABC146012.pdf
/home/archive/ has several directories 2010/, 2011/, 2012, 2013 etc
Basically, I need to break up the number to find the correct subdirectory to copy the file. First I extract the number
study_number=`echo $file | sed 's/[^0-9]//g'`
Then the year
year=20`echo $study_number | cut -c 1-2`
All the above pdf files belong in the subdirectory of 2014. Within 2014 or any other year directories there are the following subdirectories 2014/Blue/, /2014/Red/and/2014/Green/`. This corresponds to the 3rd integer in the number Blue(0), Red(4) and Green(6).
I use cases here to find what I have called study type
type_int=`echo $study_number | cut -c 3`
case "$type_int" in
0)
type_string="Blue"
;;
4) type_string="Red"
;;
6) type_string="Green"
;;
*) echo "$date: $file has unknown study type. Do not know where to place it" >> $logfile
continue
;;
esac
I now know the following files go in the following directories
ABC140003.pdf -> /home/archive/2014/Blue/
ABC140124.pdf -> /home/archive/2014/Blue/
.
.
ABC144201.pdf -> /home/archive/2014/Red/
.
ABC146012.pdf -> /home/archive/2014/Green/
I'd be happy if this was the end of the directory structure. However, there is another layer of subdirectories have been introduced so that no directory has more than 100 pdf files (Not my call).
For example /home/archive/2014/Blue/ has the following directories:
140001-0100/ 140101-0200/ 140201-0300/ 140301-0400/ 140401-0500/ 140501-0600/
etc
I now need to come up some logic such that the following files go to the following directories
ABC140003.pdf -> /home/archive/2014/Blue/140001-0100
ABC140124.pdf -> /home/archive/2014/Blue/140100-0124
.
.
ABC144201.pdf -> /home/archive/2014/Red/144200-4300
.
ABC146012.pdf -> /home/archive/2014/Green/146000-6100
I am stumped on how to logically determine that study ABC146012 should go in 146000-6100 in an elegant manner without resorting to multiple if statements for each of Red/ Blue/ and Green/
Here is a simplified version that needs some work but you get the idea (for a nice final solution, see #glenn jackman's solution):
Declare an associative array for the colors
$ declare -A colors
$ colors[0]=Blue
$ colors[4]=Red
$ colors[6]=Green
Then extract the needed information
$ study_number=$(sed 's/[^0-9]//g' <<< ABC140124.pdf);
$ year=${study_number:0:2};
$ type=${study_number:2:1};
$ color=${colors[$type]};
$ from="${study_number:0:$((${#study_number}-2))}01"
$ to="$((${study_number:0:$((${#study_number}-2))}+1))00"
and that gives:
$ echo /home/archive/$year/$color/$from-$to
/home/archive/14/Blue/140101-140200
(I assumed you wanted your intervals to be consistently numbered 'x01-(x+1)00')
You can create a function to simplify the process
build_dir() {
study_number=$(sed 's/[^0-9]//g' <<< $1);
year=${study_number:0:2};
type=${study_number:2:1};
color=${colors[$type]};
from="${study_number:0:$((${#study_number}-2))}01"
to="$((${study_number:0:$((${#study_number}-2))}+1))00"
echo "/home/archive/$year/$color/$from-$to"
}
It needs a bit of more defensive programming-related lines of code, but it can be used like this:
$ build_dir ABC146012.pdf
/home/archive/14/Green/146001-146100
colors=([0]=Blue [4]=Red [6]=Green)
get_destination() {
if [[ $1 =~ ([0-9][0-9])([0-9])([0-9]) ]]; then
printf "/home/archive/20%s/%s/%s%s%d01-%s%d00" \
${BASH_REMATCH[1]} \
${colors[${BASH_REMATCH[2]}]} \
${BASH_REMATCH[1]} \
${BASH_REMATCH[2]} \
${BASH_REMATCH[3]} \
${BASH_REMATCH[2]} \
$(( 1 + ${BASH_REMATCH[3]} ))
fi
}
for file in ABC140003.pdf ABC140124.pdf ABC144201.pdf ABC146012.pdf; do
echo "$file -> $(get_destination $file)"
done
ABC140003.pdf -> /home/archive/2014/Blue/140001-0100
ABC140124.pdf -> /home/archive/2014/Blue/140101-0200
ABC144201.pdf -> /home/archive/2014/Red/144201-4300
ABC146012.pdf -> /home/archive/2014/Green/146001-6100
I have many data files in this format:
-1597.5421
-1909.6982
-1991.8743
-2033.5744
But I would like to merge them all into one data file with each original data file taking up one row with spaces in between so I can import it in excel.
-1597.5421 -1909.6982 -1991.8743 -2033.5744
-1789.3324 -1234.5678 -9876.5433 -9999.4321
And so on. Each file is named ALL.ene and every directory in my working directory contains it. Can someone give me a quick fix? Thanks!
:edit. Each file has 11 entries. Those were just examples.
for i in */ALL.ene
do
echo $(<$i)
done > result.txt
Assumptions:
I assume all your data files are of this format:
<something1><newline>
<something2><newline>
<something3><newline>
So for example, if the last newline is missing, the following script will miss the field corresponding to <something3>.
Usage: ./merge.bash -o <output file> <input file list or glob>
The script appends to any existing output files from previous runs. It also does not make any assumptions to how many fields of data every input file has. It blindly puts every line into a line in the output file separated by spaces.
#!/bin/bash
# set -o xtrace # uncomment to debug
declare output
[[ $1 =~ -o$ ]] && output="$2" && shift 2 || { \
echo "The first argument should always be -o <output>";
exit -1; }
declare -a files=("${#}") row
for file in "${files[#]}";
do
while read data; do
row+=("$data")
done < "$file"
echo "${row[#]}" >> "$output"
row=()
done
Example:
$ cat data1
-1597.5421
-1909.6982
-1991.8743
-2033.5744
$ cat data2
-1789.3324
-1234.5678
-9876.5433
-9999.4321
$ ./merge.bash -o test data{1,2}
$ cat test
-1597.5421 -1909.6982 -1991.8743 -2033.5744
-1789.3324 -1234.5678 -9876.5433 -9999.4321
This is what coreutils paste is good at, try:
paste -s data_files*