Filename manipulation in cygwin - string

I am running cygwin on Windows 7. I am using a signal processing tool and basically performing alignments. I had about 1200 input files. Each file is of the format given below.
input_file_ format = "AC_XXXXXX.abc"
The first step required building some kind of indexes for all the input files, this was done with the tool's build-index command and now each file had 6 indexes associated with it. Therefore now I have about 1200*6 = 7200 index files. The indexes are of the form given below.
indexes_format = "AC_XXXXXX.abc.1",
"AC_XXXXXX.abc.2",
"AC_XXXXXX.abc.3",
"AC_XXXXXX.abc.4",
"AC_XXXXXX.abc.rev.1",
"AC_XXXXXX.abc.rev.1"
Now, I need to use these indexes to perform the alignment. All the 6 indexes of each file are called together and the final operation is done as follows.
signal-processing-tool ..\path-to-indexes\AC_XXXXXX.abc ..\Query file
Where AC_XXXXXX.abc is the index associated with that particular index file. All 6 index files are called with **AC_XXXXXX.abc*.
My problem is that I need to use only the first 14 characters of the index file names for the final operation.
When I use the code below, the alignment is not executed.
for file in indexes/*; do ./tool $file|cut -b1-14 Project/query_file; done
I'd appreciate help with this!

First of all, keep in mind that $file will always start with "indexes/", so trimming first 14 characters would always include that folder name in the beginning.
To use first 14 characters in a variable, use ${file:0:14}, where 0 is the starting string index, and 14 is the length of the desired substring.
Alternatively, if you want to use cut, you need to run it in a subshell: for file in indexes/*; do ./tool $(echo $file|cut -c 1-14) Project/query_file; done I changed the arg for cut to -c for characters instead of bytes

Related

How to concat string in gnu-parallel?

I try to realize a tiny function that is to read users name from **fastalist ** file and to parallely create name.txt. But the spliced filenames confused me.
As shown in the below, the first one name is '1pazA' but the output is '.txtA'. It is likely the first 3 letters are replaced. However, for the second out is right.
# cmd
cat BuildFeatures/example/fastalist | parallel -j 5 echo {}.txt
# out
.txtA
T0968s1.txt
# fastalist file content
1pazA
T0968s1
I expect to get the right spliced string.

how can i make the lines variable in a file?

I am using a unix based program. I want to automate the code so as not to copy and paste data one by one. For this, I need to define line-by-line data in a file as a variable for the code
The program converts xyz coordinates to local coordinates. How can I run the coordinates in the xyz_coordinates file I created, one by one, in the code below? In the program I use, the conversion code works like this:
echo 4208830.039709186 2334850.551667509 4171267.377406844 -6.753E-01 4.493E-01 2.849E-01 | xyz2env.py
and this is the file i am trying to run:
2679689.926729193 -727950.9964290063 5722789.538975053 7.873E-02 3.466E-01 6.410E-01
2679689.927123377 -727950.9971557076 5722789.540522 7.912E-02 3.458E-01 6.425E-01
2679689.930567728 -727950.9979971027 5722789.550832021 8.257E-02 3.450E-01 6.528E-01
2679689.931029495 -727950.9992263148 5722789.549927638 8.303E-02 3.438E-01 6.519E-01
2679689.929031829 -727950.9981009626 5722789.546359798 8.103E-02 3.449E-01 6.484E-01
........
and it goes on like this. Also, there are spaces between the lines. Will this be a problem?
You can use xargs to invoke the command for a specific number of arguments (6 in your case) and have the advantage of skipping empty lines automatically
< file.txt xargs -n 6 xyz2env.py

Multiple condition in Bash for loop to get unique output

hope everyone fine. I am stacked badly so need your help
I am using a for loop to collect all folder names
for tamplate in /root/tool/nuclei-templates/*/
do
echo $tamplate
done
Output
/root/tool/nuclei-templates/brute-force/
/root/tool/nuclei-templates/cves/
/root/tool/nuclei-templates/dns/
/root/tool/nuclei-templates/files/
/root/tool/nuclei-templates/generic-detections/
/root/tool/nuclei-templates/panels/
/root/tool/nuclei-templates/payloads/
/root/tool/nuclei-templates/security-misconfiguration/
/root/tool/nuclei-templates/subdomain-takeover/
/root/tool/nuclei-templates/technologies/
/root/tool/nuclei-templates/tokens/
/root/tool/nuclei-templates/vulnerabilities/
/root/tool/nuclei-templates/workflows/
I am using this output on a tool that need those folder path. That tool also give output like this
nuclei -l url.txt -t /root/tool/nuclei-templates/brute-force/ -o result.brute-force
Output
result.brute-force
But as i am using a for loop to automate this scan part i also need to generate unique output for each result.
I am expecting a output like this
result.brute-force
reesult.cves
result.dns
result.files
Generally for each tamplate load with for loop it should generate a output with the name of that specific tamplate folder.
If everything work well this should give me 13 unique result with that output pattern i mentioned.

Cannot append to a file: Append replaces the content

The following command does not append but replaces the content
echo 0 >> /sys/block/nvme0n1/queue/nomerges
I don't want to replace but append. But I'm curious Is there something special about this file?
It also doesn't allow more than one character as its input.
Look at https://serverfault.com/questions/865787/what-does-the-nomerge-mean-in-linux-system
It might help you in understanding, that there are only 3 options that the file can take.
Also:
nomerges enables the user to disable the lookup logic involved with IO
merging requests in the block layer. By default (0) all merges are
enabled. When set to 1 only simple one-hit merges will be tried. When
set to 2 no merge algorithms will be tried (including one-hit or more
complex tree/hash lookups).

Matching text files from a list of system numbers

I have ~ 60K bibliographic records, which can be identified by system number. These records also hold full-text (individudal text files named by system number).
I have lists of system numbers in bunches of 5K and I need to find a way to copy only the text files from each 5K list.
All text files are stored in a directory (/fulltext) and are named something along these lines:
014776324.txt.
The 5k lists are plain text stored in separated directories (e.g. /5k_list_1, 5k_list_2, ...), where each system number matches to a .txt file.
For example: bibliographic record 014776324 matches to 014776324.txt.
I am struggling to find a way to copy into the 5k_list_* folders only the corresponding text files.
Any idea?
Thanks indeed,
Let's assume we invoke the following script this way:
./the-script.sh fulltext 5k_list_1 5k_list_2 [...]
Or more succinctly:
./the-script.sh fulltext 5k_list_*
Then try using this (totally untested) script:
#!/usr/bin/env bash
set -eu # enable error checking
src_dir=$1 # first argument is where to copy files from
shift 1
for list_dir; do # implicitly consumes remaining args
while read bibliographic record sys_num rest; do
cp "$src_dir/$sys_num.txt" "$list_dir/"
done < "$list_dir/list.txt"
done

Resources