I have a compiled re.Regex,
which I use as a regex,
but I also want to show it to the user,
for which I need it as a string.
strformat seems not to know how to do that.
re2str.nim:
import re
import strformat
let R_LICENSE = re"(?i)^.*(LICENSE|COPYING).*$"
echo fmt"We are using the regex '{R_LICENSE}' to look for a license file."
compile and run with:
nim compile --run re2str.nim
output:
re2str.nim(6, 9) template/generic instantiation of `fmt` from here
/home/user/.choosenim/toolchains/nim-1.6.4/lib/pure/strformat.nim(568, 23) Error: type mismatch: got <Regex>
but expected one of:
func `$`(x: float | float32): string
...
One solution is, to circumvent the issue
by keeping the string the regex was compiled from
available separately.
str2re.nim:
import re
import strformat
let RS_LICENSE = "(?i)^.*(LICENSE|COPYING).*$"
let R_LICENSE = re(RS_LICENSE)
echo fmt"We are using the regex '{RS_LICENSE}' to look for a license file."
compile and run with:
nim compile --run str2re.nim
output:
...
We are using the regex '(?i)^.*(LICENSE|COPYING).*$' to look for a license file.
Related
I am a student studying angr, first time.
I'm watching the code in this url.
https://github.com/Dvd848/CTFs/blob/master/2020_GoogleCTF/Beginner.md
import angr
import claripy
FLAG_LEN = 15
STDIN_FD = 0
base_addr = 0x100000 # To match addresses to Ghidra
proj = angr.Project("./a.out", main_opts={'base_addr': base_addr})
flag_chars = [claripy.BVS('flag_%d' % i, 8) for i in range(FLAG_LEN)]
flag = claripy.Concat( *flag_chars + [claripy.BVV(b'\n')]) # Add \n for scanf() to accept the input
state = proj.factory.full_init_state(
args=['./a.out'],
add_options=angr.options.unicorn,
stdin=flag,
)
# Add constraints that all characters are printable
for k in flag_chars:
state.solver.add(k >= ord('!'))
state.solver.add(k <= ord('~'))
simgr = proj.factory.simulation_manager(state)
find_addr = 0x101124 # SUCCESS
avoid_addr = 0x10110d # FAILURE
simgr.explore(find=find_addr, avoid=avoid_addr)
if (len(simgr.found) > 0):
for found in simgr.found:
print(found.posix.dumps(STDIN_FD))
https://github.com/google/google-ctf/tree/master/2020/quals/reversing-beginner/attachments
Which is the answer of googlectf beginner.
But, the above code does not work. It doesn't give me the answer.
I want to know why the code is not working.
When I execute this code, the output was empty.
I run the code with python3 in Ubuntu 20.04 in wsl2
Thank you.
I believe this script isn't printing anything because angr fails to find a solution and then exits. You can prove this by appending the following to your script:
else:
raise Exception('Could not find the solution')
If the exception raises, a valid solution was not found.
In terms of why it doesn't work, this code looks like copy & paste from a few different sources, and so it's fairly convoluted.
For example, the way the flag symbol is passed to stdin is not ideal. By default, stdin is a SimPackets, so it's best to keep it that way.
The following script solves the challenge, I have commented it to help you understand. You will notice that changing stdin=angr.SimPackets(name='stdin', content=[(flag, 15)]) to stdin=flag will cause the script to fail, due to the reason mentioned above.
import angr
import claripy
base = 0x400000 # Default angr base
project = angr.Project("./a.out")
flag = claripy.BVS("flag", 15 * 8) # length is expected in bits here
initial_state = project.factory.full_init_state(
stdin=angr.SimPackets(name='stdin', content=[(flag, 15)]), # provide symbol and length (in bytes)
add_options ={
angr.options.SYMBOL_FILL_UNCONSTRAINED_MEMORY,
angr.options.SYMBOL_FILL_UNCONSTRAINED_REGISTERS
}
)
# constrain flag to common alphanumeric / punctuation characters
[initial_state.solver.add(byte >= 0x20, byte <= 0x7f) for byte in flag.chop(8)]
sim = project.factory.simgr(initial_state)
sim.explore(
find=lambda s: b"SUCCESS" in s.posix.dumps(1), # search for a state with this result
avoid=lambda s: b"FAILURE" in s.posix.dumps(1) # states that meet this constraint will be added to the avoid stash
)
if sim.found:
solution_state = sim.found[0]
print(f"[+] Success! Solution is: {solution_state.posix.dumps(0)}") # dump whatever was sent to stdin to reach this state
else:
raise Exception('Could not find the solution') # Tell us if angr failed to find a solution state
A bit of Trivia - there are actually multiple 'solutions' that the program would accept, I guess the CTF flag server only accepts one though.
❯ echo -ne 'CTF{\x00\xe0MD\x17\xd1\x93\x1b\x00n)' | ./a.out
Flag: SUCCESS
The remark site has a link to an AST explorer for the output of remark - https://astexplorer.net/#/gist/0a92bbf654aca4fdfb3f139254cf0bad/ffe102014c188434c027e43661dbe6ec30042ee2
What I cannot find is how to do the parsing to AST - all the examples convert to html.
I have this code
import {unified} from 'unified'
import remarkParse from 'remark-parse'
import remarkGfm from 'remark-gfm' // git flavoured markdown
const content = `
# My header
This is my content
- abc
- def
`;
unified()
.use(remarkParse)
.use(remarkGfm)
.process('# Hi\n\n*Hello*, world!')
.then((file) => {
console.log(String(file))
})
but am getting a couple of errors here that I do not know how to get around
[remark-gfm] Warning: please upgrade to remark 13 to use this plugin
file:///markdown/node_modules/unified/lib/index.js:520
throw new TypeError('Cannot `' + name + '` without `Compiler`')
^
TypeError: Cannot `process` without `Compiler`
You almost have it. Below I've simplified your code, removing unused and unnecessary parts.
import {unified} from 'unified'
import remarkParse from 'remark-parse'
let myResult = unified()
.use(remarkParse)
.parse('# Hi\n\n*Hello*, world!');
console.log(JSON.stringify(myResult, null, " "));
According to ChristianMurphy's GitHub Q/A:
unified.process() will try to take text, turn it into an AST, and back into text.
For your stated purpose, "...parsing to AST [JSON]", you don't need or want the "full-cycle process" that unified.process() is failing to complete, TypeError: Cannot 'process' without 'Compiler'. You merely want to parse the input and emit the syntax tree (AST) as JSON. The error here is because process(), after parsing your input (markdown string) and turning it into a syntax tree (AST), is then trying to "compile" it to some output format. However, you haven't supplied a compiler. But, as I understand your post, you don't need or want to compile the syntax tree into another output (language). So, change from process() to parse() and emit the resulting syntax tree as JSON.
I am trying to print a custom help message for a bash script. Inside the script, I call a function that then uses python to parse the script, find the functions and the function help string, and then print them. This all works. But, when I try to sort the list of tuples that contains the function name and help string, the sort seems to be ignored. Similar code works as expected in a pure python environment.
Edit: Just noticed I tried the sort two different ways. AFAIK either should have sorted the list, but neither worked.
Edit again: see the accepted answer below for code that actually works. I need to remember to reread my problem code in the morning ;)
SCRIPT_PATH=$(realpath $0)
function build() { ## builds source and wheel package
echo "foo"
}
function aa() { ## foo
echo "foo"
}
function release() { ## package and upload a release
echo "foo"
}
function project:init:all() { ## Initialize a venv, update pip, wheels, and setuptools, and install both requirements files.
echo "foo"
}
function venv:init() { ## makes a venv in the project directory
echo "foo"
}
function print:help() {
echo $SCRIPT_PATH
python3 - << EOF
from pathlib import Path
from operator import itemgetter
import re
script_path = Path("$SCRIPT_PATH")
with open(script_path) as file:
for line in file:
match = re.match(r'^function\s*([a-zA-Z0-9\:-]*)\(\)\s*{\s*##\s*(.*)', line)
matches = []
if match is not None:
target, help = match.groups()
matches.append((target,help))
#for help_line in sorted(matches,key=lambda func: func[1]):
matches.sort(key=itemgetter(1))
for help_line in matches:
print(" {0:20} {1}".format(target,help))
EOF
}
results in:
build builds source and wheel package
aa foo
release package and upload a release
project:init:all Initialize a venv, update pip, wheels, and setuptools, and install both requirements files.
venv:init makes a venv in the project directory
but I expected:
aa foo
build builds source and wheel package
project:init:all Initialize a venv, update pip, wheels, and setuptools, and install both requirements files.
release package and upload a release
venv:init makes a venv in the project directory
You need to get all the functions, then sort and print :
function print:help() {
echo $SCRIPT_PATH
python3 - << EOF
from pathlib import Path
from operator import itemgetter
import re
script_path = Path("$SCRIPT_PATH")
with open(script_path) as file:
functions = []
for line in file:
match = re.match(r'^function\s*([a-zA-Z0-9\:-]*)\(\)\s*{\s*##\s*(.*)', line)
if match is not None:
functions.append(match.groups())
for target, help in sorted(functions):
print(" {0:20} {1}".format(target,help))
EOF
}
I want a Scala string to be ${foo.bar} (literally, for testing some variable substitution later).
I tried:
val str = "${foo.bar}"
val str = """${foo.bar}"""
val str = "\${foo.bar}"
val str = "$${foo.bar}"
val str = "$\{foo.bar}"
All giving compile errors like Error:(19, 15) possible missing interpolator: detected an interpolated expression or invalid escape character.
This is not a question about String interpolation (or variable substitution), This normally works without problems. Starting the Scala REPL (Scala 2.11.3, Java 1.8) works as expected. Somewhere there must be an SBT a setting (other than -Xlint or a hidden Xlint) which apparently is causing this behavior (from commandline and IntelliJ).
The s or f interpolator will emit a constant:
$ scala -Xlint
Welcome to Scala 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144).
Type in expressions for evaluation. Or try :help.
scala> "${foo.bar}"
<console>:12: warning: possible missing interpolator: detected an interpolated expression
"${foo.bar}"
^
res0: String = ${foo.bar}
scala> f"$${foo.bar}"
res1: String = ${foo.bar}
It's usual to use -Xfatal-warnings to turn the warning into an error. IntelliJ reports it as an error at the source position, whereas scalac reports it as a warning, but with a summary error message that will fail a build.
\$ and \{ are invalid escape characters and will not compile. The other versions compile just fine on 2.12.6 though perhaps there are problems in earlier versions.
I've been trying to use the Mafft alignment tool from Bio.Align.Applications. Currently, I've had success writing my sequence information out to temporary text files that are then read by MafftCommandline(). However, I'd like to avoid redundant steps as much as possible, so I've been trying to write to a memory file instead using io.StringIO(). This is where I've been having problems. I can't get MafftCommandline() to read internal files made by io.StringIO(). I've confirmed that the internal files are compatible with functions such as AlignIO.read(). The following is my test code:
from Bio.Align.Applications import MafftCommandline
from Bio import SeqIO
from Bio.Seq import Seq
from Bio.SeqRecord import SeqRecord
import io
from Bio import AlignIO
sequences1 = ["AGGGGC",
"AGGGC",
"AGGGGGC",
"AGGAGC",
"AGGGGG"]
longest_length = max(len(s) for s in sequences1)
padded_sequences = [s.ljust(longest_length, '-') for s in sequences1] #padded sequences used to test compatibilty with AlignIO
ioSeq = ''
for items in padded_sequences:
ioSeq += '>unknown\n'
ioSeq += items + '\n'
newC = io.StringIO(ioSeq)
cLoc = str(newC).strip()
cLocEdit = cLoc[:len(cLoc)] #create string to remove < and >
test1Handle = AlignIO.read(newC, "fasta")
#test1HandleString = AlignIO.read(cLocEdit, "fasta") #fails to interpret cLocEdit string
records = (SeqRecord(Seq(s)) for s in padded_sequences)
SeqIO.write(records, "msa_example.fasta", "fasta")
test1Handle1 = AlignIO.read("msa_example.fasta", "fasta") #alignIO same for both #demonstrates working AlignIO
in_file = '.../msa_example.fasta'
mafft_exe = '/usr/local/bin/mafft'
mafft_cline = MafftCommandline(mafft_exe, input=in_file) #have to change file path
mafft_cline1 = MafftCommandline(mafft_exe, input=cLocEdit) #fails to read string (same as AlignIO)
mafft_cline2 = MafftCommandline(mafft_exe, input=newC)
stdout, stderr = mafft_cline()
print(stdout) #corresponds to MafftCommandline with input file
stdout1, stderr1 = mafft_cline1()
print(stdout1) #corresponds to MafftCommandline with internal file
I get the following error messages:
ApplicationError: Non-zero return code 2 from '/usr/local/bin/mafft <_io.StringIO object at 0x10f439798>', message "/bin/sh: -c: line 0: syntax error near unexpected token `newline'"
I believe this results due to the arrows ('<' and '>') present in the file path.
ApplicationError: Non-zero return code 1 from '/usr/local/bin/mafft "_io.StringIO object at 0x10f439af8"', message '/usr/local/bin/mafft: Cannot open _io.StringIO object at 0x10f439af8.'
Attempting to remove the arrows by converting the file path to a string and indexing resulted in the above error.
Ultimately my goal is to reduce computation time. I hope to accomplish this by calling internal memory instead of writing out to a separate text file. Any advice or feedback regarding my goal is much appreciated. Thanks in advance.
I can't get MafftCommandline() to read internal files made by
io.StringIO().
This is not surprising for a couple of reasons:
As you're aware, Biopython doesn't implement Mafft, it simply
provides a convenient interface to setup a call to mafft in
/usr/local/bin. The mafft executable runs as a separate process
that does not have access to your Python program's internal memory,
including your StringIO file.
The mafft program only works with an input file, it doesn't even
allow stdin as a data source. (Though it does allow stdout as a
data sink.) So ultimately, there must be a file in the file system
for mafft to open. Thus the need for your temporary file.
Perhaps tempfile.NamedTemporaryFile() or tempfile.mkstemp() might be a reasonable compromise.