I'm using the smart_str headers for a short FastCGI script (not related to PHP), but can't use smart_str_alloc correctly. My code:
smart_str json = {0, 0, 0};
smart_str_alloc(&json, 1024 * 20, 0);
This gives error:
php_smart_str.h:72:37: error: ‘newlen’ undeclared (first use in this function)
smart_str_alloc4((d), (n), (what), newlen)
And the macro for smart_str_alloc4 is:
#define smart_str_alloc4(d, n, what, newlen) do { \
if (!(d)->c) { \
(d)->len = 0; \
newlen = (n); \
(d)->a = newlen < SMART_STR_START_SIZE \
? SMART_STR_START_SIZE \
: newlen + SMART_STR_PREALLOC; \
SMART_STR_DO_REALLOC(d, what); \
} else { \
newlen = (d)->len + (n); \
if (newlen >= (d)->a) { \
(d)->a = newlen + SMART_STR_PREALLOC; \
SMART_STR_DO_REALLOC(d, what); \
} \
} \
} while (0)
The newlen argument is passed from the smart_str_alloc definition like so:
#define smart_str_alloc(d, n, what) \
smart_str_alloc4((d), (n), (what), newlen)
Funny thing is, in the PHP source code, smart_str_alloc is used without problem (from json.c):
smart_str_alloc(buf, len+2, 0);
What am I missing?
The macro smart_str_alloc needs a defined variable newlen in it's scope:
size_t newlen;
Related
I try to build a fast sparse solver with Eigen and OpenMP for Python. For the interface between this solver and Python I use the PyBind11 package. Basically, the solver works fine, but unfortunately it only runs on one core and I cannot figure out how to use all cores of my cpu.
Although the OpenMP test functions uses the entire cpu.
Here is the C++ code:
#include <iostream>
#include <cmath>
#include <omp.h>
#include <unistd.h>
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
namespace py = pybind11;
#include <Eigen/Sparse>
#include <Eigen/IterativeLinearSolvers>
void openmp_test()
{
int N;
N = 8;
// omp_set_num_threads(4);
#pragma omp parallel for
for (int i=0; i<N; i=i+1)
{
sleep(10);
}
}
void eigen_test(int N)
{
Eigen::SparseMatrix<double> A(N, N);
Eigen::SparseMatrix<double> b(N, 1);
Eigen::SparseMatrix<double> x(N, 1);
Eigen::BiCGSTAB<Eigen::SparseMatrix<double>> solver;
A.reserve(5*N);
b.reserve(N);
x.reserve(N);
for(int i=0; i<N; i++)
{
b.insert(i, 0) = 20.0;
for(int j=(i-2); j<=(i+2); j++)
{
if(j == i)
{
A.insert(j, i) = 10.0;
}
else if((j >= 0) && (j < N))
{
A.insert(j, i) = 5.0;
}
}
}
solver.compute(A);
x = solver.solve(b);
}
PYBIND11_MODULE(mytest, m)
{
m.def("openmp_test", &openmp_test, py::call_guard<py::gil_scoped_release>());
m.def("eigen_test", &eigen_test, py::call_guard<py::gil_scoped_release>());
}
Here is the Compiler code:
g++ \
-O3 \
-Wall \
-shared \
-std=c++14 \
-fopenmp \
-fPIC \
-I /usr/local/lib/python3.10/dist-packages/pybind11/include \
-I /usr/include/python3.10 \
-I /workspaces/pybind11/external/eigen \
mytest.cpp \
-o mytest.so
And finally here the Python code:
from time import time
import mytest
N = 10 * 10**3
start = time()
mytest.openmp_test()
print("runtime: {:.3f} s".format(time() - start))
start = time()
mytest.eigen_test(N)
print("runtime: {:.3e} s".format(time() - start))
Has anyone an idea how to fix this problem?
Thanks a lot.
See documentation:
Currently, the following algorithms can make use of multi-threading:
general dense matrix - matrix products
PartialPivLU
row-major-sparse * dense vector/matrix products
ConjugateGradient with Lower|Upper as the UpLo template parameter.
BiCGSTAB with a row-major sparse matrix format.
LeastSquaresConjugateGradient
Only the row-major sparse matrix format is supported for multi-threading. Therefore, you will need to have:
Eigen::SparseMatrix<double, Eigen::RowMajor> A(N, N);
Eigen::BiCGSTAB<Eigen::SparseMatrix<double, Eigen::RowMajor>> solver;
I have noticed that inserting elements to the matrix takes longer than calculating the solution. Since the prior is sequential, it may not be feasible to observe the parallelism. You can increase the maximum number of solver iterations (i.e solver.setMaxIterations(1e9)) to force solution to take longer (thus easier to observe CPU occupation). You can also print checkpoints to observe which part of your code is executing at that moment. For example:
void eigen_test(int N)
{
Eigen::SparseMatrix<double, Eigen::RowMajor> A(N, N);
Eigen::SparseMatrix<double> b(N, 1);
Eigen::SparseMatrix<double> x(N, 1);
Eigen::BiCGSTAB<Eigen::SparseMatrix<double, Eigen::RowMajor>> solver;
solver.setMaxIterations(1e9);
A.reserve(5*N);
b.reserve(N);
x.reserve(N);
std::cout << "checkpoint: insert elems..." << std::endl;
for(int i=0; i<N; i++)
{
b.insert(i, 0) = 20.0;
for(int j=(i-2); j<=(i+2); j++)
{
if(j == i)
{
A.insert(j, i) = 10.0;
}
else if((j >= 0) && (j < N))
{
A.insert(j, i) = 5.0;
}
}
}
std::cout << "checkpoint: find solution..." << std::endl;
solver.compute(A);
x = solver.solve(b);
std::cout << "checkpoint: done!" << std::endl;
}
If PHP is running on Windows, escapeshellarg() escapes file names (for example) in a certain way and then adds " (DOUBLE) quotes around it.
If PHP is running on Linux, escapeshellarg() uses Linux-based escaping and then adds ' (SINGLE) quotes around it.
In my situation, I'm generating a SHA256SUMS file on Windows, but aimed for Linux. Since I use escapeshellarg() to escape the file name, I end up with a file like:
cabcdccas12exdqdqadanacvdkjsc123ccfcfq3rdwcndwf2qefcf "cool filename with spaces.zip"
However, Linux tools probably expect:
cabcdccas12exdqdqadanacvdkjsc123ccfcfq3rdwcndwf2qefcf 'cool filename with spaces.zip'
Looking in the manual, there seems to be no way to do something like: escapeshellarg($blabla, TARGET_OS_LINUX); in order for it to use the rules for Linux instead of the OS running the script (Windows).
I can't just str_replace the quotes because it would not take into consideration all the platform-specific rules.
Also, yes, I need spaces in the file name (and any other cross-platform-valid character).
I sadly found no mention whatsoever about the preferred quote style on the only source of information I have for this: https://help.ubuntu.com/community/HowToSHA256SUM
Maybe the SHA256 security verification tools which read that SHA256SUMS file understand and can parse both kinds?
The behavior of escapeshellarg() is hard-coded depending on whether PHP is running on Windows or any other operating system. You should reimplement escapeshellarg() for consistent behavior.
Here is my attempt at reimplementing escapeshellarg() with a Windows/other-OS toggle in PHP:
<?php namespace polyfill;
const TARGET_OS_WINDOWS = 1;
const TARGET_OS_UNIX = 2;
function escapeshellarg(string $input, int $os_mode = 0): string
{
if (false !== strpos($input, "\x00"))
{
throw new \UnexpectedValueException(__FUNCTION__ . '(): Argument #1 ($input) must not contain any null bytes');
}
if ($os_mode == 0)
{
$os_mode = TARGET_OS_UNIX;
if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN')
$os_mode = TARGET_OS_WINDOWS;
}
$maxlen = 4096;
if ($os_mode === TARGET_OS_WINDOWS) $maxlen = 8192;
if (strlen($input) > $maxlen - 2) return "";
if ($os_mode === TARGET_OS_WINDOWS)
{
$output =
str_replace(['"', '%', '!'],
[' ', ' ', ' '],
$input);
# https://bugs.php.net/bug.php?id=69646
if (substr($output, -1) === "\\")
{
$k = 0; $n = strlen($output) - 1;
for (; $n >= 0 && substr($output, $n, 1) === "\\"; $n--, $k++);
if ($k % 2) $output .= "\\";
}
$output = "\"$output\"";
}
else
{
$output = str_replace("'", "'\''", $input);
$output = "'$output'";
}
if (strlen($output) > $maxlen) return "";
return $output;
}
It should be almost functionally equivalent to the native PHP escapeshellarg(), except that:
it takes a second argument that sets whether you want the output in Windows mode or not Windows mode,
it raises an \UnexpectedValueException instead of some kind of PHP error if the input string contains null bytes,
it doesn't emit errors due to the input being too long, and
it has 4096 hard-coded as the maximum argument length on Unix-like platforms.
To use this replacement function:
# In Unix/Linux/macOS mode
\polyfill\escapeshellarg($blabla, \polyfill\TARGET_OS_UNIX);
# In Windows mode
\polyfill\escapeshellarg($blabla, \polyfill\TARGET_OS_WINDOWS);
# In auto-detect (running OS) mode
\polyfill\escapeshellarg($blabla);
Reference
Here is the full C implementation from PHP 7.3.10 (./ext/standard/exec.c):
PHPAPI zend_string *php_escape_shell_arg(char *str)
{
size_t x, y = 0;
size_t l = strlen(str);
zend_string *cmd;
uint64_t estimate = (4 * (uint64_t)l) + 3;
/* max command line length - two single quotes - \0 byte length */
if (l > cmd_max_len - 2 - 1) {
php_error_docref(NULL, E_ERROR, "Argument exceeds the allowed length of %zu bytes", cmd_max_len);
return ZSTR_EMPTY_ALLOC();
}
cmd = zend_string_safe_alloc(4, l, 2, 0); /* worst case */
#ifdef PHP_WIN32
ZSTR_VAL(cmd)[y++] = '"';
#else
ZSTR_VAL(cmd)[y++] = '\'';
#endif
for (x = 0; x < l; x++) {
int mb_len = php_mblen(str + x, (l - x));
/* skip non-valid multibyte characters */
if (mb_len < 0) {
continue;
} else if (mb_len > 1) {
memcpy(ZSTR_VAL(cmd) + y, str + x, mb_len);
y += mb_len;
x += mb_len - 1;
continue;
}
switch (str[x]) {
#ifdef PHP_WIN32
case '"':
case '%':
case '!':
ZSTR_VAL(cmd)[y++] = ' ';
break;
#else
case '\'':
ZSTR_VAL(cmd)[y++] = '\'';
ZSTR_VAL(cmd)[y++] = '\\';
ZSTR_VAL(cmd)[y++] = '\'';
#endif
/* fall-through */
default:
ZSTR_VAL(cmd)[y++] = str[x];
}
}
#ifdef PHP_WIN32
if (y > 0 && '\\' == ZSTR_VAL(cmd)[y - 1]) {
int k = 0, n = y - 1;
for (; n >= 0 && '\\' == ZSTR_VAL(cmd)[n]; n--, k++);
if (k % 2) {
ZSTR_VAL(cmd)[y++] = '\\';
}
}
ZSTR_VAL(cmd)[y++] = '"';
#else
ZSTR_VAL(cmd)[y++] = '\'';
#endif
ZSTR_VAL(cmd)[y] = '\0';
if (y > cmd_max_len + 1) {
php_error_docref(NULL, E_ERROR, "Escaped argument exceeds the allowed length of %zu bytes", cmd_max_len);
zend_string_release_ex(cmd, 0);
return ZSTR_EMPTY_ALLOC();
}
if ((estimate - y) > 4096) {
/* realloc if the estimate was way overill
* Arbitrary cutoff point of 4096 */
cmd = zend_string_truncate(cmd, y, 0);
}
ZSTR_LEN(cmd) = y;
return cmd;
}
// … [truncated] …
/* {{{ proto string escapeshellarg(string arg)
Quote and escape an argument for use in a shell command */
PHP_FUNCTION(escapeshellarg)
{
char *argument;
size_t argument_len;
ZEND_PARSE_PARAMETERS_START(1, 1)
Z_PARAM_STRING(argument, argument_len)
ZEND_PARSE_PARAMETERS_END();
if (argument) {
if (argument_len != strlen(argument)) {
php_error_docref(NULL, E_ERROR, "Input string contains NULL bytes");
return;
}
RETVAL_STR(php_escape_shell_arg(argument));
}
}
/* }}} */
The logic is fairly simple. Here are some equivalent functional test cases in prose:
The input string cannot contain NUL characters.
Applied to the input string,
in Windows mode,
Prepend a " character.
Replace all ", %, and ! characters with .
If the end consists of an odd number of \ characters, add one \ character to the end. (Bug #69646)
Append a " character.
in other platforms mode,
Prepend a ' character.
Replace all ' characters with '\''
Append a ' character.
On Windows, if the output is longer than 8192 characters, emit an E_ERROR and return an empty string.
On other platforms, if the output is longer than 4096 characters (or whatever the overridden maximum is at compile time), emit an E_ERROR and return an empty string.
I use spike to run the test program in "riscv-tools/riscv-tests/build/benchmarks":
$ spike multiply.riscv
And the output shows:
mcycle = 24096
minstret = 24103
Why mcycle is less than minstret?
Dose it means that spike can run more than one instructions in one cycle?
(I tried to trace spike code but cannot find how mcycle is counted.)
The printing of mcycle and minstret values are not from Spike in this case, it is from the test (benchmark). There is the code:
https://github.com/ucb-bar/riscv-benchmarks/blob/master/common/syscalls.c
#define NUM_COUNTERS 2
static uintptr_t counters[NUM_COUNTERS];
static char* counter_names[NUM_COUNTERS];
static int handle_stats(int enable)
{
int i = 0;
#define READ_CTR(name) do { \
while (i >= NUM_COUNTERS) ; \
uintptr_t csr = read_csr(name); \
if (!enable) { csr -= counters[i]; counter_names[i] = #name; } \
counters[i++] = csr; \
} while (0)
READ_CTR(mcycle);
READ_CTR(minstret);
#undef READ_CTR
return 0;
}
There is some code between reading mcycle & minstret. And now you know how much code is there between readings.
In Spike mcycle & minstret are always equal by definition (they are handled by the same code): https://github.com/riscv/riscv-isa-sim/blob/9e012462f53113dc9ed00d7fbb89aeafeb9b89e9/riscv/processor.cc#L347
case CSR_MINSTRET:
case CSR_MCYCLE:
if (xlen == 32)
state.minstret = (state.minstret >> 32 << 32) | (val & 0xffffffffU);
else
state.minstret = val;
break;
The syscalls.c was linked into multiply.riscv binary from https://github.com/ucb-bar/riscv-benchmarks/blob/master/multiply/bmark.mk - multiply_riscv_bin = multiply.riscv
$(multiply_riscv_bin): ... $(patsubst %.c, %.o, ... syscalls.c ... )
There is _init in syscalls.c function which calls main of the test and prints values, recorded on SYS_stats "syscall" with handle_stats.
void _init(int cid, int nc)
{
init_tls();
thread_entry(cid, nc);
// only single-threaded programs should ever get here.
int ret = main(0, 0);
char buf[NUM_COUNTERS * 32] __attribute__((aligned(64)));
char* pbuf = buf;
for (int i = 0; i < NUM_COUNTERS; i++)
if (counters[i])
pbuf += sprintf(pbuf, "%s = %d\n", counter_names[i], counters[i]);
if (pbuf != buf)
printstr(buf);
exit(ret);
}
I am currently struggling with pset2, specifically with vigenere.
Here is my code :
# include <cs50.h>
# include <ctype.h>
# include <stdio.h>
# include <stdlib.h>
# include <string.h>
int main (int argc, string argv[])
{
//Assess the fact that there is only 1 command line argument
if(argc != 2)
{
printf("You should only have 1 command line argument !\n") ;
return 1 ;
}
string k = argv[1] ;
int klength = strlen(k) ;
for(int i = 0; i < klength; i++)
{
if(!isalpha(k[i]))
{
printf("Please make sure the argument is only composed of alphabetical characters\n") ;
return 1 ;
}
}
//Get the text to be crypted
string s = GetString() ;
int slength = strlen(s) ;
//Deliver the crypted text
for( int i = 0, j = 0 ; i < slength ; i++)
{
int kindex = j % klength ;
if(isalpha(s[i]))
{
if(isupper(s[i]))
{
if(isupper(k[kindex]))
{
int crypt = (((s[i] - 'A') + (k[kindex] - 'A') % 26)) + 'A' ;
printf("%c", crypt ) ;
}
else
{
int crypt = (((s[i] - 'A') + (k[kindex] - 'a')) % 26) + 'A' ;
printf("%c", crypt ) ;
}
}
if(islower(s[i]))
{
if(isupper(k[kindex]))
{
int crypt = (((s[i] - 'a') + (k[kindex] - 'A')) % 26) + 'a' ;
printf("%c", crypt) ;
}
else
{
int crypt = (((s[i] - 'a') + (k[kindex] - 'a')) % 26) + 'a' ;
printf("%c", crypt ) ;
}
}
j++ ;
}
else
{
printf("%c" , s[i]) ;
}
}
printf("\n") ;
return 0 ;
}
With check50, here are the errors I receive :
:( encrypts "BaRFoo" as "CaQGon" using "BaZ" as keyword
\ expected output, but not "CakGon\n"
:( encrypts "BARFOO" as "CAQGON" using "BAZ" as keyword
\ expected output, but not "CAkGOh\n"
Here is my sandbox : sandbox
I don't understand why the two outputs are not the same (cakgon vs cakoh) and why it differs from what is expected. The problem probably resides in the part "//Deliver the crypted test".
I have spent a few hours trying to figure it out without success.
Thanks in advance for any help / tip / piece of advice.
Baptiste
I finally get it.
Parentheses were missing before one of the "%26".
I searched internet and found examples with setting only one field of property:
xprop -id "$windowid" -f _NET_WM_STATE 32a -set _NET_WM_STATE _NET_WM_STATE_ABOVE
but how can I set multiple fields?
I tried:
xprop -id "$windowid" -f _NET_WM_STATE 32a -set _NET_WM_STATE '_NET_WM_STATE_ABOVE, _NET_WM_STATE_SKIP_TASKBAR'
and
xprop -id "$windowid" -f _NET_WM_STATE 32aa -set _NET_WM_STATE _NET_WM_STATE_ABOVE,_NET_WM_STATE_SKIP_TASKBAR
and many other variants with no luck.
Is it possible ? :)
Ok....
I wrote a patch for xprop to fix this, and It works, but don't know is it correct.
Thanks to #MichałGórny.
(xprop.c,v 1.6)
--- xprop.c 2012-07-31 11:24:01.178117974 +0400
+++ xprop.mod 2012-07-31 11:23:19.434784430 +0400
## -1487,11 +1487,20 ##
break;
}
case 'a': {
- static Atom avalue;
- avalue = Parse_Atom(value, False);
- type = XA_ATOM;
- data = (unsigned char *) &avalue;
- nelements = 1;
+ static unsigned long data32[MAXELEMENTS];
+ char * value2 = strdup(value);
+ char * tmp = strtok(value2,",");
+ nelements = 0;
+ while( NULL != tmp ){
+ data32[nelements] = Parse_Atom(tmp, False);
+ nelements +=1;
+ if(nelements >= MAXELEMENTS)
+ break;
+ tmp = strtok(NULL,",");
+ }
+ type = XA_ATOM;
+ data = (unsigned char *) data32;
+ free(value2);
break;
}
case 'm':
Looking at the xprop's code, it's not possible.
case 'a': {
static Atom avalue;
avalue = Parse_Atom(value, False);
type = XA_ATOM;
data = (unsigned char *) &avalue;
nelements = 1;
break;
}
This is the code parsing the value to -set.
static Atom
Parse_Atom (const char *name, int only_if_exists)
{
/* may return None = 0 */
return XInternAtom(dpy, name, only_if_exists);
}
So it parses only a single atom.
I've also opened a bug for it; maybe they'll add this.