New style format specifier throwing tokenization error - string

I have the below string in the variable jss.
jss = '''
Java.perform(
function()
{
var item = Java.use("java.util.Random");
console.log("HOOKING random");
item.nextInt.overload("int").implementation = function(a)
{
var ret = this.nextInt(a);
return {0};
}
}
);
'''.format("1234")
and I am trying to use a format specifier to simply pass it some value ("1234" in this case).
When I try running this in iPython, I get the following error:
ERROR:root:An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line string', (1, 0))
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-1-a900f760c449> in <module>()
12 }
13 );
---> 14 '''.format("1234")
KeyError: '\n var item = Java'
Not sure what is going wrong here. Can someone please help understand?

you need to escape your other braces with {{ and }}- otherwise the python format tokenizer wants to make sense of the pattern. (only tested in python3...)
jss = '''
Java.perform(
function()
{{
var item = Java.use("java.util.Random");
console.log("HOOKING random");
item.nextInt.overload("int").implementation = function(a)
{{
var ret = this.nextInt(a);
return {0};
}}
}}
);
'''.format("1234")
assuming the braces you do not want to use as formatting options are always preceeded by more than one whitespace (and the format braces are preceeded by one whitespace at most) you could replace those:
import re
# with your original jss
sanitized = re.sub('(\s\s+){', r'\1{{', jss)
sanitized = re.sub('(\s\s+)}', r'\1}}', sanitized)
print(sanitized.format("1234"))
...there may be much more sensible regular expressions to suit your needs...

Related

How to parse markdown to json using remark

The remark site has a link to an AST explorer for the output of remark - https://astexplorer.net/#/gist/0a92bbf654aca4fdfb3f139254cf0bad/ffe102014c188434c027e43661dbe6ec30042ee2
What I cannot find is how to do the parsing to AST - all the examples convert to html.
I have this code
import {unified} from 'unified'
import remarkParse from 'remark-parse'
import remarkGfm from 'remark-gfm' // git flavoured markdown
const content = `
# My header
This is my content
- abc
- def
`;
unified()
.use(remarkParse)
.use(remarkGfm)
.process('# Hi\n\n*Hello*, world!')
.then((file) => {
console.log(String(file))
})
but am getting a couple of errors here that I do not know how to get around
[remark-gfm] Warning: please upgrade to remark 13 to use this plugin
file:///markdown/node_modules/unified/lib/index.js:520
throw new TypeError('Cannot `' + name + '` without `Compiler`')
^
TypeError: Cannot `process` without `Compiler`
You almost have it. Below I've simplified your code, removing unused and unnecessary parts.
import {unified} from 'unified'
import remarkParse from 'remark-parse'
let myResult = unified()
.use(remarkParse)
.parse('# Hi\n\n*Hello*, world!');
console.log(JSON.stringify(myResult, null, " "));
According to ChristianMurphy's GitHub Q/A:
unified.process() will try to take text, turn it into an AST, and back into text.
For your stated purpose, "...parsing to AST [JSON]", you don't need or want the "full-cycle process" that unified.process() is failing to complete, TypeError: Cannot 'process' without 'Compiler'. You merely want to parse the input and emit the syntax tree (AST) as JSON. The error here is because process(), after parsing your input (markdown string) and turning it into a syntax tree (AST), is then trying to "compile" it to some output format. However, you haven't supplied a compiler. But, as I understand your post, you don't need or want to compile the syntax tree into another output (language). So, change from process() to parse() and emit the resulting syntax tree as JSON.

Python: [Informix][Informix ODBC Driver]Invalid string or buffer length. SQLCODE=-11071 when I fetch data

When I try to retreive my table in informix with ifxpy package, I get this error:
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
<ipython-input-6-b0557e7f099b> in <module>
16 while dictionary != False:
17 tph.append(dictionary)
---> 18 dictionary = IfxPy.fetch_assoc(stmt)
19 print(pd.DataFrame(tph))
Exception: [Informix][Informix ODBC Driver]Invalid string or buffer length. SQLCODE=-11071
This is my code:
import IfxPy
import pandas as pd
ConStr = "SERVER=informix1;DATABASE=ir_fmois;HOST=127.0.0.1;SERVICE=9092;UID=informix;PWD=1234;"
# netstat -a | findstr 9088
try:
# netstat -a | findstr 9088
conn = IfxPy.connect( ConStr, "", "")
except Exception as e:
print ('ERROR: Connect failed')
print ( e )
quit()
sql = "SELECT * FROM oih"
stmt = IfxPy.exec_immediate(conn, sql)
dictionary = IfxPy.fetch_assoc(stmt)
tph=[]
while dictionary != False:
tph.append(dictionary)
dictionary = IfxPy.fetch_assoc(stmt)
print(pd.DataFrame(tph))
When I print the dataframe I see that I got only the first 4 rows.
I tried also this code and i doesn't trhow any exception but it returns also the first 4 rows of my table:
import IfxPyDbi as dbapi2
ConStr = "SERVER=informix1;DATABASE=ir_fmois;HOST=127.0.0.1;SERVICE=9092;UID=informix;PWD=1234;"
conn = dbapi2.connect( ConStr, "", "")
cur = conn.cursor()
sql = "SELECT * FROM oih"
rows = cur.fetchall()
len(rows)
>>>4
EDIT
I tried importing columns one by one, and the same error has occured when I tried to select a byte (blob) column (with text data type). This column has emty values in the first 4 rows but the fifth wasn't empty, and I think this is why the error occured in line 5.
I would really appreciate if anyone has any idea on how to solve this.
The finderr command reports:
$ finderr -11071
-11071 Invalid string or buffer length.
This Informix CLI error code is the same as SQLSTATE value S1090. The
following functions can return this error code: SQLBindCol(),
SQLBindParameter(), SQLBrowseConnect(), SQLColAttributes(),
SQLColumnPrivileges(), SQLColumns(), SQLConnect(), SQLDataSources(),
SQLDescribeCol(), SQLDriverConnect(), SQLDrivers(), SQLExecDirect(),
SQLExecute(), SQLForeignKeys(), SQLGetCursorName(), SQLGetData(),
SQLGetInfo(), SQLNativeSql(), SQLPrepare(), SQLPrimaryKeys(),
SQLProcedureColumns(), SQLProcedures(), SQLPutData(), SQLSetCursorName(),
SQLSpecialColumns(), SQLStatistics(), SQLTablePrivileges(), and SQLTables().
The value specified for the argument cbValueMax is less than zero.
Supply a value for the argument cbValueMax that is zero or greater.
For all functions, an argument that specified a string or buffer length, such
as cbCursor, cbConnStrIn, or cbSqlStr, had one or more of the following
problems: (1) It was less than 0, (2) It was less than 0 but not equal to
SQL_NTS or SQL_NULL_DATA, (3) It was less than 0 but the corresponding
pointer was not a null pointer, (4) It was equal to 1, or (5) It was too
large. Set the string or buffer length to a valid value.
Additionally for SQLExecDirect() and SQLExecute(), a parameter value that was
set with SQLBindParameter() had one of the following problems: (1) It was a
null pointer, and the parameter length was not 0, SQL_NULL_DATA,
SQL_DATA_AT_EXEC, or less than or equal to SQL_LEN_DATA_AT_EXEC_OFFSET, or
(2) It was not a null pointer, and the parameter length was less than 0 but
was not SQL_NTS, SQL_NULL_DATA, SQL_DATA_AT_EXEC, or less than or equal to
SQL_LEN_EXEC_DATA_AT_EXEC_OFFSET. Set the parameter value to a valid value.
$
Since your Python code is not calling any of those functions directly, that suggests there is a bug in the ifxpy driver. It is probably calling one of the listed functions with an incorrect argument.
As such, you should probably report it as an 'issue' on the GitHub site for the driver: https://github.com/OpenInformix/IfxPy

psycopg2 named cursor withhold=True

I am using a named cursor to fetch 200K+ rows, and using the attribute, 'withhold=True', this way I can iterate by fetching many (50K) at a time - but my cursor is not persisting...
Here is the error / stacktrace
Traceback (most recent call last):
File "/home/me/code/etl/etl.py", line 179, in main
_pg_data = _fetch(_some)
psycopg2.ProgrammingError: named cursor isn't valid anymore
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/me/code/etl/etl.py", line 330, in <module>
main()
File "/home/me/code/etl/etl.py", line 271, in main
logging.error(Fore.LIGHTRED_EX + e + Fore.RESET, exc_info=True)
TypeError: must be str, not ProgrammingError
Here is my code
from colorama import Fore
from datetime import datetime
import argparse, logging, psycopg2, pyodbc, sys, time, yaml
import _classes.Utils as cpu
def main():
_cfg_path = "/home/me/code/etl/db.yml"
with open(_cfg_path, 'r') as _ymlfile:
_cfg = yaml.load(_ymlfile, Loader=yaml.CLoader)
# create a connection to the database
_conn = psycopg2.connect("host={0} dbname={1} user={2} password={3} port={4}".format(_cfg['local_postgres']['host'], _cfg['local_postgres']['db'],
_cfg['local_postgres']['user'], _cfg['local_postgres']['passwd'],
_cfg['local_postgres']['port']))
_curs_pgsql = _conn.cursor()
_curs_pgsql.callproc('usp_outbound', ['curs'])
_curs2_pgsql = _conn.cursor('curs', withhold=True)
_push_date = datetime.now().strftime("%Y-%m-%d")
_some = 50000
_fetch = _curs2_pgsql.fetchmany
while True:
_pg_data = _fetch(_some)
if not _pg_data:
break
for _row in _pg_data:
_params = ()
_sql = "INSERT INTO dbo.tbl VALUES (?, ?, ?)"
_params = (_row[0], _row[1], _row[2])
# ...insert into destination database
# ...now update source database and set the push and push date flags
_curs_pgsql.execute("UPDATE products SET pushed = TRUE, pushed_date = (%s) WHERE id = (%s)", (_push_date, _row[2],))
_conn.commit()
if _conn:
# close cursor / close the communication with the PostgreSQL database server
_curs2_pgsql.close()
_curs_pgsql.close()
_conn.close()
Clearly I am missing something with my named cursor and how it's supposed to be defined...
According to the documentation -
Set the value before calling execute() or use the connection.cursor() withhold parameter, otherwise the value will have no effect.
... ... ...
Trying to fetch from a named cursor after a commit() or to create a named cursor when the connection is in autocommit mode will result in an exception. It is possible to create a WITH HOLD cursor by specifying a True value for the withhold parameter to cursor() or by setting the withhold attribute to True before calling execute() on the cursor.
What am I missing?
The code seems reasonable. It's hard to say what the stored procedure (usp_outbound) might be doing though. Your named cursor is being created after the procedure - is that creating it? Is something happening there that might close it? Perhaps the stored procedure needs a WITH HOLD?
Try reorganizing the code to something like this and see if it helps (or you get an error that provides a hint).
with psycopg2.connect("your_connection_string") as _conn:
# first (unnamed) cursor
_curs_pgsql = _conn.cursor()
with _conn.cursor(name='curs', withhold=True) as _curs2_pgsql:
_curs2_pgsql.itersize = 50000 # fetch rows from database in batches of this value
_push_date = datetime.now().strftime("%Y-%m-%d")
_curs_pgsql.callproc('usp_outbound', ['curs'])
for _row in _curs2_pgsql:
_sql = "INSERT INTO dbo.tbl VALUES (?, ?, ?)"
_params = (_row[0], _row[1], _row[2])
# ...insert into destination database
# ...now update source database and set the push and push date flags
_curs_pgsql.execute("UPDATE products SET pushed = TRUE, pushed_date = (%s) WHERE id = (%s)", (_push_date, _row[2],))
# if there are no errors, connection will automatically
# commit() here at end of with block (but remain open)
_conn.close()

Groovy: Get a specific value in a file

I'm quite new in Groovy.
Basically I load a text file, then I need to get a specific value at one line (actually the 6th).
The line is like:
STATIC_ASSERT(VERSION == 888888, "blablabla");
I need to get the 888888 value.
I found a way using multiple split but it's ugly.
I also think of using something like:
line.substring(ind+"VERSION ==".length(), line.length()-10).trim();
But the "blablabla" length can change..
Thank you.
Edit: It works using an hardcoded string like this.
But when I try to run it from the file I get this error:
test' is failed: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
script1516208488151762512206.groovy: 4: expecting '}', found '' # line 4, column 69.
ne.contains('VERSION ==')
^
1 error
Here is my code:
${groovy:
String Ver
def file = new File("C:\\test.cpp")
def data = file.filterLine { line ->
line.contains('VERSION ==')
}
Ver = data.split("==")[1].split(",")[0].trim()
logger.info(Ver)
}
--
I also tried something like this:
${groovy:
String Ver
def file = new File("C:\\test.cpp")
while ((line = file.readLine())!=null)
{
int ind = line.indexOf("VERSION ==")
if (ind >= 0)
{
Ver = line.split("==")[1].split(",")[0].trim()
}
}
logger.info(Ver)
}
But I get same kind of weird error:
expecting '}', found '' # line 9, column 58.
("==")[1].split(",")[0].trim()
^
1 error
:(
You do as shown below:
def line = 'STATIC_ASSERT(VERSION == 888888, "blablabla");'
println line.split('==')[1].split(',')[0].trim()

Unexpected character encountered while parsing value: c. Path '', line 0, position 0.

I know the question is asked before but i tried all the options and could not get ride of it . my code is something like this
WebClient client = new WebClient();
string url = "https://demos.telerik.com/kendo-ui/service/StockData";
string EncryptedJson = client.DownloadString(url);
var dataresponse = (JObject)JsonConvert.DeserializeObject(EncryptedJson);
and at deserialzeObject it is throwing the exception
Unexpected character encountered while parsing value: c. Path '', line
0, position 0.
while the client.DownloadString(url) is giving data in the following format
callback([{"Date":"/Date(1196467200000)/","Close":40.635,"Volume":1650185491,"Open":40.640,"High":40.680,"Low":39.090}])
That's not JSON, it's JSONP.
Remove the callback( at the start and the ) at the end to make it JSON:
EncryptedJson = EncryptedJson.Substring(9, EncryptedJson.Length - 10);

Resources