I set on Windows 7 java_opts -Xmx512M;
the file is about 15Mb and fails in XMLParsing - I am executing from from the command terminal (it times out also in groovyConsole)
Short version of the same file executes correctly on W 7
BTW Unchanged XML code executes correctly on SunOS 64Bit in 24 sec.
can you pls advise what can I do on Windows 7?
Snippet from the code:
import groovy.util.XmlParser
import javax.xml.xpath.*
import groovy.time.*
inpXMLFile='c:/EnvFiles/CCC.xml'
entry=new File("$inpXMLFile")
assert entry.exists()
println " ... file existence validated"
...
def node= new XmlParser().parse( new File( inpXMLFile ) ) // Line 23
// .... the rest of the script
Full trace is enclosed:
... file existence validated
Caught: java.net.ConnectException: Connection timed out: connect
java.net.ConnectException: Connection timed out: connect
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:629)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEntityManager.java:1291)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startDTDEntity(XMLEntityManager.java:1258)
at com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.setInputSource(XMLDTDScannerImpl.java:260)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.dispatch(XMLDocumentScannerImpl.java:1151)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.next(XMLDocumentScannerImpl.java:1047)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:960)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:607)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:116)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:488)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:835)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:764)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1210)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:568)
at compCurrent.run(compCurrent.groovy:23)
Looks like the error is accessing a DTD element and not related to large file size.
java.net.ConnectException: Connection timed out: connect
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startDTDEntity
You have two options:
1. You can remove or comment out DTD reference and entities in your XML file, or
2. Disable parsing DTD and external entities on the XMLParser.
def parser = new XmlParser()
parser.setFeature("http://apache.org/xml/features/nonvalidating/load-dtd-grammar", false)
parser.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false)
Related
I am having a hard time using an environment variable with a space in a properties file read by WildFly (24) in Linux using Oracle 19 in RDS. One like:
SELECT 1 FROM DUAL
The issue is that wildfly won't even parse the file if the spaces are in there with the normal quoting methods.
I have it setup so that variable is in a file called datasource.properties that gets read from standalone.conf where this variable sits:
JAVA_OPTS="$JAVA_OPTS -DDATABASE_CONNECTION_CHECK=${DATABASE_CONNECTION_CHECK}"
It's read in with the following in standalone.conf:
set -a
. /opt/wildfly_config/datasource.properties
set +a
That in turn gets populated in standalone.xml with:
<connection-url>${env.DATABASE_JDBC_URL}</connection-url>
I try putting it in quotes and oddly enough it doesn’t start at all. Standalone.sh is no longer able to parse it:
Error: Could not find or load main class 1 Caused by: java.lang.ClassNotFoundException: 1
I have tried many things such as:
DATABASE_CONNECTION_CHECK="SELECT{ }1{ }FROM{ }DUAL"
DATABASE_CONNECTION_CHECK="'SELECT 1 FROM DUAL'"
DATABASE_CONNECTION_CHECK='SELECT 1 FROM DUAL'
DATABASE_CONNECTION_CHECK="SELECT+1+FROM+DUAL"
DATABASE_CONNECTION_CHECK="SELECT\ 1\ FROM\ DUAL"
DATABASE_CONNECTION_CHECK="\"SELECT 1 FROM DUAL\""
DATABASE_CONNECTION_CHECK="\"'SELECT 1 FROM DUAL'\""
DATABASE_CONNECTION_CHECK="SELECT%201%20FROM%20DUAL"
DATABASE_CONNECTION_CHECK="SELECT\{ }1\{ }FROM\{ }DUAL"
DATABASE_CONNECTION_CHECK='SELECT{ }1{ }FROM{ }DUAL'
DATABASE_CONNECTION_CHECK="'SELECT{ }1{ }FROM{ }DUAL'"
DATABASE_CONNECTION_CHECK="''SELECT{ }1{ }FROM{ }DUAL''"
DATABASE_CONNECTION_CHECK="SELECT%1%FROM%DUAL"
(I realize some of these don't make sense but I was looking for anything different.)
Startup looks good in the log output this with some of these, but then java doesn’t like it, for some reason it sees the escape usage:
Caused by: Error : 936, Position : 9, Sql = SELECT+1+FROM+DUAL, OriginalSql = SELECT+1+FROM+DUAL, Error Msg = ORA-00936: missing expression
or
Caused by: Error : 911, Position : 6, Sql = SELECT%1%FROM%DUAL, OriginalSql = SELECT%1%FROM%DUAL, Error Msg = ORA-00911: invalid character
or
WARN [org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory] (ServerService Thread Pool -- 46) IJ030027: Destroying connection that is not valid, due to the following exception: oracle.jdbc.driver.T4CConnection#2c1456f8: java.sql.SQLException: Non supported SQL92 token at position: 7
This last one is the only one that really netted anything different. I got that with:
DATABASE_CONNECTION_CHECK="SELECT{}1{}FROM{}DUAL"
I can use sed to change the value in the standalone.xml, but all of the other properties I am doing work fine with the exception of this one. I had a hard time with a semicolon in the jdbc string with MSSQL and putting the semicolon in braces like "{;}" fixed that. This DB apparently does not follow the same syntax.
Is there an encoding type that will help this with Oracle and keeps wildfly happy?
EDIT: More tests:
DATABASE_CONNECTION_CHECK=\"SELECT' '1' 'FROM' 'DUAL\"
gets
Caused by: Error : 900, Position : 0, Sql = "SELECT 1 FROM DUAL", OriginalSql = "SELECT 1 FROM DUAL", Error Msg = ORA-00900: invalid SQL statement'
(doesn't seem to like the quotes)
But without the escaping of the quotes I get:
Caused by: Error : 923, Position : 9, Sql = SELECT' '1' 'FROM' 'DUAL, OriginalSql = SELECT' '1' 'FROM' 'DUAL, Error Msg = ORA-00923: FROM keyword not found where expected
A better solution was to change the sourcing of the file from:
set +a
. /opt/PrimeKey/wildfly_config/datasource.properties
set -a
to
. /opt/PrimeKey/wildfly_config/datasource.properties
and make it so all the variables brought in were variables and not properties:
export DATABASE_CONNECTION_CHECK="SELECT 1 FROM DUAL"
On Ubuntu with groovy:
$ groovy --version
Groovy Version: 2.1.9 JVM: 11.0.10 Vendor: Oracle Corporation OS: Linux
I have this script:
myScript.groovy
import org.apache.commons.lang.RandomStringUtils
String charset = (('A'..'Z') + ('0'..'9')).join()
Integer length = 9
String randomString1 = RandomStringUtils.random(length, charset.toCharArray())
String randomString2 = RandomStringUtils.random(length, charset.toCharArray())
String randomString3 = RandomStringUtils.random(length, charset.toCharArray())
From terminal I run it:
$ groovy sampleScript.groovy
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass$3$1 (file:/home/user/.sdkman/candidates/groovy/current/lib/groovy-2.1.9.jar) to method java.lang.Object.finalize()
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass$3$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/home/samples/myScript.groovy: 3: unable to resolve class org.apache.commons.lang.RandomStringUtils
# line 3, column 1.
import org.apache.commons.lang.RandomStringUtils
^
If I search for that in maven central I get:
https://search.maven.org/search?q=org.apache.commons.lang.RandomStringUtils
<dependency>
<groupId>io.github.showthat</groupId>
<artifactId>DHStringUtils</artifactId>
<version>0.1.5</version>
<type>aar</type>
</dependency>
I have then tried to update my script with:
#Grab(group='io.github.showthat', module='DHStringUtils', version='0.1.5')
import org.apache.commons.lang.RandomStringUtils
String charset = (('A'..'Z') + ('0'..'9')).join()
Integer length = 9
String randomString1 = RandomStringUtils.random(length, charset.toCharArray())
String randomString2 = RandomStringUtils.random(length, charset.toCharArray())
String randomString3 = RandomStringUtils.random(length, charset.toCharArray())
But that gives:
General error during conversion: Error grabbing Grapes -- [unresolved dependency: io.github.showthat#DHStringUtils;0.1.5: not found]
java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: io.github.showthat#DHStringUtils;0.1.5: not found]
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAc
Based on:
https://search.maven.org/search?q=org.apache.commons.lang
I have also tried with:
#Grab(group='org.apache.directory.studio', module='org.apache.commons.lang', version='2.6')
But that gives:
General error during conversion: Error grabbing Grapes -- [unresolved dependency: org.apache.directory.studio#org.apache.commons.lang;2.6: not found]
java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: org.apache.directory.studio#org.apache.commons.lang;2.6: not found]
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
Any suggestions?
Guessing from your very old Groovy version you inherited some old
scripts. The packages of apache-commons has changed over time. The
RandomStringUtils are now part of
#Grab(group='org.apache.commons', module='commons-lang3', version='3.12.0')
And the package changed to:
import org.apache.commons.lang3.RandomStringUtils
(note the 3 at the end of lang).
I'd advise to update the deps and adjust the code. If you have to have
to stick with the old code, you might have to search for the file via
findjar or other sites and only take groups/artifacts that clearly state
"apache commons". Anything else might just have copied it and might
not even be complete. So I'd stay clear of anything non-apache-commons.
I am trying to create a ssh session using pexpect.pxssh as follows:
from pexpect import pxssh
connection = pxssh.pxssh()
connection.login('localhost', username, password, port=port, check_local_ip=False)
"""
Fails with the following error
pexpect.pxssh.ExceptionPxssh: Could not establish connection to host
"""
Also I create two sessions one after the other, the first session connects without a problem but the second session fails to connect with the same code. Also, sometimes the code works properly and is able to connect both times. I have also added retries just to be sure that it's not a random event.
Another thing to note is that this code runs without a problem with Python 2 but with Python 3 this happens. I couldn't find any difference in the connection mechanism b/w Python2 and Python3. Any help will be appreciated!
EDIT: After adding logging as per comment:
2021-06-25 10:49:37 INFO - Attempting to connect to device on port 10022.
Connecting to USB device...
Jun 25 10:49:37 tcprelay[203] : Created thread to connect [::1]:10022->[::1]:58316<12> to unix:0<15>
user#localhost's password: xxxxx
Jun 25 10:49:37 tcprelay[203] : Exiting thread to connect [::1]:10022->[::1]:58316 to unix:0
Connecting to USB device...
Jun 25 10:49:38 tcprelay[203] : Created thread to connect [::1]:10022->[::1]:58317<12> to unix:0<15>
user#localhost's password: xxxxx
Jun 25 10:49:39 tcprelay[203] : Exiting thread to connect [::1]:10022->[::1]:58316 to unix:0
The code retries 2 times and then fails.
Note: I am adding a port offset of 10000 using tcprelay
EDIT:
Sorry I was not logging the error properly.
2021-06-25 15:45:34 - ERROR - Failed to connect. Retrying...
2021-06-25 15:45:34 - ERROR - End Of File (EOF). Empty string style platform.
<pexpect.pxssh.pxssh object at 0x127feb0a0>
command: /usr/bin/ssh
args: ['/usr/bin/ssh', '-q', '-oNoHostAuthenticationForLocalhost=yes', '-p', 'xxxxx', '-l', 'xxxxx', 'localhost']
buffer (last 100 chars): b''
before (last 100 chars): b' \r\n'
after: <class 'pexpect.exceptions.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: True
pid: 30020
child_fd: 26
closed: False
timeout: 60
delimiter: <class 'pexpect.exceptions.EOF'>
logfile: <_io.BufferedWriter name='<stdout>'>
logfile_read: None
logfile_send: None
maxread: 2000
ignorecase: False
searchwindowsize: None
delaybeforesend: 0.05
delayafterclose: 0.1
delayafterterminate: 0.1
searcher: searcher_re:
0: re.compile(b'(?i)are you sure you want to continue connecting')
1: re.compile(b'[#$]')
2: re.compile(b'(?i)(?:password:)|(?:passphrase for key)')
3: re.compile(b'(?i)permission denied')
4: re.compile(b'(?i)terminal type')
5: TIMEOUT
Traceback (most recent call last):
File "/src/helpers/utilities.py", line 590, in try_connect_ssh
connection.make_connection(ipaddress=ipaddress, user=user,
File "/src/transport/myssh.py", line 26, in make_connection
self.ssh_process.login(ipaddress, user, password, port=port, sync_multiplier=5, check_local_ip=False)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pexpect/pxssh.py", line 418, in login
i = self.expect(session_regex_array)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pexpect/spawnbase.py", line 343, in expect
return self.expect_list(compiled_pattern_list,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pexpect/spawnbase.py", line 372, in expect_list
return exp.expect_loop(timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pexpect/expect.py", line 179, in expect_loop
return self.eof(e)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pexpect/expect.py", line 122, in eof
raise exc
pexpect.exceptions.EOF: End Of File (EOF). Empty string style platform.
<pexpect.pxssh.pxssh object at 0x127feb0a0>
EDIT:
Using macOS Mojave
pExpect 3.8.0
Device asks for password but after password is sent the connection returns EOF
I am having problems with connection to vehicle. First, I could not connect to the vehicle even with USB (I used "/dev/ttyUSB0" connection string and got an error). Later I got it working with connection string '/dev/serial/by-id/usb-3D_Robotics_PX4_FMU_v2.x_0-if00' and was able to send commands and receive response. Now I want to test it with the telemetry block connected to laptop USB. I tried the same way - with connection string "/dev/serial/by-id/usb-Silicon_Labs_CP2102_USB_to_UART_Bridge_Controller_0001-if00-port0", but it gives timeout message.
USB connection test output:
>>> PreArm: Check FS_THR_VALUE
>>> PreArm: Throttle below Failsafe
>>> APM:Copter V3.5.4 (284349c3)
>>> PX4: 0384802e NuttX: 1bcae90b
>>> Frame: QUAD
>>> PX4v3 0035003B 3136510A 34313630
Mode: STABILIZE
Autopilot Firmware version: APM:Copter-3.5.4
Autopilot capabilities (supports ftp): False
Global Location: LocationGlobal:lat=40.3985757,lon=49.8104986,alt=38.7
Global Location (relative altitude): LocationGlobalRelative:lat=40.3985757,lon=49.8104986,alt=38.7
Local Location: LocationLocal:north=None,east=None,down=None
Attitude: Attitude:pitch=-0.013171303086,yaw=0.0626983344555,roll=-0.0145587390289
Velocity: [-0.01, -0.01, 0.03]
GPS: GPSInfo:fix=3,num_sat=5
Groundspeed: 0.0168827120215
Airspeed: 0.263999998569
Gimbal status: Gimbal: pitch=None, roll=None, yaw=None
Battery: Battery:voltage=0.0,current=None,level=None
EKF OK?: False
Last Heartbeat: 0.967473479002
Rangefinder: Rangefinder: distance=None, voltage=None
Rangefinder distance: None
Rangefinder voltage: None
Heading: 3
Is Armable?: False
System status: STANDBY
Mode: STABILIZE
Armed: False
I am opening a connection like this:
vehicle = connect('/dev/serial/by-id/usb-Silicon_Labs_CP2102_USB_to_UART_Bridge_Controller_0001-if00-port0', wait_ready=True)
This results in the following traceback:
>>> Link timeout, no heartbeat in last 5 seconds
>>> No heartbeat in 30 seconds, aborting.
Traceback (most recent call last):
File "x.py", line 6, in <module>
vehicle = connect('/dev/serial/by-id/usb-Silicon_Labs_CP2102_USB_to_UART_Bridge_Controller_0001-if00-port0', wait_ready=True)
File "/home/seyid/.local/lib/python2.7/site-packages/dronekit/__init__.py", line 2845, in connect
vehicle.initialize(rate=rate, heartbeat_timeout=heartbeat_timeout)
File "/home/seyid/.local/lib/python2.7/site-packages/dronekit/__init__.py", line 2117, in initialize
raise APIException('Timeout in initializing connection.')
dronekit.APIException: Timeout in initializing connection.
Telemetry block is working when using MavProxy.
What is the problem here? Thank you
There are a couple of problems that can cause dronekit to fail with a connection timeout:
Ensure you have the pyserial module installed.
Specify the baud rate for your connection explicitly, as in:
vehicle = connect('/dev/ttyUSB0',
wait_ready=True,
baud=57600,
)
If connections with mavproxy to the same serial port work on your system it is likely that the second one is the culprit.
I am using requests stream for preforming a 'GET' download of a remote very large CSVs, then chunking response using response.iter_content(). This has been working for multiple data providers.
However, for one remote data provider, when using response.iter_content(), occasionally I am getting a ChunkedEncodingError, specifically:
ChunkedEncodingError: (
ProtocolError(
'Connection broken: IncompleteRead(921 bytes read, 103 more expected)',
IncompleteRead(921 bytes read, 103 more expected)),
)
Here is the Python 3 code, and I would like to know of an alternative to resolving this chunking exception problem:
tmp_csv_chunk_sum = 0
with open(
file=tmp_csv_file_path,
mode='wb',
encoding=encoding_write
) as csv_file_wb:
try:
for chunk in response.iter_content(chunk_size=8192):
if not chunk:
break
tmp_csv_chunk_sum += 8192
csv_file_wb.write(chunk)
csv_file_wb.flush()
os.fsync(csv_file_wb.fileno())
except Exception as ex:
self.logger.error(
"Request CSV Download: Exception",
extra={
'error_details': str(ex),
'chunk_total_sum': tmp_csv_chunk_sum
}
)
raise
I truly appreciate assistance, Thank you