I have an instance of SOLR 3.6 running, with JSON as the default updateHandler.
I am able to delete individual documents with the following:
curl "http://myURL/update?commit=true" -H 'Content-type:application/json' -d '{"delete": {"id":"1730887464"}}'
What is the right way to delete a range of IDs?
I have tried the following:
curl "http://myURL/update?commit=true" -H 'Content-type:application/json' -d '{"delete": {"id":"[* TO 1730887464]"}}'
curl "http://myURL/update?commit=true" -H 'Content-type:application/json' -d '{"delete": {"id":"[0 TO 1730887464]"}}'
And I get the following exception. Any help/pointers would be appreciated.
I've looked up:
http://wiki.apache.org/solr/UpdateJSON
http://lucene.apache.org/solr/api/doc-files/tutorial.html
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 500 For input string: "[* TO 1730887464]"
java.lang.NumberFormatException: For input string: "[* TO 1730887464]"
at java.lang.NumberFormatException.forInputString(NumberFormatException. java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:295 )
at org.apache.solr.schema.TrieField.toInternal(TrieField.java:307)
at org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandle r2.java:288)
at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(Run UpdateProcessorFactory.java:68)
at org.apache.solr.update.processor.LogUpdateProcessor.processDelete(Log UpdateProcessorFactory.java:137)
at org.apache.solr.handler.JsonLoader.processUpdate(JsonLoader.java:120)
at org.apache.solr.handler.JsonLoader.load(JsonLoader.java:75)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co ntentStreamHandlerBase.java:58)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl erBase.java:129)
at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handle Request(RequestHandlers.java:244)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter .java:365)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte r.java:260)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet Handler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3 99)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav a:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1 82)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7 66)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand lerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection. java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1 52)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54 2)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnectio n.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector. java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.j ava:582)
</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing /solr/update. Reason:
<pre> For input string: "[* TO 1730887464]"
java.lang.NumberFormatException: For input string: "[* TO 1730887464]"
at java.lang.NumberFormatException.forInputString(NumberFormatException. java:65)
at java.lang.Long.parseLong(Long.java:438)
at java.lang.Long.parseLong(Long.java:478)
at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:295 )
at org.apache.solr.schema.TrieField.toInternal(TrieField.java:307)
at org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandle r2.java:288)
at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(Run UpdateProcessorFactory.java:68)
at org.apache.solr.update.processor.LogUpdateProcessor.processDelete(Log UpdateProcessorFactory.java:137)
at org.apache.solr.handler.JsonLoader.processUpdate(JsonLoader.java:120)
at org.apache.solr.handler.JsonLoader.load(JsonLoader.java:75)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co ntentStreamHandlerBase.java:58)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl erBase.java:129)
at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handle Request(RequestHandlers.java:244)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter .java:365)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte r.java:260)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet Handler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3 99)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav a:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1 82)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7 66)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand lerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection. java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1 52)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:54 2)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnectio n.java:945)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector. java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.j ava:582)
you need to use a delete by query:
curl "http://myURL/update?commit=true" -H 'Content-type:application/json' -d '{"delete": {"query":"id:[* TO 1730887464]"}}'
I don't work a lot with json so not sure if the 'query' part is right...but I guess you see what i mean
Related
The USGS has a page for user to make GET queries:
https://nationalmap.gov/epqs/
I attempted to fill the 4 fields provided:
X: -96.808971 // longitude
Y: 32.7792009 // Latitude
Units: Feet
Output: XML
After clicking on "Get Elevation" I get an error message:
"This XML file does not appear to have any style information associated with it. The document tree is shown below."
<USGS_Elevation_Point_Query_Service>
<Elevation_Query x="-96.808971" y="32.7792009">
<Data_Source>3DEP 1/3 arc-second</Data_Source>
<Elevation>420.61</Elevation>
<Units>Feet</Units>
</Elevation_Query>
</USGS_Elevation_Point_Query_Service>
They provide a sample URL GET:
"The following is a sample HTTP GET request and response. The placeholders shown need to be replaced with actual values."
GET pqs.php?x=string&y=string&units=string&output=string HTTP/1.1
Host: nationalmap.gov/epqs/pqs.php
That is where I am stuck.
I have curl and and urlgrabber. Have only used them for simple requests.
TIA
$ curl 'https://nationalmap.gov/epqs/pqs.php?x=-96.808971&y=32.7792009&units=Feet&output=xml'
<?xml version="1.0" encoding="utf-8" ?><USGS_Elevation_Point_Query_Service><Elevation_Query x="-96.808971" y="32.7792009"><Data_Source>3DEP 1/3 arc-second</Data_Source><Elevation>420.61</Elevation><Units>Feet</Units></Elevation_Query></USGS_Elevation_Point_Query_Service>
if you want it prettified, you could use php to format it, eg:
$ curl 'https://nationalmap.gov/epqs/pqs.php?x=-96.808971&y=32.7792009&units=Feet&output=xml' -s | php -r '$domd=#DOMDocument::loadXML(stream_get_contents(STDIN));$domd->formatOutput=1;echo $domd->saveXML();'
<?xml version="1.0" encoding="utf-8"?>
<USGS_Elevation_Point_Query_Service>
<Elevation_Query x="-96.808971" y="32.7792009">
<Data_Source>3DEP 1/3 arc-second</Data_Source>
<Elevation>420.61</Elevation>
<Units>Feet</Units>
</Elevation_Query>
</USGS_Elevation_Point_Query_Service>
or if you just want the raw data, you can parse it with PHP like
curl 'https://nationalmap.gov/epqs/pqs.php?x=-96.808971&y=32.7792009&units=Feet&output=json' -s | php -r '$data=json_decode(stream_get_contents(STDIN));echo json_encode($data->USGS_Elevation_Point_Query_Service->Elevation_Query,-1);'
{
"x": -96.808971,
"y": 32.7792009,
"Data_Source": "3DEP 1/3 arc-second",
"Elevation": 420.61,
"Units": "Feet"
}
.. or if you just want the elevation,
$ curl 'https://nationalmap.gov/epqs/pqs.php?x=-96.808971&y=32.7792009&units=Feet&output=json' -s | php -r '$data=json_decode(stream_get_contents(STDIN));echo $data->USGS_Elevation_Point_Query_Service->Elevation_Query->Elevation;'
420.61
There is probably a simple solution to my problem, but I am very new to python3 so please go easy on me;)
I have a simple script running, which already successfully parses information from an xml-file using this code
import xml.etree.ElementTree as ET
root = ET.fromstring(my_xml_file)
u = root.find(".//name").text.rstrip()
print("Name: %s\n" % u)
The xml I am parsing looks like this
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="/3.2/style/exchange.xsl"?>
<example:world-data xmlns="http://www.example.org" xmlns:ops="http://example.oorg" xmlns:xlink="http://www.w3.oorg/1999/xlink">
<exchange-documents>
<exchange-document system="acb.org" family-id="543672" country="US" doc-number="95962" kind="B2">
<bibliographic-data>
<name>SomeName</name>
...and so on... and ends like this
</exchange-document>
</exchange-documents>
</example:world-data>
(Links are edited due to stackoverflow policy)
Output as expected
SomeName
However, if I try to parse another xml from the same api using the same python commands, I get this error-code
AttributeError: 'NoneType' object has no attribute 'text'
The second xml-file looks like this
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="/3.2/style/pub-ftxt-claims.xsl"?>
<ops:world-data xmlns="http://www.example.org/exchange" xmlns:example="http://example.org" xmlns:xlink="http://www.example.org/1999/xlink">
<ftxt:fulltext-documents xmlns="http://www.examp.org/fulltext" xmlns:ftxt="ww.example/fulltext">
<ftxt:fulltext-document system="example.org" fulltext-format="text-only">
<bibliographic-data>
<publication-reference data-format="docdb">
<document-id>
<country>EP</country>
<doc-number>10000</doc-number>
<kind>A</kind>
</document-id>
</publication-reference>
</bibliographic-data>
<claims lang="EN">
<claim>
<claim-text>1. Some text.</claim-text>
<claim-text>2. Some text.</claim-text>
<claim-text>2. Some text.</claim-text>
</claim>
</claims>
</ftxt:fulltext-document>
</ftxt:fulltext-documents>
</ops:world-data>
I tried again
root = ET.fromstring(usr_str)
u = root.find(".//claim-text").text.rstrip()
print("Abstract: %s\n" % u)
Expected output
1. Some text.
But it only prints the above mentioned error message.
Why can I parse the first xml but not the second one using these commands?
Any help is highly appreciated.
edit: code by Jack Fleeting works in python console, but unfortunately not in my PyCharm
from lxml import etree
root = etree.XML(my_xml.encode('ascii'))
root2 = etree.XML(my_xml2.encode('ascii'))
root.xpath('//*[local-name()="name"]/text()')
root2.xpath('//*[local-name()="claim-text"]/text()')
Could this be a bug in my PyCharm? My first mentioned code snippet still prints a correct result for name...
edit: Turns out I had to force the output using
a = root3.xpath('//*[local-name()="claim-text"]/text()')
print(a, flush=True)
A couple of problems here before we get to a possible solution. One, the first xml snippet you provided is invalid (for instance, the <bibliographic-data> isn't closed). I realize it's just a snippet but since this is what we have to work with, I modified the snippet below to fix that. Two, both snippets have xmlns declaration with unbound (unused) prefixes (example:world-datain the first, and ops:world-data in the second). I had to remove these prefixes, too, for the rest to work.
Given these modifications, using the lxml library should work for you.
First modified snippet:
my_xml = """<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="/3.2/style/exchange.xsl"?>
<world-data xmlns="http://www.example.org" xmlns:ops="http://example.oorg" xmlns:xlink="http://www.w3.oorg/1999/xlink">
<exchange-documents>
<exchange-document system="acb.org" family-id="543672" country="US" doc-number="95962" kind="B2">
<bibliographic-data>
<name>SomeName</name>
...and so on... and ends like this
</bibliographic-data>
</exchange-document>
</exchange-documents>
</world-data>"""
And:
my_xml2 = """<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="/3.2/style/pub-ftxt-claims.xsl"?>
<world-data xmlns="http://www.example.org/exchange" xmlns:example="http://example.org" xmlns:xlink="http://www.example.org/1999/xlink">
<ftxt:fulltext-documents xmlns="http://www.examp.org/fulltext" xmlns:ftxt="ww.example/fulltext">
<ftxt:fulltext-document system="example.org" fulltext-format="text-only">
<bibliographic-data>
<publication-reference data-format="docdb">
<document-id>
<country>EP</country>
<doc-number>10000</doc-number>
<kind>A</kind>
</document-id>
</publication-reference>
</bibliographic-data>
<claims lang="EN">
<claim>
<claim-text>1. Some text.</claim-text>
<claim-text>2. Some text.</claim-text>
<claim-text>3. Some text.</claim-text>
</claim>
</claims>
</ftxt:fulltext-document>
</ftxt:fulltext-documents>
</world-data>"""
And now to work:
from lxml import etree
root = etree.XML(my_xml.encode('ascii'))
root2 = etree.XML(my_xml2.encode('ascii'))
root.xpath('//*[local-name()="name"]/text()')
output:
['SomeName']
root2.xpath('//*[local-name()="claim-text"]/text()')
Output:
['1. Some text.', '2. Some text.', '3. Some text.']
Eclipse Hono and Eclipse Ditto have been connected successfully. And when I try to send the data from the Hono I will get 202 Accepted response as shown below.
(base) vignesh#nb907:~$ curl -X POST -i -u sensor9200#tenantSensorAdaptersss:mylittle -H 'Content-
Type: application/json' -d '{"temp": 23.07, "hum": 45.85122}' http://localhost:8080/telemetry
HTTP/1.1 202 Accepted
content-length: 0
But when I checked for the digital twin value using localhost:8080/api/2/testing.ditto:9200 - It is not getting updated.
I came across the error while enquiring the logs.
connectivity_1 | 2019-10-14 15:18:26,273 INFO [ID:AMQP_NO_PREFIX:TelemetrySenderImpl-7]
o.e.d.s.c.m.a.AmqpPublisherActor akka://ditto-
cluster/system/sharding/connection/27/Amma123465/pa/$a/c1/amqpPublisherActor2 - Response dropped,
missing replyTo address: UnmodifiableExternalMessage [headers={content-
type=application/vnd.eclipse.ditto+json, orig_adapter=hono-http, orig_address=/telemetry,
device_id=9200, correlation-id=ID:AMQP_NO_PREFIX:TelemetrySenderImpl-7}, response=true, error=true,
authorizationContext=null, topicPath=ImmutableTopicPath [namespace=unknown, id=unknown, group=things,
channel=twin, criterion=errors, action=null, subject=null, path=unknown/unknown/things/twin/errors],
enforcement=null, headerMapping=null, sourceAddress=null, payloadType=TEXT, textPayload=
{"topic":"unknown/unknown/things/twin/errors","headers":{"content-
type":"application/vnd.eclipse.ditto+json","orig_adapter":"hono-
http","orig_address":"/telemetry","device_id":"9200","correlation-
id":"ID:AMQP_NO_PREFIX:TelemetrySenderImpl-7"},"path":"/","value":
{"status":400,"error":"json.field.missing","message":"JSON did not include required </path>
field!","description":"Check if all required JSON fields were set."},"status":400}, bytePayload=null']
gateway_1 | 2019-10-14 15:19:47,927 WARN [b9774050-48ae-45c4-a937-68a70f8defe5]
o.e.d.s.g.s.a.d.DummyAuthenticationProvider - Dummy authentication has been applied for the following
subjects: nginx:ditto
gateway_1 | 2019-10-14 15:19:47,949 INFO [b9774050-48ae-45c4-a937-68a70f8defe5]
o.e.d.s.m.c.a.ConciergeForwarderActor akka://ditto-cluster/user/gatewayRoot/conciergeForwarder -
Sending signal with ID <testing.ditto:9200> and type <things.commands:retrieveThing> to concierge-
shard-region
gateway_1 | 2019-10-14 15:19:48,044 INFO [b9774050-48ae-45c4-a937-68a70f8defe5]
o.e.d.s.g.e.HttpRequestActor akka://ditto-cluster/user/$C - DittoRuntimeException
<things:precondition.notmodified>: <The comparison of precondition header 'if-none-match' for the
requested Thing resource evaluated to false. Expected: '"rev:1"' not to match actual: '"rev:1"'.>.
I have set the all the json fields. But not sure what I am missing.
I also can see this in the log
nginx_1 | 172.18.0.1 - ditto [14/Oct/2019:13:19:48 +0000] "GET
/api/2/things/testing.ditto:9200 HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36"
Please let me know if I am missing something.
Did you send the message in Ditto protocol or applied payload transformation?
Looks like a duplicate of Connecting Eclipse Hono to Ditto - "description":"Check if all required JSON fields were set."},"status":400}" Error where you had the same problem and error before.
I have a small test page for Brython related tests and recently added xml.etree.Elementree module there, but it doesn't work for some reason.
I've following code (actually there is more stuff but I removed the irrelevant parts):
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<link rel="stylesheet" type="text/css" href="/js/jquery-ui-1.12.1/jquery-ui.min.css" media="screen" />
<link rel="stylesheet" type="text/css" href="/js/jquery-ui-1.12.1/jquery-ui.structure.min.css" media="screen" />
<link rel="stylesheet" type="text/css" href="/layout.css" media="screen" />
<link rel="stylesheet" type="text/css" href="/visual.css" media="screen" />
<link rel="stylesheet" type="text/css" href="/js/jquery-ui-1.12.1/jquery-ui.theme.min.css" media="screen" />
<script rel="script" type="text/javascript" src="/js/jquery-3.1.1.min.js"></script>
<script rel="script" type="text/javascript" src="/js/jquery-ui-1.12.1/jquery-ui.min.js"></script>
<script rel="script" type="text/javascript" src="/js/brython/brython.js"></script>
<script rel="script" type="text/javascript" src="/js/brython/brython_stdlib.js"></script>
</head>
<body onload='brython()'>
<script type="text/python" charset="utf-8">
import sys
from browser import alert, document as docu
from browser import ajax
from xml.etree import ElementTree as ET
def ajaxReceive(req):
alert("Input value: " + docu["numinput"].value)
alert('Ajax response: \n %s' % req.text )
if req.status == 200 or req.status == 0:
d = docu['messagebox']
d.clear()
r = ET.fromstring(req.text)
#n = r.findall('./person/name')
#a = r.findall('./person/age')
#d.text = 'Dude %s is %s old.' % (n,a)
else:
docu['messagebox'] <= 'error: ' + req.text
def ajaxSend():
req = ajax.ajax()
url = '/bryt/'
x = 1
y = 2
z = docu['numinput']
req.open('POST', url, True)
req.bind('complete', ajaxReceive)
req.set_header('content-type', 'application/x-www-form-urlencoded' )
req.send( {
'action': 'calc',
'x': x,
'y': y,
'z': z.value
})
docu['ajaxbutton'].bind('click', ajaxSend )
d = docu['messagebox']
d.clear()
d.text = 'ready'
</script>
<div id='messagebox' style='width: 20%; border: 1px solid gray; padding: 2em;' >
</div>
<br/>
<input id="numinput" / >
<br/>
<br/>
<button id='ajaxbutton' >Ajax run</button>
</body>
</html>
And at server side it only adds up 3 to given number. The problem is XML-formatted Ajax-response that is being received. It comes in in clear XML, but even building a etree root-element by calling .fromstring() funtion, it tracebacks as follows:
Error 500 means that Python module pyexpat was not found at url http://example.com/bryt/pyexpat.py
brython.js:7171:1
Error 500 means that Python module pyexpat was not found at url http://example.com/bryt/pyexpat/__init__.py
brython.js:7171:1
Error 500 means that Python module pyexpat was not found at url http://example.com/js/brython/Lib/site-packages/pyexpat.py
brython.js:7171:1
Error 500 means that Python module pyexpat was not found at url http://example.com/js/brython/Lib/site-packages/pyexpat/__init__.py
brython.js:7171:1
Error for module xml.parsers.expat
brython.js:7242:21
Error:
Stack trace:
_b_.ImportError.$factory#http://example.com/js/brython/brython.js line 6466 > eval:49:371
import_hooks#http://example.com/js/brython/brython.js:11605:7
$B.$__import__#http://example.com/js/brython/brython.js:7430:33
$B.$import#http://example.com/js/brython/brython.js:7460:43
$module<#http://example.com/js/brython/brython.js line 7242 > eval:14:9
#http://example.com/js/brython/brython.js line 7242 > eval:1:16
run_py#http://example.com/js/brython/brython.js:7242:1
exec_module#http://example.com/js/brython/brython.js:7276:1
cl_method#http://example.com/js/brython/brython.js:4729:43
import_hooks#http://example.com/js/brython/brython.js:11629:5
$B.$__import__#http://example.com/js/brython/brython.js:7430:33
$B.$import#http://example.com/js/brython/brython.js:7473:5
__init__205#http://example.com/js/brython/brython.js line 7242 > eval:5653:25
type.__call__#http://example.com/js/brython/brython.js:4674:20
factory#http://example.com/js/brython/brython.js:4741:47
XML194#http://example.com/js/brython/brython.js line 7242 > eval:5190:41
ajaxReceive3#http://example.com/js/brython/brython.js line 4294 > eval:176:32
#http://example.com/js/brython/brython.js line 7188 > eval:69:24
ajax.$factory/xmlhttp.onreadystatechange#http://example.com/js/brython/brython.js line 7188 > eval:161:13
brython.js:7243:1
16:21:17.002 args
Array [ "No module named pyexpat" ]
And there has been something similar in the past brython issue 613
where Pierre states that there is no such thing as pure python pyexpat (July 2017). However Brython standard distribution lists Lib/xml/etree and expat.py - does it mean that it's still not available?
Brython Lib/xml/etreeElementTree.py lines 1511 and onwards starts with:
class XMLParser:
def __init__(self, html=0, target=None, encoding=None):
try:
from xml.parsers import expat
except ImportError:
try:
import pyexpat as expat
except ImportError:
raise ImportError(
"No module named expat; use SimpleXMLTreeBuilder instead"
)
In my understanding it should succeed in first import from xml.parsers import expat but apparently it doesn't when it tries to pyexpat version that does not exist.
So, the question is, has anyone else stumbled into same problem and/or does anyone have a solution for this?
Some addtional (next day) observations:
Cloning and checking out a tag, building from the git repository does not really work as you would expect (no pun intentended).
% git clone https://github.com/brython-dev/brython.git brython.git
% cd brython.git/scripts
brython.git/scripts% python3 ./make_dist.py
/usr/bin
Traceback (most recent call last):
File "./make_dist.py", line 207, in <module>
run()
File "./make_dist.py", line 88, in run
import make_stdlib_list
File "brython.git/scripts/make_stdlib_list.py", line 53, in <module>
with open(os.path.join(static_doc_folder,lang,'stdlib.html'), 'w', encoding="utf-8") as out:
FileNotFoundError: [Errno 2] No such file or directory: 'brython.git/www/static_doc/en/stdlib.html'
brython.git%
That is caused by missing directories:
brython.git/scripts% mkdir -p ../www/static_doc/{en,es,fr}
brython.git/scripts% python3 ./make_dist.py
Very last build lines were:
adding xml.etree
adding xml.etree.cElementTree
adding xml.parsers
adding xml.parsers.expat
adding xml.sax
So maybe those are included.
Once targets are created, they (apparently, not really sure) appear into brython.git/setup/data directory, release zip files and naked .js files for live website. So I link into that directory in my Apache httpd webroot. But that building did not solve the traceback problem.
As side note, for an old OpenSource fart as myself, I feel very alien in this source tree, this project is done in Mouse camp (Microsoft Windows) and even one rare Makefile I managed to find, does not work with GNU Make because conflicting use of whitespace. Let alone there would be regular INSTALL, README, setup, Makefile etc files with expected content. I'm literally reading sources and guessing how all this is supposed to work. But I guess that only tells that Python is truely a cross platform language.
As being an "Open Source project", it's funny that its discussion is not for everyone:
Your application to join the Google group brython has been refused
The answer from the Issue is still relevant. Expat is a C-library which can't be used. You'll need to make do with window.DOMParser.
The preferred way to use brython is not via cloning the repo, but using pip:
$ pip install brython
and then
$ python3 -m brython install
in the directory where you are developing your app. This will copy all the necessary javascript files and create an example app from which you can start.
Note that this is all described in the "nonexistent" README.md file.
Well, digging deeper and it appears that Lib/xml/parses/expat.py contains:
"""Interface to the Expat non-validating XML parser."""
import sys
from pyexpat import *
# provide pyexpat submodules as xml.parsers.expat submodules
sys.modules['xml.parsers.expat.model'] = model
sys.modules['xml.parsers.expat.errors'] = errors
I tried to comment that pyexpat part and rebuild the pkgs, the traceback is now different. So there is no expat, no pyexpat and no ElementTree then. No XML in Ajax responses then.
When issuing the following command on jboss-cli.bat (an MS-DOS batch file for windows) to create a security realm on WildFly 8.1.0 final as mentioned in this migration guide,
./subsystem=security/security-domain=app:add(cache-type="default")
cd ./subsystem=security/security-domain=app
./authentication=classic:add(
login-modules=[ {
code="Database",
flag="required",
module-options={
dsJndiName="java:/jdbc/project_datasource",
principalsQuery="SELECT password FROM user_role_table WHERE user_id=?",
rolesQuery="SELECT group_id, 'Roles'
FROM group_table gt INNER JOIN user_role_table urt ON gt.user_group_id = urt.user_id
WHERE urt.user_id=?", hashAlgorithm="SHA-256",
hashEncoding="BASE64",
unauthenticatedIdentity="guest"
}
}, {
code="RoleMapping",
flag="required",
module-options={
rolesProperties="file:${jboss.server.config.dir}/app.properties",
replaceRole="false"
}
}
])
I get the following error on the cli prompt :
Node path format is wrong around 'cd.' (index 67)
If cd is removed, then the following error is reported.
Failed to perform read-opration-description to validate the request:
java.util.concurrent.ExecutionException: Operation failed
The command is given in a contiguous text format as follows.
./subsystem=security/security-domain=app:add(cache-type="default") cd ./subsystem=security/security-domain=app ./authentication=classic:add(login-modules=[ {code="Database",flag="required",module-options={dsJndiName="java:/jdbc/project_datasource",principalsQuery="SELECT password FROM user_role_table WHERE user_id=?",rolesQuery="SELECT group_id, 'Roles' FROM group_table gt INNER JOIN user_role_table urt ON gt.user_group_id = urt.user_id WHERE urt.user_id=?",hashAlgorithm="SHA-256",hashEncoding="BASE64",unauthenticatedIdentity="guest"}},{code="RoleMapping",flag="required",module-options={rolesProperties="file:${jboss.server.config.dir} /app.properties",replaceRole="false"}}])
What is the fix? I just do not want to copy/past the XML to the configuration file as it might be different from version to version.
The problem is with combining more commands on a single line.
The simple solution for you is to use external file to store the CLI commands.
E.g. create security-domain.cli file in wildfly-8.1.0.Final/bin folder with following content (if you want to split a command to more lines, put backslash as a last character):
/subsystem=security/security-domain=app:add(cache-type="default")
/subsystem=security/security-domain=app/authentication=classic:add()
/subsystem=security/security-domain=app/authentication=classic/login-module=Database:add( \
code="Database", \
flag="required", \
module-options=[ \
("dsJndiName"=>"java:/jdbc/project_datasource"), \
("principalsQuery"=>"SELECT password FROM user_role_table WHERE user_id=?"), \
("rolesQuery"=>"SELECT group_id, 'Roles' FROM group_table gt INNER JOIN user_role_table urt ON gt.user_group_id = urt.user_id WHERE urt.user_id=?"), \
("hashAlgorithm"=>"SHA-256"), \
("hashEncoding"=>"BASE64"), \
("unauthenticatedIdentity"=>"guest") \
])
/subsystem=security/security-domain=app/authentication=classic/login-module=RoleMapping:add( \
code="RoleMapping", \
flag="required", \
module-options=[ \
("rolesProperties"=>"file:${jboss.server.config.dir}/app.properties"), \
("replaceRole"=>"false") \
])
(Your sample contains old style of setting login modules. It's a deprecated way now, so the example usage the new way.)
Run the new file with JBoss CLI tool:
jboss-cli.bat -c --file=security-domain.cli