Below is my GC log output
<mem-info id="93" free="2404193232" total="2518572288" percent="85">
<mem type="nursery" free="0" total="412653184" percent="0" />
<mem type="tenure" free="2504193232" total="2515919104" percent="99">
<mem type="soa" free="2083398096" total="2195123968" percent="99" />
<mem type="loa" free="130795136" total="130795136" percent="100" />
</mem>
Is mem type="nursery" free="0" .... concerning ?
If if "nursery" value is concerning, then what JVM argument should i put in to overcome this concern.
Also,
i see copy-failed:
<copy-failed type="nursery" objects="133" bytes="3424" />
<copy-failed type="tenure" objects="669590" bytes="15098200" />
This is a Webspehere JVM.
Many Thanks..
Is mem type="nursery" free="0" .... concerning ? yes if its after scavenge gc. you may tune nursery for better performance
-Xmn can be used to increase nursery size
Related
First of all, thank you for the seniors who tried to help me.
I've a large XML file (below is an example with 2 parents).
Each parent(U7418、U2501) might have subchild、subsubchild...etc like parent U2501 below.
<CircuitRoot Title="projectname" Creator="Lee" Date="2020-08-27">
<Profile Platform="SeriesName" />
<U7418 PartNumber="084.21321" Footprint="PPAK-8P" Value="AONR21321" Description="FET MOS AONR21321 PC DFN 3X3 EP 8P">
<U7418 PartNumber="084.21321" Footprint="PPAK-8P" Ball="4" Pin="G" Page="74" Net="LPS_SW">
<R7403 PartNumber="ZZ.R0402" Info="0R-PAD" Net="GND">
<Q7408 PartNumber="084.03413" Pin="D" />
</R7403>
<R7402 PartNumber="64.20035" Net="LPS_SW_A">
<Q7406 PartNumber="075.00138" Pin="D2" />
</R7402>
<C7401 PartNumber="78.15224" Power="+SDC_IN" />
</U7418>
<U7418 PartNumber="084.21321" Footprint="PPAK" Ball="3" Pin="S#3" Page="74" Power="+SDC_IN" />
</U7418>
<U2501 PartNumber="072.25Q64" Footprint="SOIC8" Value="W25Q64JVSSIQ" Description="IC FLASH">
<U2501 PartNumber="072.25Q64" Footprint="SOIC8" Ball="1" Pin="CS#" Page="25" Net="SPI_CS_ROM_N0">
<R2501 PartNumber="63.47234" Power="3D3V_SPI" />
<R2403 PartNumber="ZZ.R0402" Net="SHD_CS0#">
<U2401 PartNumber="071.01515" Pin="GPIO055/PWM2/SHD_CS0#/BSS_STRAP" />
<R2405 PartNumber="63.R0034" Remark="DY#SKU1" Net="PCH_RSMRST#">
<R2404 PartNumber="ZZ.R0402" Net="PCH_RSMRST#_R">
<U2401 PartNumber="071.01515" Pin="GPIO114/PS2_CLK0A/EC_SCI#" />
</R2404>
<TP2401 PartNumber="ZZ.PAD14" Pin="1" />
<R1709 PartNumber="63.10234" Net="PM_RSMRST#">
<Q1701 PartNumber="75.27002" Pin="3" />
<CPU1 PartNumber="ZZ.00CPU" Pin="RSMRST#" />
<R6414 PartNumber="64.33035L" Net="M_BIST">
<Q6408 PartNumber="84.T3904" Pin="B" />
<U2401 PartNumber="071.01515" Pin="GPIO246/CMP_VREF0" />
<D6403 PartNumber="83.R2003" Net="GND">
<PG4408 PartNumber="ZZ.CLOSE" Net="PWR_CHG_ACOK">
<PU4401 PartNumber="074.09538" Pin="ACOK" />
<PR4426 PartNumber="64.19635" Power="GND" />
</PG4408>
<U2401 PartNumber="071.01515" Pin="VCI_OVRD_IN/GPIO172" />
</D6403>
<C6405 PartNumber="78.10523" Power="GND" />
</R6414>
</R1709>
</R2405>
</R2403>
<R1832 PartNumber="ZZ.R0402" Net="SPI">
<CPU1 PartNumber="ZZ.00CPU" Pin="SPI0_CS0#" />
</R1832>
</U2501>
<U2501 PartNumber="072.25Q64" Footprint="SOIC8" Power="3D3V_SPI" />
</U2501>
</CircuitRoot>
My goal is check whether has some specify child.tag(ex:R7403、PR4426) under parents(ex: U7418、U2501) ,also check whether Power or Net in specify child.attrib is GND.
So, I want to extract each parent's(U7418、U2501) child.tag and child.attirb
expect output is
{
U7418:{ R7402:[{PartNumber="64.20035" Net="LPS_SW_A"}], Q7406:[{PartNumber="075.00138" Pin="D2"}]... },
U2501:{ R2403:[{PartNumber="ZZ.R0402" Net="SHD_CS0#"}], PR4426:[{PartNumber="64.19635" Power="GND"}]...},
...
}
but i think this expect output can be changed, since here might have better way to achieve my goal, the key point for me is how to paring this file.
I have tryed
import xml.etree.ElementTree as ET
tree = ET.parse(xml)
parent_map = {}
for p in tree.iter():
for c in p:
if c in parent_map:
parent_map[c.tag].append(c.attrib)
else:
parent_map[c.tag] = [c.attrib]
print(parent_map)
can get all child、subshild...tag and attrib, but i can't get the corresponding parent.
[1]: https://i.stack.imgur.com/hpAmA.png
Thank you for your patience in reading to the end,
All help is greatly appreciated!
I have jdbc:inbound-channel-adapter polling 50 records.
I am trying to improve the performance by scaling pollerExecutor pool-size to 1-10, so that multiple thread can process 50 records each:
<int-jdbc:inbound-channel-adapter
id="initial.ContactType.poller"
query="${poller.ContactType.get}"
max-rows="${poller.deliveryContactType.maxRow:50}"
row-mapper="ContactTypePollerRowMapper"
data-source="dataSource" channel="ContactTypeChannel">
<int:poller fixed-rate="3000" time-unit="MILLISECONDS" task-executor="pollerExecutor">
<int:advice-chain>
<ref bean="pollerLoggingAdvice"/>
<ref bean="txAdvice" />
</int:advice-chain>
</int:poller>
</int-jdbc:inbound-channel-adapter>
<task:executor id="pollerExecutor" pool-size="1-10"
queue-capacity="0" rejection-policy="CALLER_RUNS" />
I tested the time taken to process 100,000 records is same irrespective of the pool-size.
I did three rounds of tests with pool-size=1, pool-size=1-3 and pool-size=1-10 respectively,
in all three tests 100,000 records took 1 hr each time.
I confirmed by checking the logs that pollerExecutor threads are not working parallely.
pollerExecutor-1 process all 50 records before pollerExecutor-2 starts processing.
Why container/pollerExecutor is not working parallely?
I think your problem is here:
/**
* Set the capacity for the ThreadPoolExecutor's BlockingQueue.
* Default is {#code Integer.MAX_VALUE}.
* <p>Any positive value will lead to a LinkedBlockingQueue instance;
* any other value will lead to a SynchronousQueue instance.
* #see java.util.concurrent.LinkedBlockingQueue
* #see java.util.concurrent.SynchronousQueue
*/
public void setQueueCapacity(int queueCapacity) {
So, if you specify queue-capacity="0", then you end up with a SynchronousQueue, which cannot accept new parallel tasks because there is already one busy for processing those 50 records.
Try to have some reasonable queue-capacity to observe possible parallelism.
Currently I am working with JDBC Spring Integration (to be very specific: <int-jdbc:stored-proc-outbound-gateway>) where the scenario is passing n-number of parameters to a stored procedure (ORACLE) and receive one return variable (with '1' or Error Message) along with n-number of CURSORS (sys_refcursor) which may have n-number of columns.
While working on this above scenario with Spring Framework I noticed the following issue(s):
My logic from front-end was, first read the 'return variable'. If that value is 1 then start reading all the CURSORS data or else through exception by redirecting user to the error page. Now, all my CURSORS were not having same number of columns as well as same length/type of data. And in middle layer I was having only one Model class to deal with all the CURSORS return elements. That was the challenge for me!
Previously I dealt with only one CURSOR. and therefore in middle layer I used Mapper class to map all CURSOR elements to my Model element's (getter and setter) to push data to the front. But then when used same concept to deal with n-number of CURSORS -that was truly a nightmare.
let me share what I did before and what I did later to resolve this situation at my end and then will draw my understanding for which will sick EXPARTS opinion at the later part of this post.
For one return variable (with '1' or Error Message) along with one CURSOR:
I had my Gateway definition as follows:
<!-- Stored Procedure Outbound-Gateway = To call a database stored procedure -->
<int-jdbc:stored-proc-outbound-gateway id="outbound-gateway-storedproc-personalinfo"
request-channel="procedureRequestChannel"
data-source="dataSource"
stored-procedure-name="pkg_personalinfo_spring.proc_personalinfo_spring"
expect-single-result="false"
ignore-column-meta-data="true">
<!-- Parameter Definitions -->
<int-jdbc:sql-parameter-definition name="firstname" direction="IN"/>
<int-jdbc:sql-parameter-definition name="lastname" direction="IN"/>
<int-jdbc:sql-parameter-definition name="p_RetVal" direction="OUT"/>
<int-jdbc:sql-parameter-definition name="get_ResultSet" type="#{T(oracle.jdbc.OracleTypes).CURSOR}" direction="OUT"/>
<!-- Parameter Mappings Before Passing & Receiving -->
<int-jdbc:parameter name="firstname" expression="payload[0]"/>
<int-jdbc:parameter name="lastname" expression="payload[1]"/>
<int-jdbc:returning-resultset name="get_ResultSet" row-mapper="com.support.PersonalinfoMapper"/>
</int-jdbc:stored-proc-outbound-gateway>
And in Mapper class I had the following simple Mapping rule:
...
PersonalInfo personalInfo = new PersonalInfo();
try{
personalInfo.setFirstname(resultSet.getString(DBConstants.FIRSTNAME));
personalInfo.setLastname(resultSet.getString(DBConstants.LASTNAME));
...
I did that because in my Model class i was having getter and setter for all CURSOR return elements.
Now, for one return variable (with '1' or Error Message) along with n-number of CURSORS:
I changed my Gateway definition as follows:
<!-- Stored Procedure Outbound-Gateway = To call a database stored procedure -->
<int-jdbc:stored-proc-outbound-gateway id="outbound-gateway-storedproc-personalinfo"
request-channel="procedureRequestChannel"
data-source="dataSource"
stored-procedure-name="pkg_personalinfo_spring.proc_personalinfo_spring"
expect-single-result="false"
ignore-column-meta-data="true">
<!-- Parameter Definitions -->
<int-jdbc:sql-parameter-definition name="firstname" direction="IN"/>
<int-jdbc:sql-parameter-definition name="lastname" direction="IN"/>
<int-jdbc:sql-parameter-definition name="p_RetVal" direction="OUT"/>
<int-jdbc:sql-parameter-definition name="get_curr_1" type="#{T(oracle.jdbc.OracleTypes).CURSOR}" direction="OUT"/>
<int-jdbc:sql-parameter-definition name="get_curr_2" type="#{T(oracle.jdbc.OracleTypes).CURSOR}" direction="OUT"/>
<!-- Parameter Mappings Before Passing & Receiving -->
<int-jdbc:parameter name="firstname" expression="payload[0]"/>
<int-jdbc:parameter name="lastname" expression="payload[1]"/>
</int-jdbc:stored-proc-outbound-gateway>
And secondly I removed the entire row mapping concept, means I didn't have any Mapper class. The only one thing I had was only one Model class with all the CURSORS element name and their getters and setters.
Also notice that in my 'Parameter Definitions', how added those two CURSORS definition but in 'Parameter Mappings Before Passing & Receiving' I have nothing now.
I ran the App without any exceptions and later through Fiddler observed the following JSON data came with RESPOND, means everything PERFECT! ;)
JSON
- {}
- get_curr_1
- -{}
- - firstname=Faisal
- - lastname=Quazi
- get_curr_2
- -{}
- - country=Bangladesh
- - capital=Dhaka
WHAT THE HACK??? Yes...I had the same feelings ;)
Now, can any one plz help to understand what is going on here. Will it be a true statement that "'int-jdbc:stored-proc-outbound-gateway' is able to MAP ur Model element(s) by itself magically???"
Is there any best practice to deal with such kind of situation(s)?
Thank u guys always :)
:-). I guess that by default Spring JDBC uses ColumnMapRowMapper.
So you end up after that gateway with Map<String, <Map<?, ?>>> or something similar. But it is Map anyway.
Since you further convert your payload to JSON and Jackson can get deal with maps prefectly, you finish with correct result.
I can debug it, of course, but deduction says that it can't be differently.
UPDATE
As I said. The answer is here JdbcTemplate#extractOutputParameters:
....
if (outParam.isResultSetSupported()) {
returnedResults.putAll(processResultSet((ResultSet) out, outParam));
}
else {
String rsName = outParam.getName();
SqlReturnResultSet rsParam = new SqlReturnResultSet(rsName, new ColumnMapRowMapper());
returnedResults.putAll(processResultSet((ResultSet) out, rsParam));
if (logger.isDebugEnabled()) {
logger.debug("Added default SqlReturnResultSet parameter named '" + rsName + "'");
}
}
....
i'm using wso2esb 4.7.0 and wso2dss 3.1.0.i wish to select some column from table.For that i have used following query in dss :
select roleid,rolename,rolecode from mrole where clientid = ? and ModifiedOn = ?
It's working fine in dss as well as esb also.Output is like :
{"roleid":"1077282521908571","rolename":"adminrole","rolecode":"adminrole"},{"roleid":"1077282521908571","rolename":"adminrole","rolecode":"adminrole"},{"roleid":"1077278842127704","rolename":"everyonerole","rolecode":"everyonerole"},{"roleid":"1077282521908571","rolename":"adminrole","rolecode":"adminrole"}
this is one case.Now in second case if i didn't got modifiedon then obvious it shows error in dss like :
Caused by: javax.xml.stream.XMLStreamException: DS Fault Message: Error in 'SQLQuery.processNormalQuery'
DS Code: DATABASE_ERROR
Source Data Service:-
Name: muser_DataService2.0
Location: /muser_DataService2.0.dbs
Description: N/A
Default Namespace: http://ws.wso2.org/dataservice
Current Request Name: Select_Role_Details_Op
Current Params: {clientid=214057357158656, modifiedon=}
Nested Exception:-
java.lang.NumberFormatException: For input string: ""
To solve that i'm using switch case in esb like :
<switch source="get-property('ModifiedOn')">
<case regex="''">
<property name="ModifiedOn" value="0" scope="default" type="STRING"/>
<log level="custom">
<property name="ModifiedOn in case of null: "
expression="get-property('ModifiedOn')"/>
</log>
</case>
</switch>
it shows warning message in esb like :
[2014-10-08 11:43:54,100] WARN - SwitchCase Provided character sequence for switch case condition is 'null'. Switch case will not be executed.
and without executing switch case it directly move forward and generate fault as well as dss shows error as above.How should i give the null condition in switch case?
use boolean xpath function to verify if your property exist :
<filter xpath="boolean(get-property('ModifiedOn'))">
<then>
...
</then>
<else>
...
</else>
</filter>
Hi Markloggers out there,
I bump into a strange situation.
My experience so far is that I need to set a specific index for cts: functions to perform well. Now I need to search geotagged tweets in de db I use the following query.
Seems I do not need teo make specific indexes for cts:element-child-geospatial-query() to run, but performance is poor... what am I missing here?
Query
xquery version "1.0-ml";
declare namespace j = "http://marklogic.com/xdmp/json/basic";
let $lon := 5.470047
let $lat := 51.819565
let $radius := 5.0
let $point := cts:point($lon, $lat)
let $circle := cts:circle($radius,$point)
(:
let $pattern := "/twitter/*"
let $pointSource := cts:uri-match($pattern)
:)
(: give back all tweet ids in the circle :)
let $codes := for $p in cts:search(fn:doc(), cts:element-child-geospatial-query(xs:QName("j:geo"), xs:QName("j:point"), $circle))
return $p//j:id
return fn:count($codes)
One million tweets that have to following elements in them:
<json type="object" xmlns="http://marklogic.com/xdmp/json/basic">
... other stuff ...
<dikwmetadata type="object">
<source type="string">
twitter_nederland
</source>
<timestamp type="string">
2014-03-11T21:19:03.818547
</timestamp>
<data xmlns:j="http://marklogic.com/xdmp/json/basic">
<point_was_tried>
2014-03-11T21:19:03.835457+01:00
</point_was_tried>
<postalcode_was_tried>
2014-03-11T21:19:03.835457+01:00
</postalcode_was_tried>
<geo lastupdate="">
<point>
4.65407742 , 52.28828829
</point>
</geo>
<tags lastupdate=""/>
<postalcodes lastupdate="">
<postal_code>
2135
</postal_code>
</postalcodes>
</data>
</dikwmetadata>
</json>
we added element range index type = string to geo and point elements but no improvement...
Response now 60 seconds...
Any tips?
hugo
Geospatial indexes are separate from range indexes. You can create them in Admin > Databases > DB-Name > Geospatial Indexes.
In addition to running the query unfiltered, if you simply want to count the number of matching fragments, then you can use xdmp:estimate instead of count:
xdmp:estimate(cts:search(...))
You might consider doing the cts:search unfiltered.
cts:search(fn:doc(), cts:element-child-geospatial-query(xs:QName("j:geo"), xs:QName("j:point"), $circle),("unfiltered"))
That would go by the indexes only instead of actually verifying each hit.