Ack. Using OpenLayers, I have an Ext.form.ComboBox that runs a query of a Ext.data.JsonStore to access a shapefile of ~1 Gb and return a table of results. Then, a user may select one of the results and the map zooms to that location.
This works excellent. BUT! As the ComboBox is used, the JVM size continues to grow- a memory leak?- and Geoserver throws many issues mostly all related to:
Caused by: java.io.IOException: Map failed
Caused by: java.lang.OutOfMemoryError: Map failed
ERROR [geotools.map] - Call MapContent dispose() to prevent memory leaks
I can run the ComboBox as many as five times within about a minute before a storm of errors overwhelm Geoserver (e.g. GetFeatureInfo doesn't return any results, the ComboBox query comes back empty, pink tiles appear, EDIT: and GWC stops making new tiles!).
Do I need to destroy () or clear the store or...? and how? Thoughts??
GeoExt 1.1, OpenLayers 2.12.rc6, Ext 3.4, Ext 4.2.1.883, jquery-1.6, Geoserver 2.5, JDK i586 v7, Tomcat 7
code:
THE QUERY: looks up what is typed in the ComboBox & populates the Store
Ext.onReady(function() {
Ext.override(Ext.form.ComboBox, {
doQuery: function(q, forceAll) {
console.log('queryTpl', this.queryTpl, q);
q = Ext.isEmpty(q) ? '' : q;
var qe = {
query: q,
forceAll: forceAll,
combo: this,
cancel: false
};
if (this.fireEvent('beforequery', qe) === false || qe.cancel) {
return false;
}
q = qe.query;
forceAll = qe.forceAll;
if (forceAll === true || (q.length >= this.minChars)) {
if (this.lastQuery !== q) {
this.lastQuery = q;
if (this.mode == 'local') {
this.selectedIndex = -1;
if (forceAll) {
this.store.clearFilter();
} else {
this.store.filter(this.displayField, q);
}
this.onLoad();
} else {
this.store.baseParams[this.queryParam] = this.queryTpl ? String.format(this.queryTpl, q) : q;
this.store.load({
params: this.getParams(q)
});
this.expand();
}
} else {
this.selectedIndex = -1;
this.onLoad();
}
}
},
});
})
THE STORE
var dsm = new Ext.data.JsonStore({
remoteFilter: true,
autoLoad: false,
autoDestroy: true, // doesn't appear to help
url: '../geoserver/workspace', // generalized, not for public access
storeId: 'myStore2',
baseParams: {
service: 'WFS',
version: '1.1.0',
request: 'GetFeature',
typeName: 'MRDS_LUT', // *shp in geoserver defined by var
srsName: 'EPSG:4326',
outputFormat: 'json',
propertyName: 'SiteName,Municipal,Commodity,Status,Longitude,Latitude'
},
root: 'features',
fields: [{
name: 'SiteName',
mapping: 'properties.SiteName'
}, {
name: 'Municipal',
mapping: 'properties.Municipal'
}, {
name: 'Commodity',
mapping: 'properties.Commodity'
}, {
name: 'Status',
mapping: 'properties.Status'
}, {
name: 'Longitude',
mapping: 'properties.Longitude'
}, {
name: 'Latitude',
mapping: 'properties.Latitude'
},{
name: 'bbox',
mapping: 'properties.bbox'
}],
sortInfo: {
field: "Municipal",
direction: "ASC"
}
});
The ComboBox
var panelSiteNameSearch = new Ext.form.ComboBox({
store: dsm,
fieldLabel: "<b>Search</b>",
queryParam: "cql_filter",
queryTpl: "SiteName Like '{0}%'",
minChars: 5,
displayField: "SiteName",
valueField: "SiteName",
typeAhead: false,
loadingText: "Searching...",
width: 230,
emptyText: "Mine/Mineral Site Search",
hideTrigger: true,
tpl: '<tpl for="."><div class="search-item"><b>Site Name:</b> {SiteName}<br><b>Commodity: </b> {Commodity}<br><b>Municipality:</b> {Municipal}<br><b>Status:</b> {Status}</div><hr/></tpl>',
itemSelector: "div.search-item",
listeners: {
"select": function (combo, record) {
mrdsprimary.setVisibility(true);
mrdssecondary.setVisibility(true);
mrdstertiary.setVisibility(true);
map.setCenter(new OpenLayers.LonLat(record.data.Longitude,record.data.Latitude),13);
},
"focus": function () {
keyboardnav.deactivate();
},
"blur": function () {
keyboardnav.activate();
}
}
});
This solution is a bit nebulous. I finally prevented the out of memory errors by making sure the combined total size of all combo box stores, plus the initial Java -Xms allotment, does not exceed the Java -Xmx allotment.
For example, setting -Xms to 500m, and knowing that store #1 accesses a database of 600m and store #2 accesses a database of 800m, gives a combined total of 1900mb in my Java environment heap. If the -Xmx is set to 2g, then no errors are thrown.
In theory. I can't directly affect the Java environment's Xms without interfering (poorly) with Tomcat/Java's native GC. Knowing there is total allotted -Xmx cap, and what counts directly toward it, encourages extremely trim data stores.
Unfortunately I am limited to a 32bit JDK i586, which I read somewhere that carries an inherent 2gb limitation for -Xmx (which seems about right). The 64bit is reportedly much higher.
Findings: a remote store 'loads' an entire data store, and submits the filtered results locally... clearing the either store with removeAll or clearFilter or setValue() or NULL and/or SYNC only works locally and does not actually trigger any remote GC or reduction in the remote Java's heaping pile of unnecessary storage.
Long-term Solutions: Create a server-side PHP script that passes a given cql or some other query or ComboBox value directly to a SQL or PostGIS database, and leave javascript out of it. But I don't know how to write one yet.
Related
i'm working on a Vue 3 application where data comes from a weather station (NodeJS backend). A sample of a parameter from a "module" (module is my object which holds data from different sensors, but all from one location):
{
data: "temperatur",
factor: "10",
icon: null,
key: "TemperaturOutside01",
label: "Air Temperature",
parameter: "Temperature",
type: "rw",
unit: "°C",
value: 18.1
}
The "key" is my uniqe identifier in the whole object. It can also be that there are some childrens to the entry like:
{
...
children: [
{...}
]
}
and those children can also have childrens and so on. I'm working with a recurisve function which search for the specific parameter in the object an returns the node when found.
The recursive function:
function findByParameter(tree, parameter) {
for (let node of tree) {
if (node.parameter === parameter) {
return node;
}
if (node.children) {
let desiredNode = findByParameter(node.children, parameter);
if (desiredNode) {
return desiredNode;
}
}
}
return false;
}
All of that is working, and i'm getting the values in my Vue template like that:
<InputNumber id="temp01" ref="temp01" v-model="returnParameter(theObj, 'parameter', Temperature').value" :suffix="' ' + returnParameter(theObj, 'parameter', 'Temperature').unit" />
and the result for my example would be
18.1 °C
(the returnParameter() is just a method in Vue, returning the node from the findByParameter function)
This is a real time application with Socket.io for data exchange and value updates 1x/sec. if needed.
But now for the question: if i have a few hundred sensors and most of them change every second the values, would that be a problem for the client performance? The for ... of loop should be one of the fastest ways i heard here but maybe some of you have other ways...
Thank you!
I found an important security fault in my meteor app regarding subscriptions (maybe methods are also affected by this).
Even though I use the check package and check() assuring that the correct parameters data types are received inside the publication, I have realised that if a user maliciously subscribes to that subscription with wrong parameter data types it is affecting all other users that are using the same subscription because the meteor server is not running the publication while the malicious user is using incorrect parameters.
How can I prevent this?
Packages used:
aldeed:collection2-core#2.0.1
audit-argument-checks#1.0.7
mdg:validated-method
and npm
import { check, Match } from 'meteor/check';
Server side:
Meteor.publish('postersPub', function postersPub(params) {
check(params, {
size: String,
section: String,
});
return Posters.find({
section: params.section,
size: params.size === 'large' ? 'large' : 'small',
}, {
// fields: { ... }
// sort: { ... }
});
});
Client side:
// in the template:
Meteor.subscribe('postersPub', { size: 'large', section: 'movies' });
// Malicious user in the browser console:
Meteor.subscribe('postersPub', { size: undefined, section: '' });
Problem: The malicious user subscription is preventing all other users of getting answer from their postersPub subscriptions.
Extra note: I've also tried wrapping the check block AND the whole publication with a try catch and it doesn't change the effect. The error disappears from the server console, but the other users keep being affected and not getting data from the subscription that the malicious user is affecting.
Check method and empty strings
There is one thing to know about check and strings which is, that it accepts empty strings like '' which you basically showed in your malicious example.
Without guarantee to solve your publication issue I can at least suggest you to modify your check code and include a check for non-empty Strings.
A possible approach could be:
import { check, Match } from 'meteor/check';
const nonEmptyString = Match.Where(str => typeof str === 'string' && str.length > 0);
which then can be used in check like so:
check(params, {
size: nonEmptyString,
section: nonEmptyString,
});
Even more strict checks
You may be even stricter with accepted parameters and reduce them to a subset of valid entries. For example:
const sizes = ['large', 'small'];
const nonEmptyString = str => typeof str === 'string' && str.length > 0;
const validSize = str => nonEmptyString(str) && sizes.indexOf( str) > -1;
check(params, {
size: Match.Where(validSize),
section: Match.Where(nonEmptyString),
});
Note, that this also helps you to avoid query logic based on the parameter. You can change the following code
const posters = Posters.find({
section: params.section,
size: params.size === 'large' ? 'large' : 'small',
}, {
// fields: { ... }
// sort: { ... }
});
to
const posters = Posters.find({
section: params.section,
size: params.size,
}, {
// fields: { ... }
// sort: { ... }
});
because the method does anyway accept only one of large or small as parameters.
Fallback on undefined cursors in publications
Another pattern that can support you preventing publication errors is to call this.ready() if the collection returned no cursor (for whatever reason, better is to write good tests to prevent you from these cases).
const posters = Posters.find({
section: params.section,
size: params.size === 'large' ? 'large' : 'small',
}, {
// fields: { ... }
// sort: { ... }
});
// if we have a cursor with count
if (posters && posters.count && posters.count() >= 0)
return posters;
// else signal the subscription
// that we are ready
this.ready();
Combined code example
Applying all of the above mentioned pattern would make your function look like this:
import { check, Match } from 'meteor/check';
const sizes = ['large', 'small'];
const nonEmptyString = str => typeof str === 'string' && str.length > 0;
const validSize = str => nonEmptyString(str) && sizes.indexOf( str) > -1;
Meteor.publish('postersPub', function postersPub(params) {
check(params, {
size: Match.Where(validSize),
section: Match.Where(nonEmptyString),
});
const posters = Posters.find({
section: params.section,
size: params.size,
}, {
// fields: { ... }
// sort: { ... }
});
// if we have a cursor with count
if (posters && posters.count && posters.count() >= 0)
return posters;
// else signal the subscription
// that we are ready
this.ready();
});
Summary
I for myself found that with good check matches and this.ready() the problems with publications have been reduced to a minimum in my applications.
When I load my page, I fetch a list of optboxes items.
Sources
Project's sources are online:
optboxes page ;
store (store, actions, mutations, getters).
optboxes pages
The HTTP request is well send and return adequate data:
created(){
this.getOptboxes();
},
components: {
'optbox': OptboxComponent,
},
methods: {
getOptboxes() {
optboxes.all().then((response) => {
this.setOptboxes(response.data.output);
}).catch(() = > {
this.no_optbox_message = 'there is no optbox';
logging.error(this.$t('optboxes.get.failed'))
});
}
},
vuex: {
actions: { setOptboxes: actions.setOptboxes},
getters: { optboxesList: getters.retrieveOptboxes}
}
I'm iterating over the results as follow:
<div v-for="optbox in optboxesList" class="panel panel-default">
<optbox :optbox="optbox"></optbox>
</div>
Store
const state = {
optboxes: {
/*
'akema': {
hostname: "192.168.2.23",
id: "akema",
printers: [
{
description: "bureau",
destination_port: 9100,
forward: "normal",
hostname: "1.2.3.4",
id: 0,
listening_port: 9102
}
]
}
*/
}
};
Question
If I switch to another pages and come back then the list appear. I also notice that with the Vuex extension I can commit the state and see the changes.
Why are my changes not applied automatically?
I had to change my data structure due to Change Detection Caveats.
Due to limitations of JavaScript, Vue.js cannot detect the following
changes to an Array:
When you directly set an item with the index, e.g. vm.items[0] = {};
When you modify the length of the Array, e.g. vm.items.length = 0.
Store
optboxes is now an array.
const state = {
optboxes:[]
}
Then update my mutations accordingly to edit the array.
Maybe it's an issue of reactivity?! I assume your setOptboxes mutation is not being picked up by vue:
setOptboxes(state, optboxes) {
for (var optbox of optboxes) {
state.optboxes[optbox.id] = optbox;
}
}
You can read about it here:
https://vuejs.org/guide/list.html#Caveats
https://vuejs.org/guide/reactivity.html
The docs solution is to use:
state.optboxes.$set(optbox.id, optbox);
which will trigger a view update.
Using Vue.set dit it for me. Take a look here
I am trying to use Leaflet realtime plugin (https://github.com/perliedman/leaflet-realtime), they mentioned in the documentation that we can keep previous updates by adding start:false in the constructor.
var map = L.map('map'),
realtime = L.realtime({
url: 'https://wanderdrone.appspot.com/',
crossOrigin: true,
type: 'json'
}, {
interval: 3 * 1000, start:false
}).addTo(map);
Anyone have better idea on how to do that ?
plnkr has a good demo:
http://plnkr.co/edit/NmtcUa?p=preview
If you set start: false, automatic updates will be disabled. This means you will have to call the layer's update method yourself, providing any GeoJSON data you want to add or update; you can also remove added features with the remove method. These methods can be used if you want to use something other than server polling.
You can use the following code that I copied from plnkr while changed a little bit, because L.realtime inherit L.geoJson. And L.geoJson have 'layeradd'.
realtime.on('layeradd', function(e) {
var coordPart = function(v, dirs) {
return dirs.charAt(v >= 0 ? 0 : 1) +
(Math.round(Math.abs(v) * 100) / 100).toString();
},
popupContent = function(fId) {
var feature = e.features[fId],
c = feature.geometry.coordinates;
return 'Wander drone at ' +
coordPart(c[1], 'NS') + ', ' + coordPart(c[0], 'EW');
},
bindFeaturePopup = function(fId) {
realtime.getLayer(fId).bindPopup(popupContent(fId));
},
updateFeaturePopup = function(fId) {
realtime.getLayer(fId).getPopup().setContent(popupContent(fId));
};
map.fitBounds(realtime.getBounds(), {maxZoom: 3});
Object.keys(e.enter).forEach(bindFeaturePopup);
Object.keys(e.update).forEach(updateFeaturePopup);
});
First this one works:
win.remove(formLogin, true);
win.add(changepswform);
win.doLayout();
Then this one does not wok:
win.add(changepswform);
win.remove(formLogin, true);
win.doLayout();
I get this error:
uncaught exception: [Exception... "Component returned failure code: 0x80004003 (NS_ERROR_INVALID_POINTER) [nsIDOMHTMLDivElement.insertBefore]" nsresult: "0x80004003 (NS_ERROR_INVALID_POINTER)" location: "JS frame :: http://app.localhost.com/ext-4.0.2/ext-all.js :: <TOP_LEVEL> :: line 15" data: no]
[Break On This Error] (function(){var e=this,a=Object.protot...ate("Ext.XTemplate",j,g)}return j}});
Whose fault is this? Mine? Or ExtJs has a bug?
Note:
var win = Ext.create('Ext.Window', {
title: 'Asistan Web: Online Teknik Servis Yazılımı',
width: 350,
minWidth: 350,
height: 240,
closable: false,
modal: true,
bodyStyle: 'padding:10px;',
items: [formLogin],
bbar: Ext.create('Ext.ux.StatusBar', {
id: 'win-statusbar',
defaultText: 'Giriş',
items: [
{
xtype:'splitbutton',
text:'Şifre işlemleri',
menuAlign: 'br-tr?',
menu: Ext.create('Ext.menu.Menu', {
items: [{text: LANG.LOGIN_FORGOT_PASS, handler :
function(){
if(confirm(LANG.LOGIN_MAIL_CONFIRM))doformTest(1);
Ext.getCmp('win-statusbar').setText('');
}}, {text: LANG.LOGIN_CHANGE_PASS, handler : function(){doformTest(2);}}]
})
}]
})
});
It looks like you're trying to remove a component in the second block that you've already removed in the first block. Or are you not running the two scripts sequentially?
Using true on .remove method you force garbage collection, thus removing the variable entirely AFAIK.
Edit: FOUND solution, How can I replace the content of a panel with new content?
STILL, if you want to re-use (let's say swap multiple times) your panels, you have to use false on .remove()