GEE: How to classify Sentinel-2 RED band into 5 classes, with class boundaries being dynamically derived from respective MIN-MAX range - decision-tree

I'm trying to develop a script that classifies the RED band for an AOI of a Sentinel-2 image into 5 equal classes, based on the AOI-specific spectral range of the RED band.
I managed to extract the MIN and MAX values forthe RED band of the user-defined AOI and, based on that, to define the respective class boundaries as variables. I also found several classifiers which can do the trick if the class boundaries are given as numbers, but they don't run if I give the variable for the class boundary, which would allow the classification to run automatically, once the user defines the AOI.
Is there a way to make, e.g. the ee.Algorithms.If function or the ee.Classifier.decisionTree to run with variables instead of hard-wired numbers?
Any suggestions will be appreciated.
Here the code, with a de-activated, fruitless attempt to the classification at the bottom.
// add Durban Image
var durbs = ee.Image("COPERNICUS/S2_SR/20211015T073901_20211015T080043_T36JUN");
// Draw polygon to define Area of Interest
var AOI = geometry3;
// Define the visual parameters
var visParams = {bands: ["B4","B3","B2"],
gamma: 1,
max: 1422,
min: -92,
opacity: 1};
// Set map centre
Map.centerObject(AOI, 8);
//Add images to check
Map.addLayer(durbs, visParams, 'Durban');
// Clipping of S2 RED band to AOI extent
var S2subset = durbs.clip(AOI).select('B4');
// Display clipped image in grey scale
var visParams2 = {bands: ["B4"],
gamma: 1,
max: 1422,
min: -92,
opacity: 1};
Map.addLayer(S2subset, visParams2, 'S2_B4_subset');
//extract MIN values
var min = S2subset.reduceRegion({
reducer: ee.Reducer.min(),
geometry: S2subset.geometry(),
scale: 10,
maxPixels: 1e10
});
print('band MIN', min);
print('MIN value in B4', min.get('B4'));
//extract MAX values
var max = S2subset.reduceRegion({
reducer: ee.Reducer.max(),
geometry: S2subset.geometry(),
scale: 10, // scale defines pixel size
maxPixels: 1e10
});
print('band MAX', max);
print('MAX value in B4', max.get('B4'));
// Result of reduceRegion is a dictionary.
// We can extract the values using .get() function
var max_B4 = max.get('B4'); // this is just a replication of line 48ff above
var min_B4 = min.get('B4'); // this is just a replication of lines 37ff above
// A dictionary of variables to use in expression.
var variables = {x: max_B4, y: min_B4};
// Create boundaries for 5 classes based on AOI-specific RED band range, derived from max_B4 and min_B4.
var range = ee.Number.expression('x - y', variables);
print('range', range);
var class_width = ee.Number.expression('(x - y)/5', variables);
print('class width', class_width);
var class1_min = ee.Number.expression('y', variables);
print('minimum', class1_min);
var class1_2break = ee.Number.expression('y+((x - y)/5)*1', variables);
print('1-2 break', class1_2break);
var class2_3break = ee.Number.expression('y+((x - y)/5)*2', variables);
print('2-3 break', class2_3break);
var class3_4break = ee.Number.expression('y+((x - y)/5)*3', variables);
print('3-4 break', class3_4break);
var class4_5break = ee.Number.expression('y+((x - y)/5)*4', variables);
print('4-5 break', class4_5break);
// SCRIPT RUNS FINE UNTIL HERE, BELOW ONE FRUITLESS ATTEMPT TO USE
// THE ABOVE BOUNDARIES FOR THE ACTUAL CLASSIFICATION: .... PLEASE HELP!
var class_1 = ee.Image(ee.Algorithms.If(S2subset<=class1_2break,
1, 0));
print('class1', class_1); // this was a test to see what happens
/*
var variables = {
a: class1_2break,
b: class2_3break,
c: class3_4break,
d: class4_5break
};
var DTstring = [
'1) root 9999 9999 9999',
'2) S2subset<={ee.Number.expression(a, variables)} 9999 9999 1 *',
'3) S2subset>class1_2break 9999 9999 9999', <----- I tried to use my variable from above here, unsuccessfully
'6) S2subset<=class2_3break 9999 9999 2 *',
'7) S2subset>0.4 9999 9999 9999', <---- the "0.4 etc. come from an example script
'14) S2subset<=0.6 9999 9999 3 *',
'15) S2subset>0.6 9999 9999 9999',
'30) S2subset<=0.8 9999 9999 4 *',
'31) S2subset>0.8 9999 9999 5 *'].join("\n");
var classifier = ee.Classifier.decisionTree(DTstring);
var reclassifiedImage = S2subset.select('nd').classify(classifier);
*/

Related

Why am I getting strange triplication of video using Webcam and Tensorflow.js?

I have a keras model trained and now I want to run this on the web. I thought this might be a good way to attempt testing out Tensorflow.js. I downloaded the Tesnroflow.js "Webcam-transfer-learning" tutorial and then modified it to get what I currently have. The working keras model performs emotion classification after reducing the size of the image to 48x48. Now in the keras model, I take a snapshot of the webcam, copy it and then draw my box and label. I was trying to do the same thing in tf.js, so I setup a canvas, got a reference to it and tried drawing to the canvas after my conversion to gray scale.
I am seeing a strange behavior where it is correctly showing the gray scale image but it is displaying it 3 times across and not sure what I am doing wrong. I have included the areas I believe the problem might reside below. Should any more info be needed, I can share more. It was my hope that someone that has already tried performing something similar may see right away what I am clearly doing wrong. Any info would be helpful. Thanks!
Modified webcam.js by adding function
preProc() {
return tf.tidy(() => {
// Reads the image as a Tensor from the webcam <video> element.
const webcamImage = tf.fromPixels(this.webcamElement);
//Resize to our image and get back single channel for greyscale
const croppedImage = this.cropImage(webcamImage, 1);
// Expand the outer most dimension so we have a batch size of 1.
const batchedImage = croppedImage.expandDims(0);
// Normalize the image between -1 and 1. The image comes in between 0-255,
// so we divide by 127 and subtract 1.
return batchedImage.toFloat().div(tf.scalar(127)).sub(tf.scalar(1));
});
}
/**
* Crops an image tensor so we get a square image with no white space.
* #param {Tensor4D} img An input image Tensor to crop.
*/
cropImage(img, dim=3) {
const size = Math.min(img.shape[0], img.shape[1]);
const centerHeight = img.shape[0] / 2;
const beginHeight = centerHeight - (size / 2);
const centerWidth = img.shape[1] / 2;
const beginWidth = centerWidth - (size / 2);
return img.slice([beginHeight, beginWidth, 0], [size, size, dim]);
}
From ui.js I am using drawFrame
export function drawFrame(image, canvas) {
const [width, height] = [300, 165];
const ctx = canvas.getContext('2d');
const imageData = new ImageData(width, height);
const data = image.dataSync();
for (let i = 0; i < height * width; ++i) {
const j = i * 4;
imageData.data[j + 0] = (data[i * 3 + 0] + 1) * 127;
imageData.data[j + 1] = (data[i * 3 + 1] + 1) * 127;
imageData.data[j + 2] = (data[i * 3 + 2] + 1) * 127;
imageData.data[j + 3] = 255;
}
ctx.putImageData(imageData, 0, 0);
}
Finally in index.js, when the predict button is pressed the below handler executes
async function predict() {
while (isPredicting) {
const predictedClass = tf.tidy(() => {
// Capture the frame from the webcam.
const imgmod = webcam.preProc();
ui.drawFrame(imgmod, grayframe);
// Returns the index with the maximum probability. This number corresponds
// to the class the model thinks is the most probable given the input.
//return predictions.as1D().argMax();
return imgmod;
});
const classId = (await predictedClass.data())[0];
predictedClass.dispose();
//ui.predictClass(classId);
await tf.nextFrame();
}
ui.donePredicting();
}
drawframe is drawing the image three times.
It has to do with the shape of the input image and the way height and width are used to crop the image. If the input image were of shape [298, 160], the canvas will not be rendered as there will be an error when trying to access index that are not in data. For instance the size of data is 298 * 160 whereas the last element of the loop would try to access the element 3 * 300 * 160. Since there are no error in the code, it indicates that the size of data is bigger than [298, 160]. At any rate, there is a mismatch in data dimension. The image are drawn 3 times because of the three channels, possibly because it was not removed before.
Instead of implementing your own way of drawing the imagedata, you can consider using tf.toPixel method

Wrong hour difference between 2 timestamps (hh:mm:ss)

Using moment.js, I want to get the time difference between 2 timestamps.
Doing the following,
var prevTime = moment('23:01:53', "HH:mm:SS");
var nextTime = moment('23:01:56', "HH:mm:SS");
var duration = moment(nextTime.diff(prevTime)).format("HH:mm:SS");
I get this result :
01:00:03
Why do I have a 1 hour difference? seconds and minutes seem to work well.
After doing that, I tried the following :
function time_diff(t1, t2) {
var parts = t1.split(':');
var d1 = new Date(0, 0, 0, parts[0], parts[1], parts[2]);
parts = t2.split(':');
var d2 = new Date(new Date(0, 0, 0, parts[0], parts[1], parts[2]) - d1);
return (d2.getHours() + ':' + d2.getMinutes() + ':' + d2.getSeconds());
}
var diff = time_diff('23:01:53','23:01:56');
output is : 1:0:3
The problem you are having here is that when putting the nextTime.diff() in a moment constructor, you are effectively feeding milliseconds to moment() and it tries to interpret it as a timestamp, which is why you don't get the expected result.
There is no "nice way" of getting the result you want apart from getting a time and manually reconstructing what you are looking for :
var dur = moment.duration(nextTime.diff(prevTime));
var formattedDuration = dur.get("hours") +":"+ dur.get("minutes") +":"+ dur.get("seconds");
And a more elegant version that will give you zero padding in the output :
var difference = nextTime.diff(prevTime);
var dur = moment.duration(difference);
var zeroPaddedDuration = Math.floor(dur.asHours()) + moment.utc(difference).format(":mm:ss");
Should make you happier!
EDIT : I have noticed you use HH:mm:SS for your format, but you should instead use HH:mm:ss. 'SS' Will give you fractional values for the seconds between 0 and 999, which is not what you want here.

Read inner and outer radius of arc - d3.js

I'm working on a project which deals with visual representation of large relational data. We are using pie chart to display the data components(In order). Due to lack of space we are displaying only 10 at a time.
Consider the following example:
Assume that I have 100 data components, out of which I'll display only 10 at a given point of time. The logic i'm using is, I'm setting start and end angle to 0(Zero) for other 90 components. Where in for those 10 components i'm calculation the start and end angle as below-
var angle = 360;
var count = 10;
if(data.length > count) angle = angle/count; //data is array of data component names
else angle = angle/data.length;
//Initially I'll be displaying first ten components
for(var i=0; i<data.length; i++){
var startAngle = i * angle;
var endAngle = startAngle + angle;
var pi = = Math.PI/180;
var arc = d3.svg.arc()
.innerRadius(innerRadius) //dynamic value, calculated based on available space
.outerRadius(outerRadius) //dynamic value, calculated based on available space
.startAngle((startAngle)*pi)
.endAngle((endAngle)*pi);
//Hiding rest of the data components
if(i >= count){
arc.startAngle(0);
arc.endAngle(0);
}
arcGroup.append("path")
.attr("d", arc)
.attr("stroke", "#2E2E2E")
.attr("stroke-width", "1")
.attr("fill","gold");
var text = arcGroup.append("text")
.attr("transform", "translate(" + arc.centroid() + ")")
.attr("text-anchor", "middle")
.attr("font-family","noto_sansregular")
.attr("font-size", 40)
.attr("font-weight","Bold")
.attr("fill", "#000000")
.attr("y",0)
.style("visibility", "visible")
.text(data[i]);
//Hiding text of hidden arcs
if(i >= count) text.style("visibility", "hidden");
}
Then if user wants to see the rest of the components, i'm providing two buttons to rotate (clock or anti-clock) the content.
If current view is -> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
On rotate by one cell clockwise, resulting view should be -> 100, 1, 2, 3, 4, 5, 6, 7, 8, 9
In this case I need to hide component '10' and display component '100', and shift rest of the cells. To achieve this i just need to change the start and end angles of the arcs. I can create new arc object with calculated angles.
The problem here is I don't know how to get the inner and outer radius of the arc, which are dynamically created.
Just like that,
...
arcGroup.append("path")
.filter(function(d) {
// You're Not restricted to the "filter" function
var innerRadius = d.innerRadius()(d);
var outerRadius = d.outerRadius()(d);
})
.attr("d", arc)
...
Technically, it is possible to retrieve the innerRadius and outerRadius from the d attribute of the path elements, but it will require parsing the DSL and will be tedious. These values are not stored nicely on the elements itself by d3.
Hence, it would be better if you recompute the innerRadius and outerRadius while updating the elements:
function showFromIdx(firstIndex) {
argGroup.selectAll('path')
.data( d3.range(data.length)
.map(function (d) {
return (d - firstIndex + data.length) % data.length;
})
)
.attr('d', function (d) {
// You will have to calculate the radii again here.
var innerRadius = foo(d), outerRadius = bar(d);
return d3.svg.arc()
.startAngle(i < count ? i * angle : 0)
.endAngle(i < count ? (i + 1) * angle : 0)
.innerRadius(innerRadius)
.outerRadius(outerRadius)(d);
});
}
Here are several functions that I wrote to get inner and outer radiuses of arcs created using d3. Please tell me if you find mistakes in code.
function getInnerRadiusFromArc(arc) {
var numbersInPattern = _getArcNumbers(arc);
// Possibly, that's sector, so it starts from 0.
// Or maybe that's something else.
if (numbersInPattern.length < 4) {
return 0;
}
// Getting minimum from the array.
var innerRadius = Math.min.apply(null, numbersInPattern);
return innerRadius;
}
function getOuterRadiusFromArc(arc) {
var numbersInPattern = _getArcNumbers(arc);
// Getting maximum from the array.
var outerRadius = Math.max.apply(null, numbersInPattern);
return outerRadius;
}
function _getArcNumbers(arc) {
// Path description parameter, containing necessary data.
var pathDescription = arc.getAttribute("d");
// We need to get all patterns like A<number>,<number>.
// RegExp source:
// http://www.regular-expressions.info/floatingpoint.html
const numberRegExp = /[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?/g;
var arcPattern = new RegExp("A" + numberRegExp.source + "," + numberRegExp.source, "g");
var arcParameters = pathDescription.match(arcPattern);
var numbersInPattern = [];
// We get all the numbers from array ["A<number>,<number>", "A<number>,<number>", ...].
for (let parameterIndex = 0; parameterIndex < arcParameters.length; parameterIndex++) {
let parameter = arcParameters[parameterIndex];
let numbers = parameter.match(numberRegExp);
if (numbers !== null) {
numbersInPattern = numbersInPattern.concat(numbers);
}
}
// Transform strings in our array to numbers.
numbersInPattern = numbersInPattern.map(function (numberString) {
return parseFloat(numberString);
});
return numbersInPattern;
}

What is wrong with my code to calculate a standard deviation?

I'm trying to calculate the standard deviation of a set of numbers as Excel would with the STDEVPA() function, but I'm not getting the correct result. I'm following this formula:
Here is my [Node] code:
var _ = require('underscore');
var prices = [
1.37312,
1.35973,
1.35493,
1.34877,
1.34853,
1.35677,
1.36079,
1.36917,
1.36769,
1.3648,
1.37473,
1.37988,
1.37527,
1.38053,
1.37752,
1.38652,
1.39685,
1.39856,
1.39684,
1.39027
];
var standardDeviation = 0;
var average = _(prices).reduce(function(total, price) {
return total + price;
}) / prices.length;
var squaredDeviations = _(prices).reduce(function(total, price) {
var deviation = price - average;
var deviationSquared = deviation * deviation;
return total + deviationSquared;
});
var standardDeviation = Math.sqrt(squaredDeviations / prices.length);
console.log(standardDeviation);
When I run this, I get 0.26246286981807065, and instead I should get 0.0152.
Please note that I posted on StackOverflow and not the Mathematics site because, in my opinion, this is more geared toward programming than mathematics. If I post there, they will tell me to post here because this is related to programming.
If you console.log(total) inside your calculation of squaredDeviations, you'll see you're starting out with the value 1.37312, namely the first thing in your list. You need to explicitly tell it to start at 0, which is the third optional argument to reduce. Just replace:
var squaredDeviations = _(prices).reduce(function(total, price) {
var deviation = price - average;
var deviationSquared = deviation * deviation;
return total + deviationSquared;
});
with
var squaredDeviations = _(prices).reduce(function(total, price) {
var deviation = price - average;
var deviationSquared = deviation * deviation;
return total + deviationSquared;
}, 0);
See the underscore documentation for more details. In particular, note that things work when calculating the mean without passing this additional argument because, in said case, the iteratee function will not be applied to the first element.

How to avoid the overlapping of text elements on the TreeMap when child elements are opened in D3.js?

I created a Tree in D3.js based on Mike Bostock's Node-link Tree. The problem I have and that I also see in Mike's Tree is that the text label overlap/underlap the circle nodes when there isn't enough space rather than extend the links to leave some space.
As a new user I'm not allowed to upload images, so here is a link to Mike's Tree where you can see the labels of the preceding nodes overlapping the following nodes.
I tried various things to fix the problem by detecting the pixel length of the text with:
d3.select('.nodeText').node().getComputedTextLength();
However this only works after I rendered the page when I need the length of the longest text item before I render.
Getting the longest text item before I render with:
nodes = tree.nodes(root).reverse();
var longest = nodes.reduce(function (a, b) {
return a.label.length > b.label.length ? a : b;
});
node = vis.selectAll('g.node').data(nodes, function(d, i){
return d.id || (d.id = ++i);
});
nodes.forEach(function(d) {
d.y = (longest.label.length + 200);
});
only returns the string length, while using
d.y = (d.depth * 200);
makes every link a static length and doesn't resize as beautiful when new nodes get opened or closed.
Is there a way to avoid this overlapping? If so, what would be the best way to do this and to keep the dynamic structure of the tree?
There are 3 possible solutions that I can come up with but aren't that straightforward:
Detecting label length and using an ellipsis where it overruns child nodes. (which would make the labels less readable)
scaling the layout dynamically by detecting the label length and telling the links to adjust accordingly. (which would be best but seems really difficult
scale the svg element and use a scroll bar when the labels start to run over. (not sure this is possible as I have been working on the assumption that the SVG needs to have a set height and width).
So the following approach can give different levels of the layout different "heights". You have to take care that with a radial layout you risk not having enough spread for small circles to fan your text without overlaps, but let's ignore that for now.
The key is to realize that the tree layout simply maps things to an arbitrary space of width and height and that the diagonal projection maps width (x) to angle and height (y) to radius. Moreover the radius is a simple function of the depth of the tree.
So here is a way to reassign the depths based on the text lengths:
First of all, I use the following (jQuery) to compute maximum text sizes for:
var computeMaxTextSize = function(data, fontSize, fontName){
var maxH = 0, maxW = 0;
var div = document.createElement('div');
document.body.appendChild(div);
$(div).css({
position: 'absolute',
left: -1000,
top: -1000,
display: 'none',
margin:0,
padding:0
});
$(div).css("font", fontSize + 'px '+fontName);
data.forEach(function(d) {
$(div).html(d);
maxH = Math.max(maxH, $(div).outerHeight());
maxW = Math.max(maxW, $(div).outerWidth());
});
$(div).remove();
return {maxH: maxH, maxW: maxW};
}
Now I will recursively build an array with an array of strings per level:
var allStrings = [[]];
var childStrings = function(level, n) {
var a = allStrings[level];
a.push(n.name);
if(n.children && n.children.length > 0) {
if(!allStrings[level+1]) {
allStrings[level+1] = [];
}
n.children.forEach(function(d) {
childStrings(level + 1, d);
});
}
};
childStrings(0, root);
And then compute the maximum text length per level.
var maxLevelSizes = [];
allStrings.forEach(function(d, i) {
maxLevelSizes.push(computeMaxTextSize(allStrings[i], '10', 'sans-serif'));
});
Then I compute the total text width for all the levels (adding spacing for the little circle icons and some padding to make it look nice). This will be the radius of the final layout. Note that I will use this same padding amount again later on.
var padding = 25; // Width of the blue circle plus some spacing
var totalRadius = d3.sum(maxLevelSizes, function(d) { return d.maxW + padding});
var diameter = totalRadius * 2; // was 960;
var tree = d3.layout.tree()
.size([360, totalRadius])
.separation(function(a, b) { return (a.parent == b.parent ? 1 : 2) / a.depth; });
Now we can call the layout as usual. There is one last piece: to figure out the radius for the different levels we will need a cumulative sum of the radii of the previous levels. Once we have that we simply assign the new radii to the computed nodes.
// Compute cummulative sums - these will be the ring radii
var newDepths = maxLevelSizes.reduce(function(prev, curr, index) {
prev.push(prev[index] + curr.maxW + padding);
return prev;
},[0]);
var nodes = tree.nodes(root);
// Assign new radius based on depth
nodes.forEach(function(d) {
d.y = newDepths[d.depth];
});
Eh voila! This is maybe not the cleanest solution, and perhaps does not address every concern, but it should get you started. Have fun!

Resources