Iterate using the dataset columns - hpcc-ecl

I have a dataset with more than 30 columns and I would like to calculate the correlation between each of them (and preferably put the results in a correlation matrix). Is there any way to create a loop that runs the CORRELATION() function for all possible pairs of columns?
An example to make it clearer: Given a dataset of columns y, x1, x2 and x3, I would like to create the following table with the correlations between each one:
CorrelationMatrix

Here's an example of how to accomplish that:
filelayout := RECORD
UNSIGNED myid;
REAL y;
REAL x1;
REAL x2;
REAL x3;
END;
GFred := DATASET([ {1, 1.1, 2.2, 3.3, 4.4},
{2, 5, 6, 7, 8},
{3, 9, 10, 11, 12},
{4, 13.1, 14.2, 15.3, 16.4}],
filelayout);
SetVals := [GFred.y,GFred.x1,GFred.x2,GFred.x3];
OutRecCnt := COUNT(SetVals);
OneRec := DATASET([{0}],{UNSIGNED1 h});
filelayout XF(OneRec L, INTEGER C) := TRANSFORM
SELF.myid := C;
SELF.y := CORRELATION(GFred,SetVals[C],y);
SELF.x1 := CORRELATION(GFred,SetVals[C],x1);
SELF.x2 := CORRELATION(GFred,SetVals[C],x2);
SELF.x3 := CORRELATION(GFred,SetVals[C],x3);
END;
P := NORMALIZE(OneRec,OutRecCnt,XF(LEFT,COUNTER));
OUTPUT(P);

Related

Mathematica: Excel imported list parameters not working on piecewise function

I have a piecewise function like:
q1N := (e + w)/(e + b)
FeasR := Piecewise[{{1, q1N <= q}, {0, q < q1N}}]
parameters e,w and b are imported from Excel using this code:
InData := Import["C:\\Users\\HR\\Desktop\\DatImp.csv"]
MainData := InData[[2 ;; 1001, 2 ;; 7]]
b := MainData[[All, 1]]
w := MainData[[All, 2]]
e := MainData[[All, 3]]
q := MainData[[All, 4]]
b,e,q and w shape different cases where FeasR would determine feasibility based on comparison of q1N and q.
The problem is that FeasR output is not in the form of {{1,0,1,1,0,...}}.
I would appreciate your help.
In this example InData is random so the output will differ.
InData = RandomReal[10, {10, 7}];
MainData = InData[[2 ;; 10, 2 ;; 7]];
b = MainData[[All, 1]]
w = MainData[[All, 2]]
e = MainData[[All, 3]]
q = MainData[[All, 4]]
q1N := (e + w)/(e + b)
FeasR = Piecewise[{{1, Thread[q1N <= q]}, {0, Thread[q < q1N]}}]
{ 1 {True,True,False,True,True,True,True,True,True}
{ 0 True
However, this is not the format you want. Use Boole instead.
FeasR = Boole[Thread[q1N <= q]]
{1, 1, 0, 1, 1, 1, 1, 1, 1}

Else statement executing even though the if statement is true

for school i am making an assignment, implementing astar search with the use of dictionaries. I have been trying a few things, but now i am stuck. in my first iteration the if statement if f_next < f: is true, so why does it not break out of the loop. It immediately goes to the else statement so my closed list contains both values from the if and else statement, how can i fix this. I have provided my code, some assignment information and the input below.
Given a grid-like graph, find the shortest path using A* between the two nodes with the maximum weight. Use the Euclidean as heuristic function, which is calculated between two points in a coordinate system (x1,y1) and (x2,y2) as follows:
d = p(x2 −x1)2 + (y2 −y1)2 (1) where, the nodes of the graph are represented as such points (see the input format below). You can assume that there exists a path between the two nodes with the maximum weight, and that all nodes have a unique weight.
Data format 1. The following is the input format expected by the algorithm for Task 1.
The program takes as input a list of nodes, represented as points in a coordinate system (e.g.,(x1,y1) and their weights separated by a ;.
In a new line the data-set of the graph edges are given following the pattern ”source node (xs,ys), end node (xe,ye), edge cost; ...”.
(x1), (y1), (weight1); ...(xn), (yn), (weightn);\n
(x1), (y1), (x2), (y2), (edgecost1); ...(xs), (ys), (xe), (ye), (edgecostm);\n
The expected output is the path from the start to the end node.
(xstart), (ystart)->(xi), (yi)...->(xend), (yend)\n
Input:
0, 0, 193; 0, 1, 146; 0, 2, 121; 0, 3, 174; 0, 4, 144; 1, 0, 191; 1, 1, 123; 1, 2, 151; 1, 3, 126; 1, 4, 163; 2, 0, 182; 2, 1, 107; 2, 2, 109; 2, 3, 125; 2, 4, 165; 3, 0, 169; 3, 1, 149; 3, 2, 180; 3, 3, 199; 3, 4, 199; 4, 0, 160; 4, 1, 148; 4, 2, 123; 4, 3, 197; 4, 4, 156
0, 0, 1, 0, 1; 0, 0, 0, 1, 1; 0, 1, 1, 1, 1; 0, 1, 0, 2, 1; 0, 2, 1, 2, 1; 0, 2, 0, 3, 1; 0, 3, 1, 3, 1; 0, 3, 0, 4, 1; 0, 4, 1, 4, 1; 1, 0, 2, 0, 1; 1, 0, 1, 1, 1; 1, 1, 2, 1, 1; 1, 1, 1, 2, 1; 1, 2, 2, 2, 1; 1, 2, 1, 3, 1; 1, 3, 2, 3, 1; 1, 3, 1, 4, 1; 1, 4, 2, 4, 1; 2, 0, 3, 0, 1; 2, 0, 2, 1, 1; 2, 1, 3, 1, 1; 2, 1, 2, 2, 1; 2, 2, 3, 2, 1; 2, 2, 2, 3, 1; 2, 3, 3, 3, 1; 2, 3, 2, 4, 1; 2, 4, 3, 4, 1; 3, 0, 4, 0, 1; 3, 0, 3, 1, 1; 3, 1, 4, 1, 1; 3, 1, 3, 2, 1; 3, 2, 4, 2, 1; 3, 2, 3, 3, 1; 3, 3, 4, 3, 1; 3, 3, 3, 4, 1; 3, 4, 4, 4, 1
I am trying to find a path between (0,0) and (3,4)
import collections
import math
import sys
class Graph:
"""Class to represent a Graph, as a list of weighted nodes and edges."""
def __init__(self):
"""Function to initialize a Graph object"""
self.node_g = {}
self.edge_g = collections.defaultdict(list)
pass
def add_node(self, node_id, weight):
"""Function to add a node to a Graph object."""
self.node_g[node_id] = weight
print(self.node_g)
pass
def add_edge(self, source_id, end_id, weight):
weightedge = (end_id, weight)
self.edge_g[source_id].append(weightedge)
pass
def __repr__(self):
return "nodes:% s edges:% s" % (self.node_g, self.edge_g)
def __str__(self):
return "From str method of graph: nodes are % s, " \
"edges are % s" % (self.node_g, self.edge_g)
def build_Graph(nodes, edges):
"""Function to build a grid-like Graph object from the input data.
Parameters
----------
nodes : list of nodes, each represented as coordinates, and node_weight.
For example: x1, y1, weight; x2, y2, weight; ...
edges : list of edges, each represented as source and end node coordintates, and edge_weight.
For example: x1, y1, x2, y2, weight; x3, y3, x4, y4, weight; ...
Return
----------
A Graph object.
"""
G = Graph()
for n in nodes:
aux = n.split(', ')
temp1 = int(aux[0])
temp2 = int(aux[1])
temp_node = (temp1, temp2)
G.add_node(temp_node, weight=int(aux[2]))
for e in edges:
aux = e.split(', ')
temp1 = int(aux[0])
temp2 = int(aux[1])
temp = (temp1, temp2)
auxn1 = int(aux[2])
auxn2 = int(aux[3])
auxn = (auxn1, auxn2)
G.add_edge(temp, auxn, weight=int(aux[4]))
return G
def print_output(nodes):
"""Function to print the shortest path between the two nodes with the highest weigths.
Parameters
----------
nodes : list of list of nodes (represented as: tuples of coordinates).
"""
# nodes = [x1, y1, x2, y2, x3, y3]
# Expected output: x1, y1->x2, y2->x3, y3
def astar_shortest_path(G, source_id, end_id, heuristic):
"""Function to return the shortest path between two nodes in a Graph"""
open_set = [source_id]
#print(open_set)
closed_set = []
g = 0
h = heuristic(source_id, end_id)
f = g + h
#print(G.edge_g[open_set[0]][0])
#print(G.edge_g[source_id])
#print(G.edge_g[end_id])
while len(open_set) > 0:
if source_id == end_id:
return closed_set
for index, item in enumerate(G.edge_g[open_set[0]]):
print(item)
var = item[0]
print(var)
if not index:
next = item[0]
g_next = item[1]
h_next = heuristic(item[0], end_id)
f_next = h_next + g_next
if f_next < f:
f += f_next
h += h_next
g += g_next
open_set.append(next)
print(open_set)
closed_set.append(open_set[0])
print(closed_set)
open_set.remove(open_set[0])
print(open_set)
else:
next_2 = item[0]
g_next_2 = item[1]
h_next_2 = heuristic(item[0], end_id)
f_next_2 = g_next_2 + h_next_2
if f_next_2 < f:
f += f_next_2
h += h_next_2
g += g_next_2
closed_set.append(open_set[0])
print(closed_set)
open_set.remove(open_set[0])
print(open_set)
open_set.append(next_2)
print(open_set)
#print(open_set)
def heuristic(a, b):
#print(a, b)
"""Function to compute the Euclidean distance between two nodes."""
dx = abs(b[0] - a[0]) ** 2
#print(dx)
dy = abs(b[1] - a[1]) ** 2
#print(dy)
return math.sqrt(dx + dy)
if __name__ == '__main__':
# Read the input
# The first line is made of a list of nodes, written as tuples of cartesian coordinates.
# For example: x1, y1, weight; x2, y2, weight; ...
# In the previous example, x1, y1, weight; is the first node
nodes = input().split('; ')
# The second line is made of edges, written as source and end node coordinates, and edge_weight.
# For example: x1, y1, x2, y2, weight; x3, y3, x4, y4, weight; ...
# In the previous example, x1, y1, x2, y2, weight; is the first edge
edges = input().split('; ')
# Build a grid graph from the input nodes and edges
G = build_Graph(nodes, edges)
print(G)
sort_edge = sorted(G.node_g, key=G.node_g.get)
max1 = sort_edge[-2]
#print(max1)
max2 = sort_edge[-1]
#print(max2)
# Find the two nodes with the highest weights in the graph
source_id = (0, 0) #max1
end_id = (3, 4) #max2
# Compute the path between the two nodes with the highest weight
# The source node is the one with the highest weigth
# You are free to customize the following function
s_path = astar_shortest_path(G, source_id, end_id, heuristic)
# Expected output format: list of lists of nodes (represented as: tuples of coordinates).
# For example: x1, y1->x2, y2->x3, y3
print_output(s_path)
#############

MiniZinc - Constraint to enforce two arrays be equal

I'm trying to model a satisfaction problem with MiniZinc but I'm stuck at coding this constraint:
Given two arrays A and B of equal length, enforce that A is a permutation of B
In other words, I'm looking for a constraint that enforces [1,2,2]=[1,2,2] and also [1,2,2]=[2,2,1]. Informally, A and B have to be equal. The arrays are both defined on the same set of integers, in particular the set 1..n-1, for some n. Values in both A and B can be repeated (see example).
Is there such a global constraint in MiniZinc? Thank you.
Here is the predicate I tend to use for these cases. It requires an extra array (p) which contains the permutations from array a to array b.
/*
Enforce that a is a permutation of b with the permutation
array p.
*/
predicate permutation3(array[int] of var int: a,
array[int] of var int: p,
array[int] of var int: b) =
forall(i in index_set(a)) (
b[i] = a[p[i]]
)
/\
all_different(p)
;
A simple model using this:
include "globals.mzn";
int: n = 3;
array[1..n] of var 1..2: x;
array[1..n] of var 1..2: y;
array[1..n] of var 1..n: p;
/*
Enforce that array b is a permutation of array a with the permutation
array p.
*/
predicate permutation3(array[int] of var int: a,
array[int] of var int: p,
array[int] of var int: b) =
forall(i in index_set(a)) (
b[i] = a[p[i]]
)
/\
all_different(p)
;
solve satisfy;
constraint
x = [2,2,1] /\
permutation3(x,p,y)
;
output [
"x: \(x)\ny: \(y)\np: \(p)\n"
];
Output:
x: [2, 2, 1]
y: [1, 2, 2]
p: [3, 2, 1]
----------
x: [2, 2, 1]
y: [2, 1, 2]
p: [2, 3, 1]
----------
x: [2, 2, 1]
y: [1, 2, 2]
p: [3, 1, 2]
----------
x: [2, 2, 1]
y: [2, 1, 2]
p: [1, 3, 2]
----------
x: [2, 2, 1]
y: [2, 2, 1]
p: [2, 1, 3]
----------
x: [2, 2, 1]
y: [2, 2, 1]
p: [1, 2, 3]
----------
==========
There is an alternative formulation of this which don't requires the extra permutation p (it's defined inside the predicate):
predicate permutation3b(array[int] of var int: a,
array[int] of var int: b) =
let {
array[index_set(a)] of var index_set(a): p;
} in
forall(i in index_set(a)) (
b[i] = a[p[i]]
)
/\
all_different(p)
;
For the same problem, this will only output these 3 different solutions (the first model has more solutions since the the permutations differs).
x: [2, 2, 1]
y: [2, 2, 1]
----------
x: [2, 2, 1]
y: [2, 1, 2]
----------
x: [2, 2, 1]
y: [1, 2, 2]
----------
==========
Personally I tend to use the first version since I tend to want to output the permutation and like to have control over the variables.
In addition to the predicate shown by hakank, here are two other ways to express the same predicate
include "globals.mzn";
%
% Ensure that a and b are perumutations of each other by
% counting the number of occurences of each value in the
% domains of a and b,
%
predicate permutation_count(array[int] of var int: a,
array[int] of var int: b) =
let {
set of int: I = index_set(a),
constraint assert(I = index_set(b), "Index sets of a and b must match"),
set of int: domain = dom_array(a) intersect dom_array(b),
set of int: NValues = 1..card(domain),
array[NValues] of int: values = [ v | v in domain ],
array[NValues] of var 0..card(I): counts,
} in
global_cardinality_closed(a, values, counts) /\
global_cardinality_closed(b, values, counts);
%
% Ensure that a and b are permutations of each other by
% sorting each and constraining that to be the same.
%
predicate permutation_sort(array[int] of var int: a,
array[int] of var int: b) =
let {
set of int: I = index_set(a),
constraint assert(I = index_set(b), "Index sets of a and b must match"),
set of int: domain = dom_array(a) intersect dom_array(b),
array[I] of var domain: sorted_values,
} in
sorted_values = sort(a) /\
sorted_values = sort(b);
int: n = 3;
array[1..n] of var 1..2: x;
array[1..n] of var 1..2: y;
constraint permutation_count(x, y);
solve satisfy;
The first one counts the values in both input arrays, since in permutations the counts must be the same. The second variant uses the sorting constraint to sort both a and b to check that they are the same.
Which variant works best can vary between solvers, problems, and problem isntances. The counting solution may be problematic if the domains of the inputs are large, which is worth remembering.

Revolve around a horizontal line in Mathematica?

I am very new to Mathematica. I have version 11, if that makes a difference.
I am trying to take the area formed by the following lines and and revolve it to form a 3D solid.
y = e^-x
Here is my code, in two sections
f[x_] := E^-x
g[x_] := 1
Plot[{f[x], g[x]}, {x, 0, 2}, Filling -> {1 -> {2}},
PlotLegends -> {"f[x]", "g[x]", "h[y]"}]
Next:
RevolutionPlot3D[(1 - f[x]) , {x, 0, 2}, RevolutionAxis -> "X"]
Here is the 2D and 3D representations:
The 2D one is correct, but not the 3D. I want to rotate the area about y=2 (horizontal line) as to form a shape with a hole in the center. I don't know how to set the axis of rotation to anything other than an axis line. I just want y=2.
How do you accomplish this?
RevolutionPlot3D isn't the right tool for what you want for 2 reasons. First, you want to rotate a 2D region not a line. Second, you want to rotate around a line that isn't one of the axes. RegionPlot3D is the built-in tool for the job. You can easily set up your region as a boolean region, just think about the conditions that the radius x^2 + y^2 has to satisfy
RegionPlot3D[
1 < z^2 + y^2 < (2 - Exp[-x])^2, {x, 0, 2}, {y, -3, 3}, {z, -3, 3}]
I showed the result from 2 different angles to point out the shortcomings of RegionPlot3D. You could improve this result by using a high value for the PlotPoints option, but it isn't great. That's why you should use Simon Woods's function contourRegionPlot3D, defined in this post:
contourRegionPlot3D[
1 < z^2 + y^2 < (2 - Exp[-x])^2, {x, 0, 2}, {y, -3, 3}, {z, -3, 3}]

ColorFunction with InterpolationFunction in RegionPlot3D

I am writing to ask a question regarding the implementation of a field-dependent color in a 3d region plot in Mathematica.
Specifically, I have created the following plot, where f[x,y,z] is an interpolating function of a three-dimensional array (this is done to have lower resolution plots with ease, since the amount of data in the array is significant).
The problem I am encountering is that if I run the following instruction:
RegionPlot3D[f[x, y, z] >= 0.5 && f[x, y, z] <= 0.6, {x, 0, 1}, {y, 0,0.416}, {z, 0, 0.666},
ColorFunction -> Function[{x, y, z}, Hue[Rescale[f[x, y, z], {0, 1}]]]]
The color is not imposed correctly (i get a region of uniform color). If I utilize a function g instead (can be any function, e.g. the norm of the point position) inside the Hue, so that
Hue[Rescale[g[x, y, z], {0, 1}]]
the color information is passed correctly. I assume that I am making a mistake with the handling of InterpolatingFunction objects. How should this problem be handled?
Any help is appreciated.
RegionPlot3D passes 4 arguments to ColorFunction :
Try this (just adding a dummy arg to Function )
RegionPlot3D[f[x, y, z] >= 0.5 && f[x, y, z] <= 0.6, {x, 0, 1}, {y, 0,0.416}, {z, 0, 0.666},
ColorFunction -> Function[{x, y, z, p}, Hue[Rescale[f[x, y, z], {0, 1}]]]]
or like this:
RegionPlot3D[f[x, y, z] >= 0.5 && f[x, y, z] <= 0.6, {x, 0, 1}, {y, 0,0.416}, {z, 0, 0.666},
ColorFunction -> ( Hue[Rescale[f[#1, #2, #3 ], {0, 1}]] &)]

Resources