I would like to know if its possible to extraxt the groundlevel of nodes from the resulfile. I have already extracted the water level and computed the maximum values, but in order for it to be truly useful, i need the groundlevel to calculate if the node is flooded.
I use this function to get the coordinates:
def get_node_geometry(nodes):
allnodes = [[n.ID, Point(n.XCoordinate, n.YCoordinate)] for n in nodes]
allnode_df = pd.DataFrame(allnodes, columns=['MUID', 'geometry'])
allnode_df.set_index('MUID', inplace = True)
return gpd.GeoDataFrame(allnode_df, geometry = 'geometry')
and these lines to get waterlevel:
varname = "WaterLevel"
queries_all = [QueryDataNode(varname, n.Id) for n in res1d.data.Nodes]
df_node = res1d.read(queries_all)
Related
I wrote this to simulate studying for a test
#number of questions
qList = [] * q
#question numbers
for i in range(len(qList)):
qList[i] = i + 1
#questions studied
sList = [] * s
for i in range (0,s):
x = randint(0,len(qList))
sList[i] = qList[x]
#questions passed
pList = []
for i in range(len(sList)):
if i in sList and i in qList:
pList.apppend(i)
the line sList[i] = qList[x] gives me an index out of range error. I haven't used lists in a while and I can't figure out what is wrong here.
I am trying to get the output here as three lists
a list of questions
the questions that have been studied
the questions passed
randint includes both boundaries:
Return a random integer N such that a <= N <= b. Alias for randrange(a, b+1).
Change invocation to:
x = randint(0, len(qList) - 1)
# or
x = randrange(0, len(qList))
Also you are instantiating empty arrays, you need to pass some element so it can be repeated (or better use list comprehensions):
qList = [1] * q
sList = [1] * s
Hi I'm starting to play around with NIM
I get a "can't evaluate at compile time" error on this code:
import strutils
type
Matrix[x, y: static[int], T] = object
data: array[x * y, T]
var n,m: int = 0
proc readFile() =
let f = open("matrix.txt")
defer: f.close()
var graph_size = parseInt(f.readline)
var whole_graph: Matrix[graph_size, graph_size, int]
for line in f.lines:
for field in line.splitWhitespace:
var cell = parseInt(field)
whole_graph[n][m] = cell
m = m + 1
n = n + 1
readFile()
Any help appreciated.
Unless you absolutely positively need array in this scenario while not knowing its size at compile-time, you may want to rather swap to the seq type, whose size does not need to be known at compile-time.
Together with std/enumerate you can even save yourself the hassle of tracking the index with n and m:
import std/[strutils, enumerate]
type Matrix[T] = seq[seq[T]]
proc newZeroIntMatrix(x: int, y: int): Matrix[int] =
result = newSeqOfCap[seq[int]](x)
for i in 0..x-1:
result.add(newSeqOfCap[int](y))
for j in 0..y-1:
result[i].add(0)
proc readFile(): Matrix[int] =
let f = open("matrix.txt")
defer: f.close()
let graph_size = parseInt(f.readline)
var whole_graph = newZeroIntMatrix(graph_size, graph_size)
for rowIndex, line in enumerate(f.lines):
for columnIndex, field in enumerate(line.split):
let cell = parseInt(field)
whole_graph[rowIndex][columnIndex] = cell
result = whole_graph
let myMatrix = readFile()
echo myMatrix.repr
Further things I'd like to point out though are:
array[x * y, T] will not give you a 2D array, but a single array of length x*y. If you want a 2D array, you would most likely want to store this as array[x, array[y, T]]. That is assuming that you know x and y at compile-time, so your variable declaration would look roughly like this: var myMatrix: array[4, array[5, int]]
Your Matrix type has the array in its data field, so trying to access the array with that Matrix type needs to be done accordingly (myMatrix.data[n][m]). That is, unless you define proper []and []= procs for the Matrix type that do exactly that under the hood.
I have successfully extracted the count of a specific word from a dataset but, it is taking too much time. I am new to parallel programming.
How can I create parallelism in the following code:
df = dd.read_csv('crime.csv', encoding="ISO-8859-1")
distinct_values = df.YEAR.unique().compute()
counter = len(distinct_values)
values_count = {}
for i in distinct_values:
count = df[df.YEAR == i].YEAR.value_counts().compute()
values_count.update(count)
list = []
for x, y in values_count.items():
dict = {}
for i in x, y:
dict['name'] = x
dict['value'] = y
# print(dict)
list.append(dict)
# print(list)
maximum = max(distinct_values)
mininmum = min(distinct_values)
Maybe you're looking for a groupby aggregation like the following?
df.groupby("YEAR").count.compute()
Or, if you need to do this as many operations, you should at least use the dask.compute function with many inputs rather than call the .compute method many times.
the function shown below is running quite slow even though I used swifter to call it. Does anyone know how to speed this up? My python knowledge is limited at this point and I would appreciate any help I could get. I tried using map() function but somehow it didnt work for me. I guess the nested for loop makes it rather slow, right?
BR,
Hannes
def polyData(uniqueIds):
for index in range(len(uniqueIds) - 1):
element = uniqueIds[index]
polyData1 = df[df['id'] == element]
poly1 = build_poly(polyData1)
poly1 = poly1.buffer(0)
for secondIndex in range(index + 1, len(uniqueIds)):
otherElement = uniqueIds[secondIndex]
polyData2 = df[df['id'] == otherElement]
poly2 = build_poly(polyData2)
poly2 = poly2.buffer(0)
# Calculate overlap percentage wise
overlap_pct = poly1.intersection(poly2).area/poly1.area
# Form new DF
df_ol = pd.DataFrame({'id_1':[element],'id_2':[otherElement],'overlap_pct':[overlap_pct]})
# Write to SQL database
df_ol.to_sql(name='df_overlap', con=e,if_exists='append',index=False)
This function is inherently slow for large amounts of data due to its complexity (trying every 2-combination of a set). However, you're calculating the 'poly' for the same ids multiple times, even though it seems that you can calculate them only once beforehand (which might be expensive) and store them for later usage. So try to extract the building of the polys.
def getPolyForUniqueId(uid):
polyData = df[df['id'] == uid]
poly = build_poly(polyData)
poly = poly.buffer(0)
return polyData
def polyData(uniqueIds):
polyDataList = [getPolyForUniqueId(uid) for uid in uniqueIds]
for index in range(len(uniqueIds) - 1):
id_1 = uniqueIds[index]
poly_1 = polyDataList[index]
for secondIndex in range(index + 1, len(uniqueIds)):
id_2 = uniqueIds[secondIndex]
poly_2 = polyDataList[secondIndex]
...
I have a long list of sources and targets that form a graph as follows:
id_a = [...] #source nodes
id_b = [...] #target nodes
distance = [..] #distance between source and target nodes
G = nx.Graph()
path, length = [], []
for a, b, c in zip(id_a, id_b, distance):
G.add_edge(a, b, weight=c)
cl is a subset of all the nodes in the graph and I want to extract the paths interconnecting all of cl together so I use all_simple_paths()
path = []
for i in range(len(cl)):
for j in range(len(cl)):
if i != j:
path.append(nx.all_simple_paths(G, source=cl[i], target=cl[j]))
I want to be able to list all the simple paths and their lengths so I try:
for i in range(len(path)):
total_length = 0
for j in range(len(path[i])-1):
source, target = path[i][j], path[i][j+1]
edge = G[source][target]
length = edge['weight']
total_length += length
length.append(total_length)
But I keep getting the error
object of type 'generator' has no len()
And I can't figure out how to convert the generator of all_simple_paths() to lists that I can iterate over and extract the full lengths of all the paths.
Any help is appreciated!
If you read the documentation of all_simple_paths, you will see that it returns a generator. So, just use extend instead of append method like this
path = []
for i in range(len(cl)):
for j in range(len(cl)):
if i != j:
path.extend(nx.all_simple_paths(G, source=cl[i], target=cl[j]))
For more info on why extend works in this case, see this answer.
Also I see in the last part of your code, you are setting length as length = edge['weight'], then appending using length.append(total_length). This will return as error, since the edge weight will be an int. Use different variable names something like this
path_weight = [] #<----- List to store all path's weights
for i in range(len(path)):
total_length = 0
for j in range(len(path[i])-1):
source, target = path[i][j], path[i][j+1]
edge = G[source][target]
length = edge['weight'] #<--- Get the weight
total_length += length
path_weight.append(total_length) #Append to the list