read an excel sheet which have multiple sheets and without table [closed] - sharepoint-online

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 days ago.
Improve this question
I have an excel sheet which contain 10 sheets and saved inside SharePoint document library. and i want to do the following:-
read the data of a single sheet named "CompnayInfo" , the sheet contain the first row as headers and have filters but the sheet does not have table.
So how i can read this excel sheet rows? here is a screen shot of the excel sheet:-

I'm going to give you a fairly generic answer but you can take it from there and it will give you a result.
You can use the Run script action in the Excel online connector(s). It allows you to write a script using the Office Scripts framework which composes of a set of typescript classes, etc. that allows you to work with Excel workbooks stored in SharePoint or OneDrive.
It' like VBA but for the web. However, it's not as complete as VBA, but, it does near on everything you'd ever need and that which isn't catered for can be achieved via alternative means.
This is the script I put together, it's basic but gives you all of the data. You'd need to do the work to enhance it and make it more useful for you.
function main(workbook: ExcelScript.Workbook, worksheetName: string) {
let worksheet = workbook.getWorksheet(worksheetName);
return worksheet.getUsedRange().getValues();
}
If you need to know how to create a script, follow these links and please note, for absolute ease, use the excel web client to create the script, the desktop version is around but don't complicate it for yourself and go through the web, it's tried and tested ...
https://learn.microsoft.com/en-us/office/dev/scripts/overview/excel
https://support.microsoft.com/en-us/office/introduction-to-office-scripts-in-excel-9fbe283d-adb8-4f13-a75b-a81c6baf163a
For the worksheet I ran the script over, this was the resulting JSON. It's a 2 dimensional array by rows and columns of data ...
[
[
"Date",
"Time",
"g/l ni",
"ECV808Ha",
"Multiple"
],
[
44698.64584,
44698.64584,
4,
5,
20
],
[
44698.64928,
44698.64928,
5,
15,
75
],
[
44698.65506,
44698.65506,
5.2,
4.8,
24.96
],
[
44698.66566,
44698.66566,
4,
45,
180
],
[
44699.63499,
44699.63499,
8.6,
9.3,
79.98
],
[
44699.63914,
44699.63914,
15.2,
8.2,
124.64
],
[
44699.64223,
44699.64223,
14.2,
18.6,
264.12
],
[
44699.6455,
44699.6455,
8.2,
2.6,
21.32
],
[
44699.65178,
44699.65178,
15,
15.2,
228
],
[
44699.66725,
44699.66725,
14,
12.2,
170.8
],
[
44699.67292,
44699.67292,
15,
16,
240
],
[
44699.67413,
44699.67413,
12.2,
13.4,
163.48
],
[
44700.50296,
44700.50296,
4,
2.5,
10
],
[
44700.50534,
44700.50534,
5,
8,
40
],
[
44700.52765,
44700.52765,
4.5,
8.6,
38.7
],
[
44700.52861,
44700.52861,
8.5,
4.2,
35.7
],
[
44700.5975,
44700.5975,
8,
14,
112
],
[
44700.60117,
44700.60117,
4,
8,
32
],
[
44700.60197,
44700.60197,
4,
2.5,
10
],
[
44700.60348,
44700.60348,
4.5,
5.25,
23.625
],
[
44700.60414,
44700.60414,
80,
2.5,
200
],
[
44700.60597,
44700.60597,
4,
2.5,
10
],
[
44700.61531,
44700.61531,
8,
6,
48
],
[
44700.645,
44700.645,
4,
2.5,
10
],
[
44700.64679,
44700.64679,
4,
4.5,
18
],
[
44700.6484,
44700.6484,
8,
6.2,
49.6
],
[
44700.64964,
44700.64964,
6,
6,
36
],
[
44701.38266,
44701.38266,
4,
5,
20
],
[
44701.40955,
44701.40955,
5,
5.5,
27.5
],
[
44701.41484,
44701.41484,
8,
6,
48
],
[
44701.41723,
44701.41723,
8,
65,
520
],
[
44701.42251,
44701.42251,
4,
6,
24
],
[
44701.42304,
44701.42304,
4,
8,
32
],
[
44701.42365,
44701.42365,
8,
90,
720
],
[
44701.42384,
44701.42384,
8,
60,
480
],
[
44701.42045,
44701.42045,
4,
5,
20
],
[
44701.43058,
44701.43058,
18,
16,
288
],
[
44704.3115,
44704.3115,
4,
8,
32
],
[
44704.43532,
44704.43532,
6,
525,
3150
],
[
44704.44973,
44704.44973,
5,
6,
30
]
]
... If you write your own script, you can return specific JSON structures that suit your requirement.

Related

Prevent Array Linebreaks in Console output (Visual Studio Code; Node.js)

I would like to change the settings of how many characters or elements of my array are shown before the line breaks.
This happens for some reason after 6 elements:
console.log([1, 2, 3, 4, 5])
console.log([1, 2, 3, 4, 5, 6, 7])
This will result in the following output:
[ 1, 2, 3, 4, 5 ]
[
1, 2, 3, 4,
5, 6, 7
]
However, I would prefer the following output:
[ 1, 2, 3, 4, 5 ]
[ 1, 2, 3, 4, 5, 6, 7 ]
Does anyone know, how I can change these settings?
When I tried to google it, there were all kind of suggestions about in-code line breaks and general logging, but I could not find anyone with this issue.
Hope this is a no brainer.
Thanks in advance.
Br, Till

Convert large arrays in pandas dataframe to large numpy array

I have a dateframe that has a column that looks like this:
0 [ [ 1051, 0, 10181, 62, 17, ...
1 [ [ 882, 0, 9909, 59, 23, 9...
2 [ [ 1061, 0, 10192, 60, 17, ...
3 [ [ 122, 4, 501, 2, 8, 3, ...
4 [ [ 397, 1, 859, 9, 8, 5, ...
5 [ [ 1213, 1, 10791, 23, 17, ...
6 [ [ 1395, 3, 11147, 0, 17, ...
7 [ [ 757, 3, 1900, 34, 23, 8...
8 [ [ 129, 0, 507, 10, 8, 3, ...
9 [ [ 1438, 0, 11177, 26, 2, ...
10 [ [ 1272, 1, 10901, 7, 17, ...
An example row with fewer features would be something like this:
[[1,2,3,4],[2,3,4,5],[3,4,5,6]]
The datatype is a string so json.loads has to be used to convert them to arrays that are [N_TIMESTAMPS, N_FEATURES] where each feature is a numerical value.
In order to use this data as input for a neural network I have to convert this column into a numpy array of shape: [N_SAMPLES, N_TIMESTAMPS, N_FEATURES]. So, like this:
[[[1,2,3,4],[2,3,4,5],[3,4,5,6]],[[1,2,3,4],[2,3,4,5],[3,4,5,6]]]
This is how I am doing it now:
train_x = np.array(
df.time_stream.apply(json.loads).apply(np.array).apply(
lambda x: x.reshape(N_TIMESTAMPS,N_FEATURES).tolist()).values.tolist()
)
For a dataset that has 268,521 rows, this computation takes 12.5 mins. Not ideal, but it was working; however, it's not scalable. For the new dataset with 756,961 rows it never finishes (N_TIMESTAMPS = 100; N_FEATURES = 54) because it uses up all of the RAM and the computer crashes.
I'm looking for recommendations for how to make this faster, and perhaps more memory efficient. One issue is that a lot of swap is being used.

Tell flake8 to ignore a few consecutive lines

I'm writing a game which involves a predefined table that looks like this:
SCORES = [
[ 1, 123, 0, 456],
[ 78, 90, -1, 234],
[ 567, 8, 90, 12],
[ 3, 4, 567, 890]
]
To no one's surprise, flake8 gets angry about it. I know I can add # noqa for these lines, but I have several similar tables and they're actually much larger than 4*4. It would be best to have something like
# noqa-start
my_ugly_tables
# noqa-end
def regular_function_that_should_be_checked():
pass
I'm using flake8 3.5.0 on Python 3.6. Any chance?

image preprocessing with python: Lab <-> rgb

I want to write a network which works with images. For it I need to convert rgb to Lab, work with it and convert back: Lab to rgb. At first I wanted to test how it works:
from skimage import io, color
rgb = io.imread(filename)
lab = color.rgb2lab(rgb)
rgb2 = color.lab2rgb(lab)
And I see that all components of rgb2 don't correspond to rgb:
In [5]: arr_rgb
Out[5]:
array([[[ 2.34194446e-17, 3.92156863e-03, 0.00000000e+00],
[ 7.84313725e-03, 1.56862745e-02, 1.17647059e-02],
[ 1.96078431e-02, 2.74509804e-02, 2.35294118e-02],
...,
[ 1.09803922e-01, 1.68627451e-01, 1.56862745e-01],
[ 1.45098039e-01, 2.03921569e-01, 1.92156863e-01],
[ 1.37254902e-01, 1.96078431e-01, 1.84313725e-01]]
In [6]: image
Out[6]:
array([[[ 0, 1, 0],
[ 2, 4, 3],
[ 5, 7, 6],
...,
[ 28, 43, 40],
[ 37, 52, 49],
[ 35, 50, 47]],

Why does my export stop updating when the array it references is sliced?

I've been using node.js for a few years now and generally believed that I understood how exports worked, but an edge case today has left me a bit confused. My code when simplified, behaved like the following:
a.js
var history = []
var i = 0
exports.updateHistory = function(){
history.push(i)
i++
if(history.length > 10){
history = history.slice(5)
}
}
exports.history = history
b.js
var a = require('./a')
setInterval(function(){
a.updateHistory()
console.log(a.history)
}, 200)
I expected the output of this to be the following:
[ 0 ]
[ 0, 1 ]
[ 0, 1, 2 ]
[ 0, 1, 2, 3 ]
[ 0, 1, 2, 3, 4 ]
[ 0, 1, 2, 3, 4, 5 ]
[ 0, 1, 2, 3, 4, 5, 6 ]
[ 0, 1, 2, 3, 4, 5, 6, 7 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
[ 5, 6, 7, 8, 9, 10 ]
[ 5, 6, 7, 8, 9, 10, 11 ]
[ 5, 6, 7, 8, 9, 10, 11, 12 ]
[ 5, 6, 7, 8, 9, 10, 11, 12, 13 ]
[ 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 ]
[ 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ]
[ 10, 11, 12, 13, 14, 15 ]
[ 10, 11, 12, 13, 14, 15, 16 ]
[ 10, 11, 12, 13, 14, 15, 16, 17 ]
...
But instead, this was my output:
[ 0 ]
[ 0, 1 ]
[ 0, 1, 2 ]
[ 0, 1, 2, 3 ]
[ 0, 1, 2, 3, 4 ]
[ 0, 1, 2, 3, 4, 5 ]
[ 0, 1, 2, 3, 4, 5, 6 ]
[ 0, 1, 2, 3, 4, 5, 6, 7 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ]
I was able to solve the problem by adding exports.history = history after it slices each time, implying that the slice call is somehow changing the export's reference, but I have no idea why this would be. Can someone explain this behavior to me?
Thanks
You're only updating the local variable and not the exported value.
Change this:
exports.history = history = history.slice(5)
The reason this is needed is because array.slice() returns a copy of that portion of the array and that is not visible outside of the module (exports.history still references the original array).

Resources