How to avoid reloading data in each run - python-3.x

How can I avoid reloading data each time when I want to check if my script works? I work with spyder and python.
I have to load around 1000 .csv files, it takes just a few seconds but it is unnecessary to repeat this step each time I change a parameter or a name somewhere else in the code.
One simple option is to comment a part of code. What would be a more efficient way?

(Spyder maintainer here) You can use cells in Spyder, which are blocks of code delimited by comments of the form #%%. You can evaluate cells with keyboard shortcuts Shift+Enter (run and advance to the next cell) or Ctrl+Enter (run and stay in the same cell).
This way you can have a cell to load your files and then another cell to do you computations with them, like this
#%%
data = load_my_files('my-dir')
#%%
compute(data)

Related

ExcelScript - run on cell selection / editing (bind?)

How can I run a piece of code on a cell selection? And how can I run it again, when I start editing the value in the cell?
I been going through the documentation, and in no way I can find anything (a specific binding???) to achieve the desired result.
You can use workbook.getActiveCell() to run a piece of code on a cell selection, but the second part to your question is not possible as Office Scripts does not support events that trigger scripts to run.

How to record data automatically on excel that is generated on Matlab

I am new to MATLAB and I currently have a script that generates different values for a variable n every seconds. So I end up with 100s of data that needs to be transferred to excel. Currently, I do this manually by copying pasting each but it does take really long.
I thought about using xlswrite command but that just writes data on the first column and keeps overwriting that column as new data is generated.
Would you be able to help me or lead me to easier way with this?
If you really want to write directly to excel while your code runs, you can use activex. Some well documented example code can be found in this answer. Somewhere in the middle you find this code:
% select a 5x5 range, and fill it with some numeric values
sheet.Range('A1:E5').Value = num2cell(magic(5));
Here you have replace it with a loop which continuously writes the data.
Be careful when using this solution.
Problems with Excel can stop your MATLAB process
Especially with large data it will be much slower than post-processing
You are forced to have Excel and MATLAB on the same PC installed
Using a simple text file which is continuously written might be the better choice, either fprintf(use the same file handle and it will append) or dlmwrite(use append option).

How big is too big?

I have a 60mb excel file that has all of the "sins"... Array formulas, VBA and conditional formats. It crashes all the time and it's now being suggested that it's too big. There are about 20 worksheets and 3 of them are 15mb in size and each have about 35,000,000 active cells. Is this why the sheet crashes?
Yea I have this problem also.
This is a compilation of hat works for me. Some of this may be relevant, some may not.
Turn off automatic calculation. If you are desperate about this, then turn it off while doing the rest of the list but turn on afterwards
Make a backup. Everytime something on this list works, make a new backup. Excel has no brain, make sure to use your own.
Remove empty-but-not-empty cells... Sometimes, for a number of reasons, excel finds it necessary to keep track of all possible rows and columns.
I fix this by recreating whatever chart I have made by copying the specific areas that are needed on new sheets and then delete the old one and rename the new one to match. Search/replace will fix formulas afterwards if they get buggered. You can also mark the rows/cols and right-click -> delete rows/cols but that may cause a total freeze so I prefer the former method.
Get rid of the array formulas. Seriously, they can in 90-something % of the time be replaced by a cleverer sumproduct, sumifs or index(match()) formula or by adding columns or doing things in 2 steps. They are resource hogs. Leftovers from a different age. The only time I accept arrays is when I know I have exhausted all possibilities.
Very important, save the files as x64 binary formats for the files (.xlsb). Do not use the "compatibility" format (.xlsx) and for the love of god, do not use regular .xls
Move all graphics to separate sheets, and better yet, separate files. Mirror the result data via vba, copy paste, whatever to separate files and have your conditional formatting, images, graphs, progress bars, gantts, whatever there.
I hope it's not inappropriate to post this.
After some stalling and perhaps some skepticism, I replaced all of the array formulas in my sheet. There were approximately 1,500,000 cells affected by array formulas and I didn't relish the thought of redoing this. BUT WHAT AN UNBELIEVABLE DIFFERENCE! The sheet now runs faster, better and very reliably! I'll never use an array formula again. Thank you!!

Excel cell content into multiple rows into another sheet

I have a requirement in excel where i want to copy each text that ends with pulistop into a row of another sheet. ex:
Setting up the configuration.
Creating environment.
Pushing the tasks.
Now assume that above text is in one cell and i want to copy each sentence that ends with pulistop into individual rows of a new sheet. like below.
Setting up the configuration.
Creating environment.
Pushing the tasks.
Please help me in doing this as i have many no. of sheets which needs this modification.
Thanks in advance.
Chakri.
Check if this can be useful for you:
Copy the cell(s) into clipboard (Ctrl+C).
Paste into Notepad or similar text editor. Note: in Notepad the lines will appear next to each other, but nevermind, the line jumps are still there.
Replace in text editor all " (double quotes) with nothing.
Copy the whole text of the editor.
Paste into Excel.
This works in my case. If you have many cells like this next to each other, with this method you can process all at once with the same effort. Will this work for you?

increment pasted number keyboard

Probably this question is absurd for you and i don't know if there is solution.
What i would like to do is increment the number that i have copied with control-c after each control-v that i do..
For example if i copy with control-c the number 10 from a site, then when i paste with control-v in another program the number pasted should be 11 etc..
Is there a way to do this???
Because i have to insert about 2000 number in a program and in this specific case is not possible to do script in order to automate the work..
I hope in some answer
Thanks
I can't think of a great way to do it.
What comes immediate to mind is you can open Excel and have two cells, let's call them incoming and outgoing. You paste into incoming, and outgoing is setup to be incoming + 1, which you can copy and paste into your program. So you double the number of copies and pastes you have to make but it won't be the end of the world. It sure beats manually typing.
It's possible, using a small (custom written) program that would run in the background. It would have to look at the contents of the clipboard, test if it's numeric, and if so, increment the value, if it's not already been incremented. Shouldn't be all that hard for a windows programmer worth their salt to cook up.
I did this in excel when i needed it multiple views. row() function enters current row number. Copy this cell for as many times as you want one below other and then copy paste in android studio.
=CONCATENATE("dateBoxView"|ROW()|" = fragmentView.findViewById(R.id.dateBox"|ROW()|");")
You can do this using something like Pulover's Macro Creator. You press a hotkey to copy a number. Everytime you press ctrl-v a the copied number gets +1 and that new number copied to clipboard for the next paste. When you want to stop doing this just stop the macro.

Resources