In a project I am running an AWS Sagemaker Jupyter notebook instance that heavily interacts with files (gathering, converting, computing, interacting) and after every step the files are moved from folder to folder to prepare for the next interaction. I was wondering if there was any way to set some form of chart (like excel) that creates/updates a row when a file enters a folder. The charts end goal is to be used as some form of tracker, to see what stage all the different files are in.
Examples of how the desired chart should look like below
Chart Style 1
Chart Style 2
One way in which you could achieve this is by pushing the data programmatically to a central Google Spreadsheet and use the data from there to create charts.
You could use the GSpread Library in Python to push the data after certain steps in your code are run successfully. We use this extensively to push data to Google Sheets every day.
You would need to check the API rate limits enforced by Google Cloud which can be increased as per your requirements.
Related
Good Day!
I am struggling with the best way to approach this via Power Automate.
Desired Flow: This will be a scheduled flow that would trigger every 10 minutes. Cycle through a SharePoint List that contains a list of subsites within my enterprise along with the path to a specific folder in a specific library. Using the 'Apply to Each' function, identify any NEW files since the last sweep. Copy those files to a different directory in SharePoint (details of which are also included in that same SharePoint list).
I have got all the pieces to this EXCEPT: How to identify the new files since the last sweep.
Can't use the "when a new file is created" trigger, as this listing has close to 300 subsites on it....which would mean 300 different flows.
Appreciate any guidance!
UPDATE:
Getting an error when using the Get Files. Here is the snippet of the flow:
Snippet of the Flow Where error is occuring
Snippet of the SharePoint List
I would use a Get Files (properties only) action with a Filter Query which uses an Created greater than 10 minutes ago.
Below is an example.
Created ge '#{addMinutes(utcNow(), -10)}'
This is for the second part of your question.
After some months I could say I am getting the hang of Microsoft Flow, however I could use some help with the following issue:
In a flow for reporting purposes, a temporary file (.xlsx) is created in a sharepoint folder by means of a template. This temporary file is then filled with rows and info from other sources. So far so good.
I use the body of this newly created and furnished file as an attachment for an e-mail to the chief. However, the attachment came out identical to the (empty) template file, without the rows and furnishing.
Adding a delay of two minutes before attaching and sending the mail solved it for relatively small reports, but this is not ideal as I want it to work regardless of file size. Furthermore I do not understand why it would send an empty (old) version of the temp. file in the first place, as all the furnishing operations should have executed before copying and attaching (the flow is entirely in series).
Sorry for the long story. Does anyone have a more elegant solution than using a Delay-node?
I'm fairly noob on the whole RESTful/SOAP arena, I've been looking around for samples using node.js to trigger a Rest or SOAP API to be able to export a batch of smartsheets to excel
I already got a sample for google drive export but its not quite what I need
I'm not sure how to search for code or samples using microsoft excel export and then batch from a list of files or even a smartsheets workspace
I expect a folder with all the exported excel files in the server at a specified location. I'm also thinking of how this could be done using node-red instead as an alternative, what is the suggested path of least resistance.
You can do a GET Sheet request via the Node.js SDK and have the results returned as an Excel file:
https://smartsheet-platform.github.io/api-docs/?javascript#get-sheet-as-excel-pdf-csv
Note, this requires the Sheet ID and is done as a request for each sheet. There isn't a method to request a batch export of a collection of sheets via the API. You would need to gather a list of Sheet IDs and loop through it to request each one as an Excel file while storing them in your desired location.
You could do a GET Workspace request to get a list of all of the sheets in a workspace for creating your list of Sheet IDs. You could use the List Sheets request to get a list of all of the sheets you have access to as well. Both of those requests are supported by the Node.js SDK.
Log data from a test is uploaded to a web service, and the processed CSV is downloaded back into Excel for viewing in charts. At the moment, this is done via copy and paste for short CSV files and the Data > From Text feature for larger CSV files. Unfortunately, this takes a bunch of time for every test, and I need to make the process very simple for someone else to update the Excel spreadsheet.
The Excel spreadsheet contains 5 raw-data pages which are used to store the CSV from the server. I have no issues selecting Data > From Text, entering the website URL, and completing the format to import. This process can be repeated (same as the Copy and Paste) for all 5 pages to import the data.
This process only allows me to put in one filename, so I am using the same URL for the data, and having PHP return the CSV of the latest (or a specifically configured) test whenever the website is accessed. I've verified that this process is working correctly.
Unfortunately, when I do 'Refresh All', it prompts for a filename unless I go to Data > Connections > Properties, and uncheck 'Prompt for file name on refresh'.
However, even when I do that, I'm getting mixed results. Sometimes only one of the pages will update. (Seems to be the last one I set up.) Sometimes none of them do. I need a solution which updates all 5 pages based on the current CSV from the server without having to set up the connections again every time. Ideally I'd like to just hide these raw data sheets so we can have an Excel file that's just the final charts.
Surely this is a common function and I am doing something wrong, yet all the guides I try on the Internet don't seem to work. For example, this one:
http://www.kimgentes.com/worshiptech-web-tools-page/2010/8/18/web-connecting-csv-files-as-external-data-to-excel-spreadshe.html [URL is corrected]
Seems like they only set up one connection. I can get one working to refresh, but not more than one.
I have seen this happen and finally figured it out. There are actually 3 things that can happen to give this result, and a separate solution for each:
First, Excel software uses the IE 11 web object to when it does web
retrieval of data. This means it will be "sticky" to sessions using
IE11 to access the data. Most websites these days are run by cloud
servers, which generate sessions on the server with the most load.
This normally has no impact on users on web browsers since they
login and can visually enter their credentials etc. But when a
program accesses a website and must use a specific web browser, it
must use the properties of that browser and how it works. I ran into
this a lot when I would generate and be able to download my CSV
files on the website in Chrome, then try to use Excel to import the
same files wouldn't work (it would say they weren't there). The
solution to this, at least for now, is to use IE 11, login to the
website, generate the CSV files and test that they can be
downloaded. Then use Excel to run the web import and it should pick
up the same sticky session to get the CSV files.
Second, password entry is a different thing, but also has to do with the stickiness
of the data. For some reason Excel will not cache your credential
responses for logging into a website without you entering them 3
times. This experience may change for you, but I found that I must
enter a new credential set (for a new web import of a CSV) 3 times
before it becomes permanently cached by Excel. After this, I don't
have the problem.
Third, most complex Excel programs that require
web import may also require that you either import local data you
downloaded from a website, import data from a website into a sheet
or run more complex objects like Macros. All of these need proper
permissions. You may need to set your Trust Center settings to allow
you to use your Excel program on your computer in this way. That is
part of MS office. You can set add and update those as per MS info
here:
https://support.microsoft.com/en-us/office/add-remove-or-change-a-trusted-location-7ee1cdc2-483e-4cbb-bcb3-4e7c67147fb4
Each month I have to manually create a report for my team. For this I use a combination of numbers generated from a groovy script I've written, screenshot, cut & paste from a series of charts in our JIRA instance and cut and pasted excel charts.
I'm finding it more and more boring to do this by hand and I suspect that there is an easier way to automatically generate this report. I can generate the word document with data from the groovy script using this application (http://www.docmosis.com/) but I have not yet found a way of auto including JIRA charts in the generation or excel so my question is:
Is it possible to generate a word document that contains generated data from various sources including (in order of importance):
JIRA charts
Programmatically generate data and
Excel charts?
As you already coded your logic in Groovy, best would be skipping the Excel step and generate your report directly from JIRA using the JIRA PDF View Plugin. This plugin will reduce your work to a single click and download you a nice PDF document (PDF is more portable than Word docs).
These short tutorials help you:
Reuse your Groovy scripts for reporting logic (also see the "Charts" tutorial there)
Bring the data, charts and the template together
Update 2 years after my original answer:
JIRA PDF View Plugin 3.5.0+ can also export JIRA charts (gadgets in JIRA dashboards, to be precise). So, now you can execute your Groovy scripts, include JIRA charts and bring all the result together in a professional PDF document.
Even better, now there exists a free plugin that will make the monthly process 100% automatic for you! It will generate your PDF document and send that to your team members' mailboxes, according to the preferred schedule (CRON trigger).
Related tutorials:
Exporting JIRA dashboards to PDF
Automating JIRA PDF exports
Discl: I'm a developer working on these products, yet this is the best solution to your problem. :-)
Most of the charts generated in Jira are created from data returned from search results. So you can get data from Jira using REST or SOAP api and generate your own charts by other means.
I am not sure if there is a no-coding method of doing it. But I would write a C# program that adds data to the Word template from your sources (Jira, other programs, Excel). And would create a template that generated charts from inserted data.
For Word document generation I would have a look here -http://www.codeproject.com/Articles/20287/Generating-Word-Reports-Documents
For chart creation in the document I would have a look here - http://office.microsoft.com/en-us/word-help/create-a-chart-in-a-document-HP005190046.aspx
Easiest if you convert your JIRA chart to an image first. When you say JIRA chart, do you mean something from this plugin?
Once you have an image, your document generation tool ought to be able to include that.
There are many document generation tools which can include programmatically generated data, and some of those can include Excel charts.
There is now a JIRA plugin to generate charts in Word documents which may help you.
https://marketplace.atlassian.com/plugins/com.clariostechnology.officecharts.officecharts
There is now also Intelligent Reporter for JIRA which is a no-code method for creating word documents from your JIRA data, including native Word charts with full formatting from JIRA data.
https://marketplace.atlassian.com/plugins/com.clariostechnology.intelligentreports
Disclaimer: I work for the company that created this plugin. If there is anything you need that it cannot do, please let me know.