I am able to download the data from https://www.investing.com/economic-calendar/cb-consumer-confidence-48 usinf the "Data from web feature"
Is there a way, I can modify the request for Max data retrieval as opposed to only 5 records ?
I tried the data from web to get the index and it only retrieves 5 records.
Related
I have the below external link
https://creator.zohopublic.com/inovadms/inova-dms/view-embed/SPO_Export_Public/M1ZxYdK6k6TYpgZzX7BgHsSQEwSQ2aY27Fx7DHTfCtNgp3tvsUHTajsMxdekudO9hyR9MFe0SgbF8MBngY6UbW5YBmgjWuvk5WfQ/hotelrecid=1721972000009874027&type=1
I would like to connect it in Excel 2013 using web query (with the Power Query it works but I would like to implement it as web query from the Data tab).
In a new excel sheet I am doing the below
1) Open a blank sheet
2) Go to Data-> From Web
3) I add the link and click GO. It returns me the below. I am not sure if this error is the reason of my problem
4) By clicking 'Yes' I have the below (everything fine until now)
But when I click the import button (and select the first cell for this import) it returns an 'Open Page Builder' message. Why this happens?
I did not have this problem until last year and I do not understand what is the issue here.
Zoho Support told me that it is Microsoft issue. Can you please reproduce it and tell me why it does not show me the page in an excel table?
It should return be something like that (without the color format)
It works for me by following this method.
1) Open a blank sheet
2) Go to Data-> From Web
3) I add the link and click GO. It returns me the below
4). I choose Basic option and enter URL and click OK
5). I get the following screen
6). I Choose the second option that is Table. As per my experience data desired by us is contained in Tables.
7). I click Load button to get the data, which loads on web page correctly.
HTH
A). If you choose Advanced option in step 3 then we will get the following dialog box. In this box in upper part we can add a URL by its constituent parts. For example http://community.powerbi.com/t5/Desktop/How-to-setting-a-dynamic-URL-connection/td-p/57072
has been shown entered into its constituent parts in the snapshot shown below. Further details to set a dynamic URL connection can be referred in linked document.
B). In the lower part of the dialog box there is a drop down to select HTTP Request Header Parameters like
Accept Required. Set this to application/json.
Accept-Encoding Optional. Supported encodings are gzip and deflate.
Authorization Required. See authentication.
Connection Optional. It is recommended that Keep-Alive be enabled.
Content-Length Optional. It is recommended taht the request body length be specified when known.
etc....
Further details can referred Query Management HTTP Request
HTTP Request Header Parameters setting for various applications like
Enter data directly into Power BI Desktop
Connect to Excel in Power BI Desktop
Analysis Services multidimensional data in Power BI Desktop
Connect to CSV files in Power BI Desktop
Connect to a Google BigQuery database in Power BI Desktop
Connect to an Impala database in Power BI Desktop
Connect to an OData feed in Power BI Desktop
etc.. can be referred to power-bi/desktop-connect-excel
Edit:
Regarding OP's comment whether same Excel file can run on another PC, my opinion is it is not possible. When try to copy file to temporary storage for transfer, we get a warning that this file can not be copied to new location with its properties. Are you sure you want to copy without its properties. Please see this warning snap shot.
As such it seems that the same query may not work in a new location. New query has to be created for that location.
I am trying to extract some data from this website which refreshes every minute. I have tried researching about web scraping and tried chrome extensions but none seem to work for me.
Some background information about the website: it is a website where people go to monitor bid prices for COE (certificate of entitlement for cars in Singapore). Every alternate Wednesday, from 1430 to 1600, I would have to manually copy and paste the data into an Excel spreadsheet before it refreshes every minute.
Details for COE
I have attached screenshots to illustrate further.
This is the website to scrape; https://www.onemotoring.com.sg/1m/coe/coeDetail.html
You can get a very low cost with AWS Lambda with node-js.
Create a Lambda function and trigger it at your cron schedule you want to crawl the website. You can use library like
https://github.com/bda-research/node-crawler
to simplify crawling.
Also,
To get the exact nodes in the page use serverside jquery or any progressive script that can extract elements from the crawled page.
Once you have the details, you can store them under DynamoDB which is a nosql with very low latency.
You can use,
ODM like https://github.com/clarkie/dynogels to access DynamoDB with very less code.
Hope it helps.
I have a requirement to display SharePoint Documents Sub Grid on the MS crm Dynamics 365 Custom Entity Form.
Can anyone let me know, how to achieve this using Supported or unsupported way?
You can find solution here:
You may use the below line of code to set the Iframe url to display the subgrid in iframe:
Xrm.Page.getControl("IFRAME_Documents").setSrc(Xrm.Page.context.getClientUrl() + "/userdefined/areas.aspx?formid=" + CurrentFormId + "&inlineEdit=1&navItemName=Documents&oId=%7b" + recordId + "%7d&oType=" + oTypeCode + "&pagemode=iframe&rof=true&security=852023&tabSet=areaSPDocuments&theme=Outlook15White");
Where:
“IFRAME_Documents” is the Iframe name.
“CurrentFormId”= Is the Current selected form id which you can get using Xrm.Page.ui.formSelector.getCurrentItem().getId().replace("{", "").replace("}", "");
“recordId” = Record’s GUID
“oTypeCode” = Entity Type Code which you can get using Xrm.Page.context.getQueryStringParameters().etc;
This is how I did in D365. (Using Chrome made it easier)
Enable document management for the entity
When Documents tab shows
for the entity record open it.
Then open Chrome dev console (F12)
Grab the panel (which is actually an iframe) source. The URL is of format:
/userdefined/areas.aspx?appid=...........&formid=.......&inlineEdit=1&navItemName=Documents&oId=.....&oType=.....&pagemode=iframe&rof=true&security=......&tabSet=areaSPDocuments&theme=Outlook15White
Replace the dotted values in the source URL with appropriate values
and make the URL fully qualified prefixing with the root.
Just ensure not using any hardcoded GUIDs or IDs in the values. In this way this can be reused easily wherever needed.
Latest: Recent version has direct OOB customization to achieve this.
Dynamics 365: Related Documents Now Display on Record’s Main Form
Add or remove the SharePoint documents tab to the main form for any table
In short: Server to Server integration approach shows physical documents in crm grid. No need of iframe solution.
Old school List component integration has a limitation, it shows document location crm records in crm grid, so you need iframe solution to show physical docs from Sharepoint.
[Applicable to Server based Integration] - It will show Documents in the grid.
Open an Entity Web form where you want to display SharePoint document library.
Click on the Insert-tab, click on Sub-Grid, specify a name to sub-grid
In “Data Source” section select “Only related Records” from Records dropdown
Select “Document Location (Regarding)” in Entity dropdown
Select “Active Document Location” from Default View section [shown in Below Image]
Click on Set. Click on save then publish the customization
Reference
Update: [Applicable to List component Integration]
Associated view will show what we want but subgrid is not working as expected. Upon research, this is product limitation. Read more
sometimes it is desirable to have an “at a glance” view of documents associated with a record. In order to view the documents in SharePoint that are related to a record, the user must navigate to a related entities area outside of the form. (Similar to Connection, or other related entities)
For most related entities, a sub-grid can be used in the form to display the relationships immediately on the form, but there is no simple workaround for the Documents.
Another alternative is showing Associated view in IFRAME.
EDIT:
In latest Dynamics 365, CRM + Sharepoint integration using List component is deprecated. Only way is Server to Server (S2S) based CRM + Sharepoint integration is possible. This S2S approach shows documents in the associated grid & subgrid, not the doc location like in List component. CRM - Sharepoint wrapper taking care of conversion from CRM FetchXML to SP CAML query & give us the result we want.
The great thing about having the documents queried by CRM is that you
can create custom views of documents in the same way you would with
any other entity in CRM. When using the list component the default
view in SharePoint was rendered in the IFRAME meaning that to get new
columns you had to have list customisation privileges on SharePoint
such that all users would see the changes. With the new server to
server integration you can select SharePoint columns to include in
your own views and even add in your own filters using the CRM advance
find interface.
Read more.
I have created 2 odata queries in excel which fetch data from 2 sharepoint lists. Then I appended the 2 queries to append the data from both list into one. Now i have three queries, Query 1 fro list 1, Query 2 for list 2, and Query append for appended data. I went to connection settings and checked the option to refresh my queries every time the excel file is opened so that I have latest data from my lists.
Query Properties screenshot.
I saved the file and uplaoded on sharepoint library. Now whenever i open the file from sharepoint in excel online I get an error "External data refresh failed".error message.
The data doesnt get refreshed if i press ok.
Strangely this problem does not comes if I download the excel file and open it locally and refresh the data.
You need to create a connection in the Secure Store and tie the credentials to later be used by your Excel spreadsheet.
Once you supply the user account and password in the Secure Store, you will need to get back to your Excel spreadsheet and change the authentication settings to match the one from the Secure Store, enter it on the Application ID.
I'm working with Power Query in Excel 2016 and when trying to setup an oData Feed (Data Tab > New Query > From Other Sources > From OData Feed) I get a message back saying (406) Not Acceptable. Normally on the OData Feed, I put in the URL (Basic is selected by default) and it opens up an authentication window (to put in username and password).
I have two excel workbooks now with one being created about four weeks ago, using the same process and it works as expected; one can even select refresh data and it makes the connection and refreshes the data.
The second excel workbook is a new, created from scratch (as I wanted to go through a bit of a different path with the Power Query editor), but I cannot get past the OData feed creation.
It seems odd that the same URL that was setup before is not working in a new spreadsheet.
It looks like it was purely an issue with selecting oData as the feed. I'm connecting to JIRA rest web services and should have selected (Data Tab > New Query > From Other Sources > From Web) (the From Web over From OData Feed made all the difference)