I've created a classification endpoint using Azure ML, the input for which is a database query to retrieve the database row to classify.
When I run my experiment in the Machine Learning Studio, it works and connects properly to my database. When I submit the same query as a web service parameter on the import data module, I get the following error:
Ignoring the dangers of an SQL query as input, why am I getting this? Shouldn't it work the same?
Sidenote: I've used an SQL query on my training endpoint in the exact same way on the same database, and this didn't cause any problems.
UPDATE: It seems as if this is only a problem when I create a new endpoint for a service. If I use the default endpoint it does indeed work, but any new endpoints do not.
UPDATE 2: It also works when I submit my request as a batch run. If I use Request-Response, it fails.
According to Microsoft this is a known bug and they are working on it. Another workaround, although NOT recommended, is to pass the password in as a web service parameter (but may be ok for test/proof of concept apps)
Here is a link to the thread on the MS forums that states this is a bug.
Related
I am attempting to write an Azure CosmosDB integration (Core SQL Api) that integrates with an external service to provide some of the query data. As an example, I need a query made on Cosmos DB to convert some of the data returned (e.g. ID's) by the query into real data by calling an external service via a REST API. This should only happen when querying certain columns.
I initially investigated using a JS stored procedure and/or a UDF to make this external call, but the JS environment seems to be extremely limited and doesn't provide any way to make external calls. I then tried using this https://github.com/Oblarg/cosmosdb-storedprocs-ts repository, which uses webpack to bundle all of node.js into the stored procedure, allowing node modules to be used in stored procedures. Whilst this does allow some node modules to be used, whenever I try and use "https", "fetch", or "axios" modules to make an HTTP GET request I get errors (the same code works fine in a normal node environment, but I'm not a JS expert and can't seem to work past these errors). After a day of attempts it seems like the stored procedure approach is not possible.
Is this the case or is there some way of making HTTP GET requests from a JS stored procedure? If not possible with stored procedures, are there any other techniques to achieve the requirement of reading data from a remote API when querying cosmos DB?
Thanks
There is no way to achieve this from CosmosDB directly, for queries you also cannot use the change feed as the document dont change, so really your only option is to use a function or some preprocessor app to handle it, as you say its not ideal but there is no other solution here. If it was an insert or an update then change feed would allow you to do this but for plain queries its not possible.
Let me start out by saying I am a SQL Server Database expert, not a coder so making API calls is certainly not an everyday task for me.
Having said that, I am trying to use the Azure Data Factory's data copy tool to import data from Clio to an Azure SQL Server database. I have had some limited success, data is copied over using the API and inserted into the target table but paging really seems to be an issue. I am testing this with the billable_clients call and the first 25 records with the fields I specify are inserted along with the paging record. As I understand, the billable_clients call is eligible for bulk actions which may be the solution, although I've not been able to figure out how it works. The url I am calling is below:
https://app.clio.com/api/v4/billable_clients.json?fields=id,unbilled_hours,name
Using Postman I've tried to make the same call while adding X-BULK true to the header but that returns no results. If there is anyone that can shed some light on how the X-BULK header flag is used when making a call, or if anyone has any experience loading Clio data into a SQL Server database I'd love some feedback on your methods.
If any additional information regarding my attempts or setup would help please let me know.
Thanks!
you need to download the json files with Bulk API and then update them in DB.
It isn't possible to directly insert the data
I've been working on a Xamarin.Forms application in Visual Studio using Azure for the backend for a while now, and I've come across a really strange issue.
Please note, that I am following the methods mentioned in this blog
For some strange reason the PullAsync() method seems to have some bizarre problems. Any data that I create and sync will only be pulled by PullAsync() from that solution. What I mean by that is that if I create another solution that accesses the exact same backend, it can perform it's own create/sync data, but will not bring over the data generated by the other solution, even though they both seem to have the exact same access. This appears to be some kind of a security feature/issue, but I can't quite make sense of it.
Has anyone else encountered this at all? Was there a work-around at all? This could potentially cause problems down the road if I were to ever want to create another solution that accesses the same system/data for whatever reason.
For some strange reason the PullAsync() method seems to have some bizarre problems. Any data that I create and sync will only be pulled by PullAsync() from that solution.
According to your provided tutorial, I found that the related PullAsync is using Incremental Sync.
await coffeeTable.PullAsync("allCoffees", coffeeTable.CreateQuery());
Incremental Sync:
the first parameter to the pull operation is a query name that is used only on the client. If you use a non-null query name, the Azure Mobile SDK performs an incremental sync. Each time a pull operation returns a set of results, the latest updatedAt timestamp from that result set is stored in the SDK local system tables. Subsequent pull operations retrieve only records after that timestamp.
Here is my test, you could refer to it for a better understanding of Incremental Sync:
Client : await todoTable.PullAsync("todoItems-02", todoTable.CreateQuery());
The client SDK would check if there has a record with the id equals deltaToken|{table-name}|{query-id} from the __config table of your SQLite local store.
If there has no record, then the SDK would send a request as following for pulling your records:
https://{your-mobileapp-name}.azurewebsites.net/tables/TodoItem?$filter=(updatedAt%20ge%20datetimeoffset'1970-01-01T00%3A00%3A00.0000000%2B00%3A00')&$orderby=updatedAt&$skip=0&$top=50&__includeDeleted=true
Note: the $filter would be set as (updatedAt ge datetimeoffset'1970-01-01T00:00:00.0000000+00:00')
While there has a record, then the SDK would pick up the value as the latest updatedAt timestamp and send the request as follows:
https://{your-mobileapp-name}.azurewebsites.net/tables/TodoItem?$filter=(updatedAt%20ge%20datetimeoffset'2017-06-26T02%3A44%3A25.3940000%2B00%3A00')&$orderby=updatedAt&$skip=0&$top=50&__includeDeleted=true
Per my understanding, if you handle the same logical query with the same query id (non-null) in different mobile client, you need to make sure the local db is newly created by each client. Also, if you want to opt out of incremental sync, pass null as the query ID. In this case, all records are retrieved on every call to PullAsync, which is potentially inefficient. For more details, you could refer to How offline synchronization works.
Additionally, you could leverage fiddler for capturing the network traces when you invoke the PullAsync, in order to troubleshoot your issue.
I have an IIS, on an azure VM, that host a web application that I use to retrive data through SOAP request. Everything works fine if I access from localhost but as soon as I access to some functions from outside I get a response formally correct but with no data in it.
Configuring different VM with the same services I had the same results. I changed <httpRuntime maxRequestLength="1024000" /> because it seems the problem was related to the amount of data retrived but I had no positive result. It isn't a timeout problem because other functions return values even after 30 seconds.
Is there anything else I might I have done wrong, or that I need to configure to allow SOAP requests from my Azure web application? Where can I look for better diagnostics to see what's going wrong?
Configuring on the target VM a development environment I discovered the problem was releted to the differences of Culture set on sever and on Client.
This caused a transformation of parameters sent to sql server for data extraction.
I had to change the code to become Culture invariant.
I have created a .net web service and when i try to call a method that saves the data in the database, the request is fired twice. I use net profiler to check if two requests are made to the server but only one request is made to server. I fail to understand why data is being entered twice in the database.I am using jsonp method to call the cross domain site
I just found something interesting. I have two servers. When i host the web service on one and call the web service using cross domaining, the data is entered once whereas on the other the data is entered twice. Do we need to take care of some IIS settings too?
So, if there aren't two requests being made then there are almost certainly two calls to the Save() method (or whatever it is called), being fired from the web service end point. But there maybe dupliacte data somewhere too.
Here are a couple of things to check:
What data is actually being
transferred? Have you checked this
using a tool like Charles?
Is the data being passed to your Save() method the same as the data being passed to the web service?
How is the data being written to the database? Is there duplicate SQL somewhere?
Heyi all, i just converted my project to visual studio 2010 and then installed it on the server. Everything is running perfectly now. Thanks