I am interested in understanding the state of the art regarding Quantified Self data sources. Specifically:
What types of data sources should I expect to see from common QS apps?
What types of variations I can expect in common formats (i.e. the GPS Exchange format) Samples of these formats or links to samples of these formats?
What are common methods to access the data as an app builder?
What are common methods to access raw data as a user/consumer?
What are the data access policies for those devices?
What devices matter most (please provide references to data, not mere opinion)?
I know this is a big, multipart question, but answering one part of it without the context of the others will make it difficult to make decisions about what to support and use. I plan on doing this research myself for the most popular apps. I will post my answers here, but I wanted others to be able to comment and extend that content. So if you have another app to add, consider putting it here.
here are the common devices:
Fitbit
Data Type: movement data interpolated into steps/sleep data.
Users can log other types of health data.
Data Variations: none
App access:
OAuth based access to XML and JSON from dev.fitbit.com
User access: data download is a premium feature. Login to fitbit.com (and pay if you have not already) to get download access. http://blog.fitbit.com/?p=360
RunKeeper HealthGraph
Data Type: GPS data from the RunKeeper app, but also can pipe through fitbit withings etc
Users can log other types of health data.
Data Variations: none
App access:
OAuth based access to XML and JSON from dev.fitbit.com
User access: RunKeeper has a data download process for all users that is described here: http://support.runkeeper.com/entries/21696773-how-to-export-your-runkeeper-data
Map My Run
Data Type: GPS running data
Users can log other types of health data.
Data Variations: none
App access:
OAuth based access to XML and JSON from dev.fitbit.com
User access: Can download a GPX with a route but not times: http://support.mapmyfitness.com/mapmyfitness/topics/when_i_export_gpx_file_i_can_not_find_attitude_and_time_information
Related
I am working on onedrive and share point api and i am observing that for file accessed events one drive and share point is giving response with tag DoNotDistributeEvent i wanted to know the importance of that tag
i am trying to filter the events generated by microsoft internally for file accessed based on DoNotDistributeEvent flag in api response
As it says in the documentation for the Microsoft Bot Framework, they have different types of data. One of them is the dialogData, privateConversationData, conversationData and userData.
By default, it seems the userData is/should be prepared to handle the persistency across nodes, however the dialogData should be used for temporary data.
As it says here: https://learn.microsoft.com/en-us/bot-framework/nodejs/bot-builder-nodejs-dialog-waterfall
If the bot is distributed across multiple compute nodes, each step of
the waterfall could be processed by a different node, therefore it's
important to store bot data in the appropriate data bag
So, basically, if I have two nodes, how/why should I used dialogData at all, as I cannot guarantee it will be kept across nodes? It seems that if you have more than one node, you should just use userData.
I've asked the docs team to remove the last portion of the sentence: "therefore it's important to store bot data in the appropriate data bag". It is misleading. The Bot Builder is restful and stateless. Each of the dialogData, privateConversationData, conversationData and userData are stored in the State Service: so any "compute node" will be able to retrieve the data from any of these objects.
Please note: the default Connector State Service is intended only for prototyping, and should not be used with production bots. Please use the Azure Extensions or implement a custom state client.
This blog post might also be helpful: Saving State data with BotBuilder-Azure in Node.js
To start: I've tried Loopback. Loopback is nice but does not allow for relations across multiple REST data services, but rather makes a call to the initial data service and passes query parameters that ask it to perform the joined query.
Before I go reinventing the wheel and writing a massive wrapper around Loopback's loopback-rest-connector, I need to find out if there are any existing libraries or frameworks that already tackle this. My extensive Googling has turned up nothing so far.
In a true microservice environment, there is a service per database.
http://microservices.io/patterns/data/database-per-service.html
From this article:
Implementing queries that join data that is now in multiple databases
is challenging. There are various solutions:
Application-side joins - the application performs the join rather than
the database. For example, a service (or the API gateway) could
retrieve a customer and their orders by first retrieving the customer
from the customer service and then querying the order service to
return the customer’s most recent orders.
Command Query Responsibility Segregation (CQRS) - maintain one or more
materialized views that contain data from multiple services. The views
are kept by services that subscribe to events that each services
publishes when it updates its data. For example, the online store
could implement a query that finds customers in a particular region
and their recent orders by maintaining a view that joins customers and
orders. The view is updated by a service that subscribes to customer
and order events.
EXAMPLE:
I have 2 data microservices:
GET /pets - Returns an object like
{
"name":"ugly",
"type":"dog",
"owner":"chris"
}
and on a completely different microservice....
GET /owners/{OWNER_NAME} - Returns the owner info
{
"owner":"chris",
"address":"under a bridge",
"phone":"123-456-7890"
}
And I have an API-level microservice that is going to call these two data services. This is the microservice that I will be applying this at.
I'd like to be able to establish a model for Pet such that, when I query pet, upon a successful response from GET /pets, it will "join" with owners (send a GET /owners/{OWNERS_NAME} for all responses), and to the user, simply return a list of pets that includes their owner's data.
So GET /pets (maybe something like Pets.find()) would return
{
"name":"ugly",
"type":"dog",
"owner": "chris",
"address": "under a bridge",
"phone": "123-456-7890"
}
Applying any model/domain logic on your API-gateway is bad decision, and considered as bad practice. API Gateway should only do your systems's CAS (with relying onto Auth service which holds the logic), And convert incoming external requests into inner system requests (different headers/ requester payload data) and proxy formatted requests to services for any other work, recieves them, cares about encapsulating errors, and presents every response in proper external form.
Another point, if there is alot of joins between two models required for application core flow (validation/scoping etc) then perhaps you should reconsider to which Business Domain your models/services are bound. If it's same BD perhaps they should be together. Priciples of Domain-Driven-Design helped me to understand where real boundaries between micro-services are.
If you work with loopback (like we are and face same problem we faced - that loopback have no proper join implementation) you can have separate Report/Combined data service, which is only one who can access to all the service databases and does it only for READ purposes - i.e. queries. Provide it with separately set-up read-only wide access to the db - instead of having only one datasource being set up (single database) it should be able to read from all the databases which are in scope of this query-join db user.
Such service should able to generate proper joins with expected output schema from configuration json - like loopback models (thats what I did in same case). Once abstraction is done it's pretty simple to build/add any equery with any complex joins. It's clean, and it's easy to reason about. Also, it's DBA friendly. For me such approach worked well so far.
I want to use the spotify api to create a webapp. Without going into too much detail about the project, I want to clear up whether it would be against the terms and conditions or not.
After reading the terms and conditions, i read this line under things NOT to do: "aggregate Metadata to create data bases, or any other compilations of Metadata".
I don't plan to do any automated requests, for example, hammering the service with different queries to build a database... I'm just wondering whether I can store results from users who have performed searches via my application to the api, so that I can build content from my database on other parts of the application.
Thanks
I'm not a lawyer, so you'll need to have a lawyer confirm this (contracts, including ToS contracts, are important), but the general gist is that if you cache the results of user-generated requests to create features then you're ok. If you start caching stuff not generated by a user, you're in muddy water.
Good:
Other users who searched for "Madonna" in MyAwesomeApp also searched for "Backstreet Boys"!
Bad:
Here's a list of all the blue cover arts on Spotify: [list]
To generate the first example you can cache and work with searches explicitly done by users of your application. The second would require scraping all of the coverart in the service, which isn't allowed.
When I launch a RemoteApp via Remote Desktop Web Access, is there a way to send data to the remote app?
Desired senario:
A user logs into a website with their credentials. They also provide demographic information such as first name, last name, address, etc.
The website connects to the RemoteApp via SSO and makes the demographic information available to the RemoteApp.
For example, if the RemoteApp is a Windows Forms app, can I get this information and display it in a message box?
Edit1: TomTom's response in this question mentions using named pipes to send data. Is that applicable to this problem?
It turns out you can pass command line parameters to the RemoteApp using the remoteapplicationcmdline property like such:
remoteapplicationcmdline:s:/Parameter1: 5234 /Parameter2: true
(The names "/Parameter1" and "/Parameter2" are just examples. Your remote app will have to define and handle these as appropriate.)
This setting is part of the RdpFileContents property of the MsRdpClientShell object.
Here is a resource for other RdpFileContents properties.
Your code might end up looking something like this:
MsRdpClientShell.PublicMode = true;
MsRdpClientShell.RdpFileContents = 'redirectclipboard:i:1 redirectposdevices:i:0 remoteapplicationcmdline:s:/Parameter1: 5234 /Parameter2: true [Other properties here...]';
MsRdpClientShell.Launch();
For larger amounts of information, we might send preliminary data to a web service, retrieve an identifier back, pass this identifier to the RemoteApp via the command line, then have the RemoteApp query the web service to get all the information.
Of course, for the parameters to be of use the program must be looking for them. Setting up a database to query has a little security issue if it is sensitive data.
If the program (RemoteApp) is looking for data in the form of a CSV or table or something, then you might be able to send a lot of data to be processed. It just depends upon what parameters (and form) the program is going to use.