General elasticsearch question:
I get a good response where the _source is formatted as:
{
"Message": {
"S": "This item has changed"
},
"Id": {
"N": "101"
}
}
I want to convert this to
{
"Message": "This item has changed",
"Id": 101,
}
I want to sanitize the response severside so that the response is only what the front end needs and there isn't any additional processing necessary.
So I suppose I can map over the object and create a switch statement for all the different types elasticsearch supports. But is there a "standard" way for approaching this problem? Feels like this would be a pretty common use case and am assuming there might be a better solution out there.
Has anyone run into this before and found a good solution for it?
Related
Please what are the constraints in generating an itemId. I generate unique itemId for each item in the form, but the API keeps telling me invalid ID.
https://developers.google.com/forms/api/reference/rest/v1/forms#Item
Please I need help with this
{
"includeFormInResponse": false,
"requests": [
{
"createItem": {
"item": {
"itemId": "4e637fjc",
"description": "First Name",
"questionItem": {
"question": {
"textQuestion": {
"paragraph": false
},
"required": true
}
}
},
"location": {
"index": 0
}
}
},
{
"createItem": {
"item": {
"itemId": "njyf3izr",
"description": "Middle Name",
"questionItem": {
"question": {
"textQuestion": {
"paragraph": false
},
"required": true
}
}
},
"location": {
"index": 1
}
}
},
}
]
When I had tested Google Forms API before, unless I'm mistaken, I had thought that the rule of item ID might be required to be 00000000 to 7fffffff as the hex value. By the way, for example, 0 is used as 00000000.
When I saw your showing request body, you are trying to use 4e637fjc and njyf3izr as the item ID. In the case of these values, the values are not hex values. I thought that by this, an error like Invalid ID occurred.
But, I think that actually, this is not published in the official document. So, I would like to tell this.
Added:
About your following reply,
Do you mean something like this, with Javascript. crypto.randomBytes(256).toString('hex').slice(0, 8)
From your tag, when you want to use Google Apps Script or Node.js, how about the following sample script? Unfortunately, Google Apps Script cannot directly use "crypto". So, I proposed the following sample script.
Sample script:
const res = Math.floor(Math.random() * parseInt("7FFFFFFF", 16)).toString(16).padStart(8, "0");
console.log(res);
In this sample script, the values of 00000000 to 7fffffff are randomly returned.
Missing documentation
I am afraid that since the Forms API is very new there is no documentation about the specific format the ID should have.
I have done a couple of tests with the API and the only thing I was able to figure out is that the ID needs an 8-character-long string to work, otherwise it would not work or would fill out the blank spaces with zeros.
When doing the testing I was also able to find out that sometimes the API would take a specific pattern of letters and numbers, but when changing the numbers and letters it stops working for no reason.
This seems like missing clarification from the documentation, and I would strongly recommend sending feedback about this problem on the API method page. You can do so by clicking the following option at the top right corner of the documentation:
Google tends to check that feedback a lot when talking about missing information. In addition to all that you can also fill out a report in Google's issue tracker so that they investigate the inconsistencies when using the batchUpdate method to update the ID.
References:
Forms Item
Method: forms.batchUpdate
I've literally researched the entire web and couldn't find a satisfactory answer for this so thought I would ask here.
Basically what I'm trying to do is build a full text search query with pagination, which returns results sorted by time.
The problem is, a naive sort like the following doesn't perform at all:
db.collection
.find({ $text: { $search: "hello" } })
.sort({ created_at: -1 })
.limit(100)
.toArray(function(....
And yes, I've of course indexed it with created_at. And as you can see it's limited to 100 items.
So far what I gather is that the full text index in MongoDB doesn't let you sort by any arbitrary attribute in the collection AT ALL, and the only way to sort it is by adding some $meta attribute to sort it based on some internal scoring mechanism.
But that doesn't work for me, and i really want to sort this by created_at.
Maybe I'm misunderstanding the whole thing, but I refuse to believe that no one has come up with a solution for this very obvious use case. Am I missing something? Does anyone know how to sort a large text search result by a collection attribute? At this point I would appreciate ANY shine of light, even if it's a hack.
[EDIT] For example without the limit and sort, the response would look something like this:
[{
"msg": "hello world",
"created_at": 1000
}, {
"msg": "hello",
"created_at": 899
}, {
"msg": "hello hello",
"created_at": 1003
}, {
...
}]
But I want to limit it to only 100, sorted by created_at, AFTER having searched the database for the occurrrence of "hello". I don't care about relevance and I only want to sort so that it's ordered by time.
[{
"msg": "hello hello",
"created_at": 1003
}, {
"msg": "hello world",
"created_at": 1000
}, {
"msg": "hello",
"created_at": 899
}, {
...
}]
Just to be clear, the query DOES work, but it takes very long time even though I have indexed it with created_at. I don't have this issue when I do a similar find-sort-limit pattern with other queries (not full text search), and I think this is specific to full text search.
I am looking for a way to somehow make this query faster.
I'm writing my first NodeJS app for Google Home (using DialogFlow - formerly API.ai).
I'm looking at the doc on this page: https://developers.google.com/actions/reference/v1/dialogflow-webhook
but I don't see any way to set session variables.
My current test program sets speech like this:
speechText = "I'm not sure that character exists!";
callback(null, {"speech": speechText});
In DialogFlow, my JSON after running looks like this, and it looks like maybe the "contexts" is where the session state would go?
{
"id": "3a66f4d1-830e-48fb-b72d-12711ecb1937",
"timestamp": "2017-11-24T23:03:20.513Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "test word",
"action": "MyAction",
"actionIncomplete": false,
"parameters": {
"WordNumber": "400"
},
"contexts": [],
"metadata": {
"intentId": "a306b829-7c7a-46fb-ae1d-2feb1c309124",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"webhookResponseTime": 752,
"intentName": "MyIntentName"
},
"fulfillment": {
"messages": [{
"type": 0,
"speech": ""
}]
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success",
"webhookTimedOut": false
},
"sessionId": "fe0b7d9d-7a55-45db-9be9-75149ff084fe"
}
I just noticed from a chat bot course that I bought that you can set up Contexts like this, but still not sure exactly how the contexts get set and passed back and forth between the response of one call of my program to the request in the next call of my program (defined via "webhook").
When I added the contexts above, DialogFlow wouldn't recognize my utterance any longer and was giving me the DefaultFallback response. When I remove them, my AWS Lambda get's called.
For starters, the documentation page you're looking at refers to a deprecated version of the API. The page that talks about the current version of the api (v2) is https://developers.google.com/actions/dialogflow/webhook. The deprecated version will only be supported for another 6 months or so.
You're on the right track using Contexts! If you were using Google's actions-on-google node.js library, there would be some additional options - but they all use Contexts under the scenes. (And since they do use Contexts under the scenes - you should make sure you pick Context names that are different from theirs.) You can also use the sessionId and keep track of things in a local data store (such as DynamoDB) indexed against that SessionID. But enough about other options...
A Context consists of three elements:
A name.
A lifetime - for how many messages from the user will this context be sent back to you. (But see below about re-sending contexts.)
An object of key-value strings.
You'll set any contexts in the JSON that you return as an additional parameter named contextOut. This will be an array of contexts. So your response may look something like this:
var speechText = "I'm not sure that character exists!";
var sessionContext = {
name: "session_variables",
lifespan: 5,
parameters: {
"remember": "one",
"something": "two"
}
};
var contextOut = [sessionContext];
var response = {
speech: speechText,
contextOut: context
};
callback(null, response);
This will include a context named "session_variables" that stores two such variables. It will be returned for the next 5 messages sent to your webhook. You can, however, add this to every message you send, and the latest lifetime and parameters will be the ones that are sent back next time.
You'll get these contexts in the JSON sent to you in the result.contexts array.
The "Context" field on the Intent screen is used for an additional purpose in Dialogflow beyond just preserving session information. This indicates that the Intent is only triggered if the specified Context exists (lifetime > 0) when the phrase tries to be matched with it (or when handling a fallback intent). If you're using a webhook, the "Context Out" field is ignored if you send back contexts yourself.
This lets you do things like ask a particular question and set a Context (possibly with parameters) to indicates that some answers should be understood as being replies to the question you just asked.
Content Moderator: https://azure.microsoft.com/en-us/services/cognitive-services/content-moderator/
My picture is taken as a porn picture. How to fix this error?
False Positives Images
add:
There is no error in the code,I use the official interface(https://learn.microsoft.com/en-us/azure/cognitive-services/content-moderator/try-image-api)
Request Body
IsImageAdultClassified:True
IsImageRacyClassified:True
Elchan,
I tried the image and I get the following response. The image was not tagged as racy or adult.
Make sure you select the correct region (same as your API key region) from this page:
https://westus.dev.cognitive.microsoft.com/docs/services/57cf753a3f9b070c105bd2c1/operations/57cf753a3f9b070868a1f66c
My request field (text entry) looks like this: (this is your image)
{
"DataRepresentation":"URL",
"Value":"https://i.stack.imgur.com/NPdxQ.jpg"
}
The response looked like this:
{
"AdultClassificationScore": 0.035331539809703827,
"IsImageAdultClassified": false,
"RacyClassificationScore": 0.043926537036895752,
"IsImageRacyClassified": false,
"AdvancedInfo": [{
"Key": "ImageDownloadTimeInMs",
"Value": "203"
}, {
"Key": "ImageSizeInBytes",
"Value": "9071"
}],
"Result": false,
"Status": {
"Code": 3000,
"Description": "OK",
"Exception": null
},
"TrackingId": "WE_a605a7bfeaca465eb2d8942389f9c86c_ContentModerator.Preview_24006777-6c57-4818-8c2e-146bcb24d3dc"
}
Having said that, you may find false positives or false negatives occurring when you use any machine-learning based API so it's quite possible that you may not get the correct results for other images. In that case, check the confidence scores and figure out a matching threshold that you use to block or pass through the images. (and yes, please send us the image so we can improve the models!)
In addition to auto-pass or auto-reject based on confidence scores, check out the human review tool and Job and Review API of Content Moderator if you have a human moderation team that can further review and change labels assigned by the Content Moderator API.
Let me know if you still have trouble. Please submit your questions via https://cognitive.uservoice.com/.
I am creating one application where for every product I have one database and I will create different document based on date. The keys in documents could be different and depend upon user, what he provides. Assumption is user will keep giving same key for tracking with changed value over time. In the end, I need to know all possible keys before creating automatic views on them.
Example:
If I had DB, say, test. It contains, say, two documents,
1. {
"_id":"1",
"_rev":"1-"
"type": "Note",
"content": "Hello World!"
}
2. {
"_id":"2",
"_rev":"1-"
"type": "Note",
"content": "Beyond Hello World!",
"extra":"Boom"
}
Then I want to list all keys in this DB. So, answer should be _id,_rev,type,content and extra.
These keys are dynamic and depend upon users. So, I couldn't assume that I knew them in advance.
I have never used stackoverflow before, I saw your question when trying to solve this problem myself so I have signed up. I think this solves your problem:
create a view where "views" includes this:
{
"keys": {
"map": "function(doc) { for (var thing in doc) { emit(thing,1); } }",
"reduce": "function(key,values) { return sum(values); }"
}
}
then query on that view with group=true e.g.:
http://localhost:5984/mydb/_design/myview/_view/keys?group=true
you should get back a list of all the keys in your database and a count of how often the occur.
does this help?