I am developing a JAMStack project using keystonejs 6 and nuxtjs. As we know, keystonejs uses prisma and graphql (I guess apollo) to serve CRUD (generic) actions on our data model. So it makes the apis (even graphql or rest) and the errors generated are also generated by keystonejs. Now the problem is when the client send a wrong request to keystonjs, then it will help the client to correct the query and gives hints on it. For example if I query the next request:
await fetch("https://host:port/api/graphql", {
"credentials": "include",
"headers": {
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0",
"Accept": "*/*",
"Accept-Language": "en-US,en;q=0.5",
"Content-Type": "application/json",
"Sec-Fetch-Dest": "empty",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "same-site"
},
"referrer": "https://host:port/",
"body": "{\"operationName\":null,\"variables\":{},\"query\":\"{\\n navigationMenus(where: {status: {not: {equals: \\\"invisible\\\"}}}) {\\n ida\\n title\\n pageURL\\n childrenCount\\n parent {\\n id\\n pageURL\\n __typename\\n }\\n __typename\\n }\\n}\\n\"}",
"method": "POST",
"mode": "cors"
});
as the field ida is not valid, is returns a bad request error with status code 400 (which is ok) and also helps the client with the following response:
errors [ {…} ]
0 Object { message: "Cannot query field \"ida\" on type \"NavigationMenu\". Did you mean \"id\"?", locations: […], extensions: {…} }
message "Cannot query field \"ida\" on type \"NavigationMenu\". Did you mean \"id\"?"
locations [ {…} ]
0 Object { line: 3, column: 5 }
line 3
column 5
extensions Object { code: "GRAPHQL_VALIDATION_FAILED" }
code "GRAPHQL_VALIDATION_FAILED"
So this will lead to some vulnerabilities since if I forget to make currect access controls on some fields, attacker may find field names with some tries and access to those data.
Although I can understand that this is not a very big issue, but is there a way to turn graphql hints off on production?
Keystone is build on top of Apollo and lets you pass additional config though to the Apollo server on start via the graphql.apolloConfig config option:
graphql.apolloConfig (default: undefined): Allows you to pass extra options into the ApolloServer constructor.
– Keystone 6: System Configuration API docs
This can be used to modify the Apollo error handling behaviour and replace it with your own. A simple example may look like this:
import { config } from '#keystone-next/keystone';
import { lists } from './schema';
import { ValidationError } from 'apollo-server-express';
import { GraphQLError } from 'graphql';
export default config({
db: {
provider: 'sqlite',
url: process.env.DATABASE_URL,
},
lists,
graphql: {
apolloConfig: {
formatError: (error: GraphQLError): any => {
if (error instanceof ValidationError) {
return new ValidationError('Invalid request.');
}
}
}
}
});
Any GraphQL queries that attempt to access non-existent field will then receive the response:
{
"errors": [
{
"message": "Invalid request.",
"extensions": {
"code": "GRAPHQL_VALIDATION_FAILED"
}
}
],
"data": null
}
The general problem of Apollo leaking data in this way is being tracked in the Apollo repo, eg: Validation errors can be used to get schema details #2247. It sounds like progress is being made.
I'm struggling in using the stackdriver logging api (googleapi.logging.entries.write) to log errors and information in the GCP project.
I'm using a service account (used also for more than 20 APIs without issue).
The scope https://www.googleapis.com/auth/logging.admin has been set to the service account, as well as the role "logging admin" in the GCP IAM console.
MoreOver, I have added a superadmin user in the impersonate mode ("subject" in the JWT options)
However, I still have the error "The caller does not have permission" while calling the logging API, although the oAuthClient has been well set.
What additional role or scope or permission to add to make this running ?
Please note that there is a lack of example and documentation to use the logging api, which makes things confusing, especially where to set the auth, since it appears in 2 different places. It should be set in the logging({}) options, and/or in the entries.write({}) options ? I tried all combinations, all of them failed with the same authorization error.
Thanks for your help
import { google } from "googleapis";
const t = await google.logging({ version: "v2", auth }).entries.write({
auth,
requestBody: {
logName: "projects/THEPROJECT/logs/THELOGS",
resource: {
type: "project",
labels: {
projectId: "THEPROJECT",
},
},
entries: [
{
severity: "INFO",
jsonPayload: {
source: "code",
value: "test",
details: "this is INFO log test",
},
},
],
},
});
console.log(t.status);
Problem
I am developing a Actions on Google app (intended for the Google Home), using the Actions on Google library for Node.js, hosted on Firebase Google Functions, via Dialogflow. I often, but irregularly (and hard to replicate), encounter an error, forcing the Actions on Google (simulator or Google Home itself) to shut down my app. I route everything from any Dialogflow intent (including fallbacks) to my webhook fulfilment, and - based on the log files - the webhook responds quickly (within ~200ms) and with valid responses (investigating the JSON responses). However, the Actions on Google seems to reject the response and triggers the Dialogflow default text response. My biggest concern, is that it happens at different stages in the conversation, sometimes already at the Welcome event. It is also noticeable that - even though the fulfillment responds in milliseconds (~200), the Actions on Google / Dialogflow takes its time and - what I believe - has a timeout. Below are my explorations into the potential cause. But frankly, I am out of ideas.
-- Edit --
The service seems to run better now - I haven't experience this error. I have changed the code to call admin.firestore() less often, by using a global databse and passing it through. I had a hunch that, potentially the https functions were called simultaneously which might have caused a malformed response somehow.
const database = admin.firestore();
// code
if (!process.env.FUNCTION_NAME || process.env.FUNCTION_NAME === 'functionname') {
exports.functionname = functions.https.onRequest((req, res) => {
require('./functions').functionname(req, res, database);
});
}
I also found a flaw in my intent handeling, which caused no intent to be matched. I've restructured the Dialogflow intents to have less possibility of this no-intent (I basically made a two-step question to determine the user's intention). However, since the webhook did response correctly, I don't believe this was the problem. Still, I find it an odd error - so if anyone has more pointers please!
-- end edit --
Logs
This is a response from the webhook as seen from the Google Functions Log - indicating a send response to the Dialogflow app.
Function execution took 289 ms, finished with status code: 200
Response {
"status": 200,
"headers": {
"content-type": "application/json;charset=utf-8"
},
"body": {
"payload": {
"google": {
"expectUserResponse": true,
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "Welcome back! Which Widget do you want to work with?"
}
}]
}
}
},
"outputContexts": [{
"name": "****anonymized****/contexts/widget",
"lifespanCount": 1
},
{
"name": "***anonymized****/contexts/_actions_on_google",
"lifespanCount": 99,
"parameters": {
"data": "{\"started\":1552071670}"
}
}
]
}
}
However, this is what the Actions on Google console shows. The textToSpeech response is the default text response where DialogFlow can fall back to in case the fulfilment fails (as suggested by https://medium.com/google-developers/debugging-common-actions-on-google-errors-7c8527378d27)
{
"conversationToken": "[]",
"finalResponse": {
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Sorry! I cannot access my online service. Please try again!"
}
}
]
}
},
"responseMetadata": {
"status": {
"code": 14,
"message": "Webhook error (206)"
},
"queryMatchInfo": {
"queryMatched": true,
"intent": "e0bf4b96-9440-4545-a8a6-d0915cacd34f"
}
}
}
The stackdriver logs also indicate the latter case, where the default response from Dialogflow is received.
Thoughts
I have tried to replicate this error on the simulator, on my phone and on a Google Home. It occurs at different moments, though seems to occur more frequently than before. My three hunches are the following:
Overload. My Google Functions hosts 8 functions, including the Dialogflow app. Perhaps if the other functions are also called frequently at a given moment, something goes wrong with the Dialogflow app handeling the https requests from Dialogflow (the Google Assistant). However, I do not see how this should happen. I could replicate (sometimes) the problem by invoking the app on two devices with two different accounts. But since it also happens with one account I don't see this can be the only problem. I've followed this tip to reduce overload (https://stackoverflow.com/a/47985480/7053198) but that did not seem to help my issue. Can then the upscaling of Google Functions be a problem here?
Promise hell. I am using a number of promises (or more callbacks) to communicate with a Firestore database to retreive user data. However, I can invoke these intents quite rapidly, and they resolve nicely - up to a point where they don't. Below is a snippet of such one intent.
Size of my app. The number of Intents has gotten quite large, and there is a lot of communication between the Google Functions and the Firestore database. I just don't know how this might influence the rejection of the seemingly correct response by Dialogflow.
Intent with promises snippet:
app.intent(I.WIDGETTEST, conv => {
let database = admin.firestore();
let offline = [];
return database.collection('users').doc(conv.user.storage.userId).collection('widgets').get()
.then((snapshot) => {
const promises = [];
snapshot.forEach(doc => {
if (doc.data().active) {
if ((moment().unix()-doc.data().server.seen) > 360){
offline.push(doc.data().name);
}
promises.push(doc.ref.update({'todo.test': true}));
}
});
return Promise.all(promises);
})
.then(()=>{
conv.contexts.set(C.WIDGET, 1);
return conv.ask("Testing in progress. Which Widget do you want to work with now?");
})
.catch((err) => {
console.log(err)
throw err;
});
});
Problem Example Log
For good measure, here three log entries from the stackdriver side of things:
{
"textPayload": "Sending request with post data: {\"user\":{\"userId\":\"***anonymized***\",\"locale\":\"en-US\",\"lastSeen\":\"2019-03-08T18:54:47Z\",\"userStorage\":\"{\\\"data\\\":{\\\"userId\\\":\\\"***anonymized***\\\"}}\",\"idToken\":\"eyJhbGciOiJSUzI1NiIsImtpZCI6ImNmMDIyYTQ5ZTk3ODYxNDhhZDBlMzc5Y2M4NTQ4NDRlMzZjM2VkYzEiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJuYmYiOjE1NTIwNzEzNjAsImF1ZCI6Ijk4NzgxNjU4MTIzMC1ibWVtMm5wcTRsNnE3c2I5MnVpM3BkdGRhMWFmajJvNy5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsInN1YiI6IjEwMjk3OTk3NzIyNTM3NjY4MDEyOSIsImVtYWlsIjoiZGF2aWR2ZXJ3ZWlqQGdtYWlsLmNvbSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJuYW1lIjoiRCBWZXJ3ZWlqIiwicGljdHVyZSI6Imh0dHBzOi8vbGg2Lmdvb2dsZXVzZXJjb250ZW50LmNvbS8taldtOFd3eE5iS3MvQUFBQUFBQUFBQUkvQUFBQUFBQUFyLUUvNEE2UmJiYVNwem8vczk2LWMvcGhvdG8uanBnIiwiZ2l2ZW5fbmFtZSI6IkQiLCJmYW1pbHlfbmFtZSI6IlZlcndlaWoiLCJpYXQiOjE1NTIwNzE2NjAsImV4cCI6MTU1MjA3NTI2MCwianRpIjoiYjE4MDYwMjc0YmE4MjJhYzFhYzc0MTYwZjI2YWM2MDk3MzBmZDY4ZSJ9.Y9G0qo0Gf28-noF7RYPhtfHRuA7Qo6bCBSuN56Y0AtgIXaQKZjnmYvABIt9u8WQ1qPWwQc3jOLyhfoXIk8j0zhcQ0M0oc7LjkBwVCgFnJHvUAiV5fGEqQa95pZyrZhYmHipTDdwk0UhJHFGJOXAHDPP6oBSHKC9h48jqUjVszz6iEy4frV0XIKIzRR2U2iY6OgJuxPsV0A7xNjvLXiMmwaRUVtlj9CPmiizd3G2PhqD5C54Fy2Qg5ch89qMOA10vNB5B4AX9pmAXHpmtIqFo7ljvAeGAj-pRuqyMllz2awAdvqqOFRERDYfm5Fyh7N0l1OhR2A2XRegsUIL1I1EVPQ\"},\"conversation\":{\"conversationId\":\"ABwppHH8PXibDZg8in1DjbP-caFy67Dtq025k_Uq2ofoPNXKtiPXrbJTmpGVUnVy-aY6H1MeZCFIpQ\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"actions.intent.MAIN\",\"rawInputs\":[{\"inputType\":\"KEYBOARD\",\"query\":\"Talk to ***anonymized appname***\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.WEB_BROWSER\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.WEB_BROWSER\"}]}],\"requestType\":\"SIMULATOR\"}.",
"insertId": "120zsprg2nqfayb",
"resource": {
"type": "assistant_action",
"labels": {
"action_id": "actions.intent.MAIN",
"project_id": "***anonymized***",
"version_id": ""
}
},
"timestamp": "2019-03-08T19:01:00.243055001Z",
"severity": "DEBUG",
"labels": {
"channel": "preview",
"source": "AOG_REQUEST_RESPONSE",
"querystream": "GOOGLE_USER"
},
{
"textPayload": "Received response from agent with body: HTTP/1.1 200 OK\r\nServer: nginx/1.13.6\r\nDate: Fri, 08 Mar 2019 19:01:10 GMT\r\nContent-Type: application/json;charset=UTF-8\r\nContent-Length: 330\r\nX-Cloud-Trace-Context: 7761b69610701e0a7d18cbc69eef9bde/3001560133061614360;o=0\r\nGoogle-Actions-API-Version: 2\r\nAccess-Control-Allow-Credentials: true\r\nVia: 1.1 google\r\nAlt-Svc: clear\r\n\r\n{\"conversationToken\":\"[]\",\"finalResponse\":{\"richResponse\":{\"items\":[{\"simpleResponse\":{\"textToSpeech\":\"Sorry! I cannot access my online service. Please try again!\"}}]}},\"responseMetadata\":{\"status\":{\"code\":14,\"message\":\"Webhook error (206)\"},\"queryMatchInfo\":{\"queryMatched\":true,\"intent\":\"e0bf4b96-9440-4545-a8a6-d0915cacd34f\"}}}.",
"insertId": "120zsprg2nqfayc",
"resource": {
"type": "assistant_action",
"labels": {
"project_id": "***anonymized***",
"version_id": "",
"action_id": "actions.intent.MAIN"
}
},
"timestamp": "2019-03-08T19:01:10.376894030Z",
"severity": "DEBUG",
"labels": {
"channel": "preview",
"source": "AOG_REQUEST_RESPONSE",
"querystream": "GOOGLE_USER"
},
"logName": "projects/***anonymized***/logs/actions.googleapis.com%2Factions",
"trace": "projects/987816581230/traces/ABwppHH8PXibDZg8in1DjbP-caFy67Dtq025k_Uq2ofoPNXKtiPXrbJTmpGVUnVy-aY6H1MeZCFIpQ",
"receiveTimestamp": "2019-03-08T19:01:10.389139428Z"
},
{
"textPayload": "MalformedResponse: Webhook error (206)",
"insertId": "1d4bzl9g3lossug",
"resource": {
"type": "assistant_action",
"labels": {
"project_id": "***anonymized***",
"version_id": "",
"action_id": "actions.intent.MAIN"
}
},
"timestamp": "2019-03-08T19:01:10.377231474Z",
"severity": "ERROR",
"labels": {
"channel": "preview",
"source": "JSON_RESPONSE_VALIDATION",
"querystream": "GOOGLE_USER"
},
"logName": "projects/***anonymized***/logs/actions.googleapis.com%2Factions",
"trace": "projects/987816581230/traces/ABwppHH8PXibDZg8in1DjbP-caFy67Dtq025k_Uq2ofoPNXKtiPXrbJTmpGVUnVy-aY6H1MeZCFIpQ",
"receiveTimestamp": "2019-03-08T19:01:10.388395945Z"
}
Help!
Any help or guidance towards unraveling this issue is much apreciated. Let me know if you have experienced this before, have a potential solution or would like to see more details of the code or logs. Many thanks!
I got the same error when I tried to modify one of my google assistant app.
Everything was working well but I suddently got the same error as you that I was not able to understand and was not related to my new developpement.
{
insertId: "102mhl8g1omvh70"
labels: {
channel: "preview"
querystream: "GOOGLE_USER"
source: "JSON_RESPONSE_VALIDATION"
}
logName: "projects/myprojectID/logs/actions.googleapis.com%2Factions"
receiveTimestamp: "2019-04-22T16:56:11.508733115Z"
resource: {
labels: {
action_id: "actions.intent.MAIN"
project_id: "myprojectID"
version_id: ""
}
type: "assistant_action"
}
severity: "ERROR"
textPayload: "MalformedResponse: Webhook error (206)"
timestamp: "2019-04-22T16:56:11.498787357Z"
trace: "projects/168413137357/traces/ABwppHG8ckJJgXMT5Jedih2WUtGNZZc9i0BVG5S-CkxCT8mkhy7mDr8L9GPd9p_EvXIIlTz3SK2z16jBK8Id"
}
I tried to solve it by disabling the webhook and set the playload in the dialogflow intent. Anyway I still got this error.
So I tried to rollback on my developpement by uploading the old version the incremeneted intent but this didn't work and I still got my error
Watching to my function logs I saw this
SyntaxError: Unexpected token : in JSON at position 6
at Object.parse (native)
at new User (/user_code/node_modules/actions-on-google/dist/service/actionssdk/conversation/user.js:75:43)
at DialogflowConversation.Conversation (/user_code/node_modules/actions-on-google/dist/service/actionssdk/conversation/conversation.js:47:21)
at DialogflowConversation (/user_code/node_modules/actions-on-google/dist/service/dialogflow/conv.js:36:9)
at WebhookClient.conv (/user_code/node_modules/dialogflow-fulfillment/src/dialogflow-fulfillment.js:415:14)
at welcome (/user_code/index.js:37:26)
at WebhookClient.handleRequest (/user_code/node_modules/dialogflow-fulfillment/src/dialogflow-fulfillment.js:273:44)
at exports.dialogflowFirebaseFulfillment.functions.https.onRequest (/user_code/index.js:247:11)
at cloudFunction (/user_code/node_modules/firebase-functions/lib/providers/https.js:37:41)
at /var/tmp/worker/worker.js:783:7
I didn't understand how the json from the assistant could not be parsed properly I was very surprised. I compared the JSON input that I got before and after the bug and I saw that the userstorage entry was not well formated.
I got something like that :
"user":{"userStorage":"\"data\":}","lastSeen":"2019-04-22T16:22:10Z","locale":"Mylanguage","userId":"ABwppHGoZFMQwTpdqZaOYBaIh32-hXH2kUN_EPJglUXMl5MeH7QlLyB9O0bCgZ0SQ9L0B2Ks-JQ9rHfRoAh-"}
instead of this :
"user":{"userStorage":"{\"data\":{}}","lastSeen":"2019-04-21T17:55:21Z","locale":"Mylanguage","userId":"ABwppHGoZFMQwTpdqZaOYBaIh32-hXH2kUN_EPJglUXMl5MeH7QlLyB9O0bCgZ0SQ9L0B2Ks-JQ9rHfRoAh-"}
And this json is going to provoke an error when you try to get the conversation from the agent such like that :
let conv = agent.conv();
I still don't understand how this is even possible but I have a trick to solve it.
I fix it by correcting the json before getting the conversation.
In my app I don't need any userstorage so I always intialise the userstorage properly before getting the conversation like that :
request.body.originalDetectIntentRequest.payload.user.userStorage = "{\"data\":{}}";
I am about to contact the dialogflow support to inform them of this I consider to be a bug
I hope this answer is helping you!
I am trying to create a checkout via Shopify admin rest API and in response, I am getting a validation error which looks like this.
{
"errors": {
"line_items": {
"0": {
"variant_id": [
{
"code": "invalid",
"message": "is invalid",
"options": {}
}
]
}
}
}
}
Request[POST]:
URL: https://mystore.myshopify.com/admin/checkouts.json
{
"checkout": {
"line_items": [
{
"variant_id": "15465470460017",
"quantity": 1
}
]
}
}
I verified the varient_id by looking into the cart JSON on the link "https://mystore.myshopify.com/cart.json".
I also followed the documentation given on Shopify. If I create empty checkout without varient_id then it works otherwise it returns the invalid variant id error.
I tried this request on PostMan. For updated and others it is working fine but for checkout create I couldn't find the point where I am going wrong.
I went to the Functions/API Key to retrieve the user&password, but I still receive this error:
Dialog node error
Mandatory action property "credentials" missing or
invalid for server-side CloudFunctions action call. The value must be
a string that references a variable such as "$my_creds" that expands
to an object like {"user":"..", "password":".."}. Dialog node:
[GetProducts]
Any ideas why?
// IBM WATSON Dialog:
// Dialog Node Name: GetProducts
// JSON Editor:
{
"context": {
"private": {
"my_creds": {
"user": "*********",
"password:": "*********"
}
}
},
"output": {
"text": {
"values": [
"Product : "<?entities.products[0].literal?>"
],
"selection_policy": "sequential"
}
},
"actions": [
{
"name": "/*****#gmail.com_dev/getProducts2",
"type": "server",
"parameters": {
"url": "<?entities.products[0].literal?>"
},
"credentials": "$private.my_creds",
"result_variable": "context.result"
}
]
}
The credentials need to be present when that dialog node is processed. The context section defines what will be present at the end of that processing. Thus, the credentials are not known to the action.
My advise is to NOT store the credentials in the workspace. This is a security issue and bad practice, even for testing. Follow the example in the Watson Assistant documentation. It has instructions on how to add the credentials to the "Try it out" panel. For production, pass in the credentials from the app or middleware. Here are some examples on how that can be done.