Google cloud tasks NodeJS api: Get queue stats? - node.js

I want to obtain the stats field for a queue in google cloud tasks using the nodejs client library #google-cloud/tasks. The stats field only exists in the v2beta3 version, however to get it we need to pass a query params readMask=*, but I don't know how to pass it using the client lib.
I tried using the otherArgs params, but its not working.
const tasks = require('#google-cloud/tasks');
const client = new tasks.v2beta3.GoogleCloudTasks()
// Get queue containing stats
const queue = await client.getQueue({name: '..'}, {otherArgs: {readMask: '*'}})

The readMask specifies which paths of the response object to fetch. The response will include all possible paths with placeholders like null, UNSPECIFIED, etc. and then contain the actual values you want.
const request = {
...
readMask: { paths: ['name', 'stats', 'state', ...] }
};
getQueue
const { v2beta3 } = require('#google-cloud/tasks');
const tasksClient = new v2beta3.CloudTasksClient();
async function main() {
const request = {
name: 'projects/PROJECT/locations/LOCATION/queues/QUEUE',
readMask: { paths: ['name', 'stats'] }
};
const [response] = await tasksClient.getQueue(request);
console.log(response);
}
main();
/*
{
name: 'projects/PROJECT/locations/LOCATION/queues/QUEUE',
...
stats: {
tasksCount: '113',
oldestEstimatedArrivalTime: null,
executedLastMinuteCount: '0',
concurrentDispatchesCount: '0',
effectiveExecutionRate: 500
}
}
*/
listQueues
const { v2beta3 } = require('#google-cloud/tasks');
const tasksClient = new v2beta3.CloudTasksClient();
async function main() {
const request = {
parent: 'projects/PROJECT/locations/LOCATION',
readMask: { paths: ['name', 'stats'] }
};
const [response] = await tasksClient.listQueues(request);
console.log(response);
}
main();
/*
[
{
name: 'projects/PROJECT/locations/LOCATION/queues/QUEUE',
...
stats: {
tasksCount: '1771',
oldestEstimatedArrivalTime: [Object],
executedLastMinuteCount: '0',
concurrentDispatchesCount: '0',
effectiveExecutionRate: 500
}
},
...
]
*/

By taking a look at the source code for the client library I see no reference for the readMask parameter as specified on the v2beta3 version of the REST API projects.locations.queues.get method.
The relevant method on the NodeJS client library getQueue() expects a type of request IGetQueueRequest that doesn't have the readMask parameter and is only expecting the name property.
Nonetheless this implementation might change in the future to include a relevant method to get the stats.
Regarding the REST API itself, there is an error on the public docs on the readMask section as * is not a valid character. If you want to get the Queue.stats field you should simply enter stats on the readMask parameter. If you want to get all the relevant fields you should enter all of them (e.g. name,rateLimits,retryConfig,state,taskTtl,tombstoneTtl,type,stats should get all the relevant fields you get from calling the method + the Queue.stats field).
The following picture should help you.
As a workaround if you click on the expand symbol on the Try this API section of the docs for the relevant method you could click on the JAVASCRIPT section and get the relevant code as how to build the request as shown on the following picture.
EDIT JANUARY 23rd 2020
The documentation was corrected to inform that in order to express that:
[Queue.stats] will be returned only if it was explicitly specified in the mask.
Which translates that simply writing stats under the readMask field will return the stats.

Related

Hubspot pagination using after in nodejs

i am building hubspot api, i having trouble paginating the contacts records.
i am using #hubspot/api-client - npm for integration with hubspot and this is the docs for that https://github.com/HubSpot/hubspot-api-nodejs
hubspotClient.crm.contacts.basicApi
.getPage(limit, after, properties, propertiesWithHistory, associations, archived)
.then((results) => {
console.log(results)
})
.catch((err) => {
console.error(err)
})
in this code there is after argument, we can provide contacts id in it, and it will provide the records including and after that particular id.
How do i use this for pagination. or there is any other way.
Take a look at API Endpoints documentation for GET /crm/v3/objects/contacts and the data you receive. The getPage method returns the following data:
{
"results": [
{
// contact detail here
}
],
"paging": {
"next": {
"after": "NTI1Cg%3D%3D",
"link": "?after=NTI1Cg%3D%3D"
}
}
}
The pagination information is available in paging.next.after (if there is a consecutive page). So you can do something like this to iterate through each page:
async function doSomethingWithEachPage() {
let after = undefined;
const limit = 10;
const properties = undefined;
const propertiesWithHistory = undefined;
const associations = undefined;
const archived = false;
do {
const response = await hubspotClient.crm.contacts.basicApi.getPage(
limit,
after,
properties,
propertiesWithHistory,
associations,
archived
);
// do something with results
console.log(response.results); // contacts list
// pick after from response and store it outside of current scope
after = response.paging?.next?.after;
} while (after);
}
I rewrote the sample code to use async/await so it better works with do...while loop and omitted error handling.
If you are dealing with reasonable small amount of data and have enough of memory, you can also skip the pagination and use the getAll method to load all the data. (In fact, this method does internally a loop similar to the one above.)

Patch K8s Custom Resource with #kubernetes/client-node

I'm building Istio/K8s-based platform for controlling traffic routing with NodeJS. I need to be able to programmatically modify Custom Resources and I'd like to use the #kubernetes/node-client for that. I wasn't able to find the right API for accessing Custome Resources in docs and the repo. Am I missing something? Thx in adv.
EDIT: When using CustomObjectApi.patchNamespacedCustomObject function, I'm getting the following error back from K8s API:
message: 'the body of the request was in an unknown format - accepted media types include: application/json-patch+json, application/merge-patch+json, application/apply-patch+yaml', reason: 'UnsupportedMediaType', code: 415
My Code:
const k8sYamls = `${path.resolve(path.dirname(__filename), '..')}/k8sYamls`
const vServiceSpec = read(`${k8sYamls}/${service}/virtual-service.yaml`)
const kc = new k8s.KubeConfig()
kc.loadFromDefault()
const client = kc.makeApiClient(k8s.CustomObjectsApi)
const result = await client.patchNamespacedCustomObject(vServiceSpec.apiVersion.split('/')[0], vServiceSpec.apiVersion.split('/')[1], namespace, 'virtualservices', vServiceSpec.metadata.name, vServiceSpec)
virtual-service.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: message-service
spec:
hosts:
- message-service
http:
- name: 'production'
route:
- destination:
host: message-service
weight: 100
retries:
attempts: 3
perTryTimeout: 2s
retryOn: 5xx
I was using the wrong type for the body object in that method. I got it to work following this example.
const patch = [
{
"op": "replace",
"path":"/metadata/labels",
"value": {
"foo": "bar"
}
}
];
const options = { "headers": { "Content-type": k8s.PatchUtils.PATCH_FORMAT_JSON_PATCH}};
k8sApi.patchNamespacedPod(res.body.items[0].metadata.name, 'default', patch, undefined, undefined, undefined, undefined, options)
.then(() => { console.log("Patched.")})
.catch((err) => { console.log("Error: "); console.log(err)});
You can use the patchClusterCustomObject or patchNamespacedCustomObject methods, depending on whether the given object is namespaced or not, of the customObjectsApi.
Since the edit queue is full for the accepted answer I'm going to provide a working example and more thorough answer here using patchNamespacedCustomObject:
Here are some helpful links:
patchNamespacedCustomObject Docs
Your patch object should be an array of this object
The op field should be one of these
import * as k8s from "#kubernetes/client-node";
import { CustomObjectsApi, KubeConfig } from "#kubernetes/client-node";
export async function patchCustomResource(newValue: any) {
const kc: KubeConfig = new k8s.KubeConfig();
kc.loadFromDefault();
const custObjApi: CustomObjectsApi = kc.makeApiClient(k8s.CustomObjectsApi);
const objectName = 'your-object';
const options = { "headers": { "Content-type": k8s.PatchUtils.PATCH_FORMAT_JSON_PATCH}};
const group = 'your-group'; // found in the CRD
const version = 'v1'; // found in the CRD
const plural = 'object-plural'; // found in the CRD
const namespace = `your-namespace`;
// this will replace the value at `"/spec/path/to/modify"` with the contents of `newValue`
const patch = [{
"op": "replace",
"path":"/spec/path/to/modify",
"value": newValue
}];
try {
await custObjApi.patchNamespacedCustomObject(group, version, namespace, plural, objectName, patch, undefined, undefined, undefined, options);
console.log(`Successfully updated custom resource`);
} catch (error) {
const {body} = error;
console.log(body.message);
}
}
I urge anyone to spend some time on the CustomObjectsAPI docs to get more familiar with the available methods. The docs are dense but it's at least worth it to glance over method names.
After fighting through endless body-parsing errors using #kubernetes/node-client, I opted to install kubectl on my NodeJS workers and use shell.js to run it with.
My conclusion is that the #kubernetes/node-client is buggy when used with Istio Custom Resources, but didn't want to spend time debugging exactly what's wrong. I will post the Github Issue about it in their repo soon.

Adding query param to mailchimp request with Node.js client library

I am trying to list out all my interests from the MailChimp api using the #mailchimp/mailchimp_marketing npm library, as that is what they use as examples for node.js in their docs.
Link to the npm library:
https://www.npmjs.com/package/#mailchimp/mailchimp_marketing
Link to the relevant documentation for this specific endpoint: https://mailchimp.com/developer/api/marketing/interests/list-interests-in-category/
Now, I can get my interests just fine with the example code there:
const run = async () => {
const response = await client.lists.listInterestCategoryInterests(
"list_id",
"interest_category_id"
);
console.log(response);
};
run();
The problem is, that by default the MailChimp API only returns the first 10 items in the list, and I need 15. There is an easy way to change this, of course, by adding a query param count=15. However, I can't find any way to pass on a query param with the listInterestCategoryInterests method as provided through the official library.
!TL;DR! So my question is:
Does anybody know how to pass on query params to the mailchimp API through the official node.js npm library, or do I really have to resort to just dropping the library entirely as it does not provide this basic functionality?
You need to list params as a string in an array:
const response = await client.lists.listInterestCategoryInterests({fields:[ "list_id,interest_category_id"]}
);
NOTE: A prefix maybe required as per below:
const response = await mailchimp.reports.getAllCampaignReports({fields:["reports.campaign_title,reports.clicks"]})
Result:
[0] {
[0] reports: [
[0] { campaign_title: 'COACT EMAIL CAMPAIGN', clicks: [Object] },
[0] { campaign_title: '', clicks: [Object] }
[0] ]
[0] }
const response = await mailchimp.lists.getListMembersInfo(listId,
{
count: 1000
});
For everyone coming here hoping to learn how to pass QUERY params into mailchimp marketing library methods:
The query parameters are taken from opts object - the object properties have to be camelCase.
In terms of which parameter for the method the opts object is - it depends on the method and you might need to check the method's source code, but probably second or third parameter.
As for the question for the concrete method, this should be the solution:
await client.lists.listInterestCategoryInterests(
"list_id",
"interest_category_id",
{ count: 15 }
);

How can one upload an image to a KeystoneJS GraphQL endpoint?

I'm using TinyMCE in a custom field for the KeystoneJS AdminUI, which is a React app. I'd like to upload images from the React front to the KeystoneJS GraphQL back. I can upload the images using a REST endpoint I added to the Keystone server -- passing TinyMCE an images_upload_handler callback -- but I'd like to take advantage of Keystone's already-built GraphQL endpoint for an Image list/type I've created.
I first tried to use the approach detailed in this article, using axios to upload the image
const getGQL = (theFile) => {
const query = gql`
mutation upload($file: Upload!) {
createImage(file: $file) {
id
file {
path
filename
}
}
}
`;
// The operation contains the mutation itself as "query"
// and the variables that are associated with the arguments
// The file variable is null because we can only pass text
// in operation variables
const operation = {
query,
variables: {
file: null
}
};
// This map is used to associate the file saved in the body
// of the request under "0" with the operation variable "variables.file"
const map = {
'0': ['variables.file']
};
// This is the body of the request
// the FormData constructor builds a multipart/form-data request body
// Here we add the operation, map, and file to upload
const body = new FormData();
body.append('operations', JSON.stringify(operation));
body.append('map', JSON.stringify(map));
body.append('0', theFile);
// Create the options of our POST request
const opts = {
method: 'post',
url: 'http://localhost:4545/admin/api',
body
};
// #ts-ignore
return axios(opts);
};
but I'm not sure what to pass as theFile -- TinyMCE's images_upload_handler, from which I need to call the image upload, accepts a blobInfo object which contains functions to give me
The file name doesn't work, neither does the blob -- both give me server errors 500 -- the error message isn't more specific.
I would prefer to use a GraphQL client to upload the image -- another SO article suggests using apollo-upload-client. However, I'm operating within the KeystoneJS environment, and Apollo-upload-client says
Apollo Client can only have 1 “terminating” Apollo Link that sends the
GraphQL requests; if one such as apollo-link-http is already setup,
remove it.
I believe Keystone has already set up Apollo-link-http (it comes up multiple times on search), so I don't think I can use Apollo-upload-client.
The UploadLink is just a drop-in replacement for HttpLink. There's no reason you shouldn't be able to use it. There's a demo KeystoneJS app here that shows the Apollo Client configuration, including using createUploadLink.
Actual usage of the mutation with the Upload scalar is shown here.
Looking at the source code, you should be able to use a custom image handler and call blob on the provided blobInfo object. Something like this:
tinymce.init({
images_upload_handler: async function (blobInfo, success, failure) {
const image = blobInfo.blob()
try {
await apolloClient.mutate(
gql` mutation($image: Upload!) { ... } `,
{
variables: { image }
}
)
success()
} catch (e) {
failure(e)
}
}
})
I used to have the same problem and solved it with Apollo upload link. Now when the app got into the production phase I realized that Apollo client took 1/3rd of the gzipped built files and I created minimal graphql client just for keystone use with automatic image upload. The package is available in npm: https://www.npmjs.com/package/#sylchi/keystone-graphql-client
Usage example that will upload github logo to user profile if there is an user with avatar field set as file:
import { mutate } from '#sylchi/keystone-graphql-client'
const getFile = () => fetch('https://github.githubassets.com/images/modules/logos_page/GitHub-Mark.png',
{
mode: "cors",
cache: "no-cache"
})
.then(response => response.blob())
.then(blob => {
return new File([blob], "file.png", { type: "image/png" })
});
getFile().then(file => {
const options = {
mutation: `
mutation($id: ID!, $data: UserUpdateInput!){
updateUser(id: $id, data: $data){
id
}
}
`,
variables: {
id: "5f5a7f712a64d9db72b30602", //replace with user id
data: {
avatar: file
}
}
}
mutate(options).then(result => console.log(result));
});
The whole package is just 50loc with 1 dependency :)
The easies way for me was to use graphql-request. The advantage is that you don't need to set manually any header prop and it uses the variables you need from the images_upload_handler as de docs describe.
I did it this way:
const { request, gql} = require('graphql-request')
const query = gql`
mutation IMAGE ($file: Upload!) {
createImage (data:
file: $file,
}) {
id
file {
publicUrl
}
}
}
`
images_upload_handler = (blobInfo, success) => {
// ^ ^ varibles you get from tinymce
const variables = {
file: blobInfo.blob()
}
request(GRAPHQL_API_URL, query, variables)
.then( data => {
console.log(data)
success(data.createImage.fileRemote.publicUrl)
})
}
For Keystone 5 editorConfig would stripe out functions, so I clone the field and set the function in the views/Field.js file.
Good luck ( ^_^)/*

How to parse struct object in the response from detectIntent of dialogflow v2?

I would like to know the best practice to parse the response of detectIntent of dialogflow v2.
The response of dialogflow v2 includes struct object which is defined in Protocol Buffer. For example, parameters in queryResult.
I know it is possible to parse it by using structToJson which is included in dialogflow v2 SDK for Node.js as sample code. So the my current code is looks like this.
const dialogflow = require("dialogflow");
const structjson = require("./dialogflow/structjson");
identify_intent(sentence, options){
const session_path = this._sessions_client.sessionPath(this._project_id, options.session_id);
// The text query request.
const request = {
session: session_path,
queryInput: {
text: {
text: sentence,
languageCode: this._language
}
}
};
// Send request and log result
return this._sessions_client.detectIntent(request).then(responses => {
let result = responses[0].queryResult;
if (result.parameters){
result.parameters = structjson.structProtoToJson(result.parameters);
}
return result;
});
}
I'm parsing the response manually by using structProtoToJson() following the sample code but it is not practical since I have to do it not only for parameters but also fo fulfillment and other object as well which is formatted in struct.
I'm wondering what is the best practice to parse the response from detectIntent in Node.js app.
You don't need to do anything to convert the queryResult.parameters object into a usable form. It is a JavaScript object with the following structure:
{
fields: {
paramName1: { stringValue: 'value1', kind: 'stringValue' },
paramName2: { stringValue: 'value2', kind: 'stringValue' }
}
}
It will have one key/value pair for each parameter in the intent's parameter list.

Resources