We recently decided to include E2E to our front-end pipeline and we are using testCafe for that and since we use AWS as our SaaS we're being asked to use Devicefarm for remote testing and I'm facing the problem to connect them. I'm based on the selenium implementations and the testCafe documentation but it seems not to be able to establish connection between them, anyone has any idea why?
My code:
const AWS = require('aws-sdk');
const createTestCafe = require('testcafe');
const PROJECT_ARN = "arn:aws:devicefarm:us-west-2:XXXXXXXX:testgrid-project:XXXXXX-XXXX-XXXX-XXXX-XXXXXXXX";
const devicefarm = new AWS.DeviceFarm({ region: "us-west-2", credentials: AWS.config.credentials });
(async () => {
const testGridUrlResult = await devicefarm.createTestGridUrl({
projectArn: PROJECT_ARN,
expiresInSeconds: 6000
}).promise();
const url = new URL(testGridUrlResult.url || '');
const testCafe = await createTestCafe(url.host, 443);
const runner = testCafe.createRunner();
const remoteConnection = await testCafe.createBrowserConnection();
remoteConnection.once('ready', async () => {
await runner
.src(['./load-mfc.ts'])
.browsers('devicefarm:firefox')
.run();
await testCafe.close();
});
})().catch((e) => console.error(e));
According to the AWS Device Farm documentation about the CreateTestGridUrl method:
Creates a signed, short-term URL that can be passed to a Selenium RemoteWebDriver constructor.
This URL can't be passed to the hostname property of the createTestCafe method because TestCafe doesn't implement the WebDriver protocol. TestCafe works differently then Selenium. Once you created a remote browser connection using TestCafe, you should navigate your remote browser to the browserConnection.url URL to connect to a TestCafe server instance and start test execution in this browser.
As the AWS Device Farm service uses the WebDriver protocol to operate remote browsers, you would need to write a custom Browser Provider Plugin for TestCafe. It may be helpful to read more in the TestCafe documentation about this topic and see how a similar thing is implemented in the BrowserStack provider.
Related
For the past few days I've been trying to solve the problem of publishing a message from Lambda to the AWS cloud, using Greengrass v2.
The code in python was even provided in the documentation, only had to be slightly reworked.
When it comes to SDK v2 JS in documentation there is only minimal mention about publish function in AWS-CRT library.
I tried to create code using components from this library, but it looks like the library also requires a script with parameters.
This is my code that requires installation of aws-iot-sdk-v2 js.
const iotsdk = require("aws-iot-device-sdk-v2");
const mqtt = iotsdk.mqtt;
const os = require("os");
const util = require("util");
const GROUP_ID = process.env.GROUP_ID;
const THING_NAME = process.env.AWS_IOT_THING_NAME;
const THING_ARN = process.env.AWS_IOT_THING_ARN;
(topic = "gg/message"),
(payload = JSON.stringify({ message: util.format("ping") }));
function greengrassHelloWorldRun() {
mqtt.MqttClientConnection.prototype.publish(topic, payload);
}
console.log(topic);
console.log(payload);
setInterval(greengrassHelloWorldRun, 5000);
exports.handler = function (event, context) {
console.log("event: " + JSON.stringify(event));
console.log("context: " + JSON.stringify(context));
};
I get errors about arguments and NAPI.
The same errors also appear when using this function as lambda component in greengrass logs
Maybe someone has some example how to publish some message on topic using Node lambda with sdk v2.
After contacting AWS Support I know it is impossible.
AWS doesn't support mqttProxy IPC for SDK V2 JS yet.
ChristopherTal
I'm also using the Greengrass SDKs for JS and indeed they're still a work in progress. But I was able to send messages to the IoTCore from Greengrass using the JS SDKs.
A few things to mention:
You seem to use the aws-iot-device-sdk-v2 SDK which is for things
The aws-greengrass-core-sdk npm package is made for components
It is important to differ between things and components and decide who's doing what.
To send data to IoTCore from a thing, you need indeed to use MQTT. On the deployment page on the Greengrass console, you need to revise the deployment and add following components:
MQTT Broker
MQTT Bridge
Client device auth
This way your thing connects to the local MQTT Broker through the client device auth component and the MQTT Bridge decides how the traffic is routed. You can read all info on the links above.
I even realised this using the standard mqtt npm package. You need to create a certificate and a thing using lambda or the console and use those certificates to access the broker.
const mqtt = require('mqtt')
const fs = require('fs')
const ca = fs.readFileSync(locationOfTheCA)
const key = fs.readFileSync(locationOfThePrivateKey)
const cert = fs.readFileSync(locationOfTheCertificate)
console.log('Welcome to MQTT Connector')
const client = mqtt.connect('mqtts://localhost:8883', {
clientId: 'yourThingNameHere',
ca: ca,
key: key,
cert: cert
})
client.on('connect', function () {
console.log('Connected to MQTT')
/* client.subscribe('$aws/*', function (err) {
if (!err) {
//client.publish('presence', 'Hello mqtt')
}
})*/
})
client.on('message', function (topic, message) {
// message is Buffer
console.log(message.toString())
client.end()
})
Hopefully this helps you out!
Warm regards
Hacor
Hi everybody and thanks for taking time to look at my issue/question.
I am getting different results when deploying my AWS Lambda stand-alone versus within an Application Stack.
I'm trying to connect to AWS Elasticache Redis from within my Lambda. I have .Net Core 3.1 Lambdas (using StackExchange.Redis) which can connect. But I also need to be able to connect from my Node.js Lambdas.
For the Node.js Lambdas, I'm using "node-redis" and "async-redis". I have two Lambdas which are essentially identical except that one is deployed in an Application Stack and the other is deployed as a stand-alone Lambda. Both Lambdas reference the same Lambda Layer (i.e. same "node_modules"), have the same VPC settings, the same Execution Role, and essentially the same code. So they've pushed it up to another group.
The stand-alone Lambda connects to Redis without issue. The Application Stack Lambda does not and exits processing before completing but without raising any error.
At first I thought I might just need to configure my Application Stack but I cannot find any information indicating we even can configure Application Stacks. So I'm at a loss.
The stand-alone Lambda:
exports.handler = async (event) => {
const asyncRedis = require("async-redis");
const redisOptions =
{
host: "XXXXXXXXX.XXXXX.XXXX.use2.cache.amazonaws.com",
port: 6379
}
console.log('A');
const client = asyncRedis.createClient(redisOptions);
console.log(client);
console.log('B');
const value = await client.get("Key");
console.log('C');
console.log(value);
console.log('D');
console.log(client);
};
The output of this function is essentially:
A
{RedisClient} --> the "client" object --> Shows connected = false
B
C
{ Correct Data From Redis }
D
{RedisClient} --> the "client" object --> Shows connected = true
The Application Stack Lambda:
async function testRedis2(event, context) {
console.log('In TestRedis2');
const asyncRedis = require("async-redis");
const redisOptions =
{
host: "XXXXXXXXX.XXXXX.XXXX.use2.cache.amazonaws.com",
port: 6379
}
console.log('A');
const client = asyncRedis.createClient(redisOptions);
console.log(client);
console.log('B');
var value = await client.get("Key");
console.log('C');
console.log(value);
console.log('D');
console.log(client);
}
module.exports = {
testRedis2
};
The output of this function is essentially:
In TestRedis2
A
{RedisClient} --> the "client" object --> Shows connected = false
B
I don't understand why these don't perform identically. And I don't get why I don't see further entries in the output.
Has anyone else experienced issues connecting to VPC resources from within an Application Stack?
Thanks
I stumbled across the answer through extensive trial and error. It may be obvious to Node/js developers but, just in case another Javascript/Node newbie has the same issue, I'll post the answer here.
The import/require and creation of the client must be at the top of the module. Not in the function itself.
So, the following does work in my application stack:
const asyncRedis = require("async-redis");
const redisOptions = {
host: "XXXXXXXXX.XXXXX.XXXX.use2.cache.amazonaws.com",
port: 6379
};
const client = asyncRedis.createClient(redisOptions);
async function redisGet(key: string){
// console.log('In redisGet');
return await client.get(key);
}
New to Google Cloud Run and trying to have two node.js microservices communicate internally via gRPC.
The client interface:
constructor(address: string, credentials: grpc.ChannelCredentials, options?: object);
The client code:
const client: MyClient = new MyClient('my-service-abcdefgh3a-ew.a.run.app:443', grpc.credentials.createSsl());
The server code:
const server = new grpc.Server();
server.addService<IMyServer>(MyService, new MyServer());
server.bind(`0.0.0.0:${process.env.PORT}`, grpc.ServerCredentials.createInsecure());
server.start();
The server is set to listen to 443.
The above seems to work when the service is open to public requests but doesn't work when you have the server set as internal. Any ideas?
You have to add the credentials in the request metadata. Here an example
...
// Create a client for the protobuf spec
const client = new protoObj.Greeter(HOST, grpc.credentials.createInsecure());
// Build gRPC request
const metadata = new grpc.Metadata();
metadata.add('authorization', `Bearer ${JWT_AUTH_TOKEN}`);
// Execute gRPC request
client.sayHello({name: GREETEE}, metadata, (err, response) => {...
Second question, how to get the JWT_AUTH_TOKEN. Here the documentation of Cloud Run to do this. But not completely, simply get the token and use it in the metadata of the request
...
request(tokenRequestOptions)
.then((token) => {
// add the token to the metadata
});
// Make the call
...
I am using actionssdk and I build my Action fulfilments using Javascript and node.js + Express.
I am looking for a way to get the url (protocol + host name + port) of the server where the fulfilment is hosted.
Is there a simple way to do this? E.g. in the MAIN intent? Is there some conv-property I can use? Can I get hold of a req-parameter in the MAIN-intent, from which I can deduct hostname etc?
const express = require('express');
const expressApp = express();
const { actionssdk, ... } = require('actions-on-google');
const app = actionssdk({
ordersv3: true,
clientId: ...
});
expressApp.post('/fulfilment', app);
app.intent('actions.intent.MAIN', (conv) => {
let myUrl: string = ... // ???????
...
});
(background: obviously I know myself to where I deployed my fulfilment code. But I have a reusable template for fulfilment code in which I want to refer to the host url, and I do not want to type that in manually each time I develop a new Action).
You can get access to the request object in a middleware via Framework Metadata which is by default of type BuiltinFrameworkMetadata which contains objects used by Express
For example, you can use it like this, which will be ran before each request:
app.middleware((conv, framework) => {
console.log(framework.express.request.headers.host)
})
I am using the npm elasticsearch package to search my AWS ES domain. Everything seems to work fine when I use Postman to make POST requests with my AWS IAM credentials.
I wanted to do the same in my code (node.js). I referred to this answer here:
How to make calls to elasticsearch apis through NodeJS?
Here is code:
const elasticsearch = require('elasticsearch');
const awsHttpClient = require('http-aws-es');
const AWS = require('aws-sdk');
const client = new elasticsearch.Client({
host: 'my-aws-es-endpoint',
connectionClass: awsHttpClient,
amazonES: {
region: 'us-east-1',
credentials: new AWS.Credentials('my-access-key','my-secret-key')
}
});
But when I run client.search(), it fails with an error:
Elasticsearch ERROR: 2018-10-31T15:12:22Z
Error: Request error, retrying
POST https://my-endpoint.us-east-1.es.amazonaws.com/my-index/student/_search => Data must be a string or a buffer
It also gives me a warning
Elasticsearch WARNING: 2018-10-31T15:12:22Z
Unable to revive connection: https://my-endpoint.us-east-1.es.amazonaws.com/
When I use just the aws-sdk, it works fine (probably because I sign the request there?)
Can someone suggest what I am I doing wrong here?
I was able to solve this by specifying the region. There is a problem with the elasticsearch client where it's not able to the pick the region which we specify in
amazonES: {
region: 'us-east-1',
credentials: new AWS.Credentials('my-access-key','my-secret-key')
}
}
I solved it by specifying the region using AWS.config.region before the above code
AWS.config.region = 'us-east-1';