I create containers which compile/interpret user'c code and pass the result back to the browser (just like JSFiddle). Now, I need to know how much CPU and memory has been used for executing that code. So, I don't need it realtime but on container's exit, so that I can pass these two parameters with the others back to the client.
I tried using pseudo-files like here, but there is no such a location on my server (Ubuntu 14.04). How I can measure these parameters?
docker has a stats api
https://docs.docker.com/engine/reference/api/docker_remote_api_v1.24/#/get-container-stats-based-on-resource-usage
"cpu_usage" : {
"percpu_usage" : [
8646879,
24472255,
36438778,
30657443
],
Related
I'm using .Net SDK for ACI (Azure Container Instances) to run an exec command
In the response, I only get back this object which doesn't tell me how to get the actual result of the command e.g. exit code and message.
My question is: how to retrieve the exec command results in .NET?
This is my .NET code:
string command = "ls";
var commandResponse = containerGroup.ExecuteCommand("container1", command, 100, 100);
You can get the LogContent of the container in order to find out what happened. If you are interested only in the newest item, then you can pass 1 as tailLineCount. ContainerExecResponse has a WebSocketUri property as well, which may explain why you do not directly get the info you need in the response object. WebSockets are duplex channels of communication, differing from the request-response protocol-type and in the case of WebSockets, you establish a connection which may run for a very long time, occasionally responding. So here expecting an HTTP-like behavior seems to be a misunderstanding, at least if I look at the docs.
I have a Python (3.x) webservice deployed in GCP. Everytime Cloud Run is shutting down instances, most noticeably after a big load spike, I get many logs like these Uncaught signal: 6, pid=6, tid=6, fault_addr=0. together with [CRITICAL] WORKER TIMEOUT (pid:6) They are always signal 6.
The service is using FastAPI and Gunicorn running in a Docker with this start command
CMD gunicorn -w 2 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8080 app.__main__:app
The service is deployed using Terraform with 1 gig of ram, 2 cpu's and the timeout is set to 2 minutes
resource "google_cloud_run_service" <ressource-name> {
name = <name>
location = <location>
template {
spec {
service_account_name = <sa-email>
timeout_seconds = 120
containers {
image = var.image
env {
name = "GCP_PROJECT"
value = var.project
}
env {
name = "BRANCH_NAME"
value = var.branch
}
resources {
limits = {
cpu = "2000m"
memory = "1Gi"
}
}
}
}
}
autogenerate_revision_name = true
}
I have already tried tweaking the resources and timeout in Cloud Run, using the --timeout and --preload flag for gunicorn as that is what people always seem to recommend when googling the problem but all without success. I also dont exactly know why the workers are timing out.
Extending on the top answer which is correct, You are using GUnicorn which is a process manager that manages Uvicorn processes which runs the actual app.
When Cloudrun wants to shutdown the instance (due to lack of requests probably) it will send a signal 6 to process 1. However, GUnicorn occupies this process as the manager and will not pass it to the Uvicorn workers for handling - thus you receive the Unhandled signal 6.
The simplest solution, is to run Uvicorn directly instead of through GUnicorn (possibly with a smaller instance) and allow the scaling part to be handled via Cloudrun.
CMD ["uvicorn", "app.__main__:app", "--host", "0.0.0.0", "--port", "8080"]
Unless you have enabled CPU is always allocated, background threads and processes might stop receiving CPU time after all HTTP requests return. This means background threads and processes can fail, connections can timeout, etc. I cannot think of any benefits to running background workers with Cloud Run except when setting the --cpu-no-throttling flag. Cloud Run instances that are not processing requests, can be terminated.
Signal 6 means abort which terminates processes. This probably means your container is being terminated due to a lack of requests to process.
Run more workloads on Cloud Run with new CPU allocation controls
What if my application is doing background work outside of request processing?
This error happens when a background process is aborted. There are some advantages of running background threads on cloud just like for other applications. Luckily, you can still use them on Cloud Run without processes getting aborted. To do so, when deploying, chose the option "CPU always allocated" instead of "CPU only allocated during request processing"
For more details, check https://cloud.google.com/run/docs/configuring/cpu-allocation
I am trying to build a service to execute programs in different languages in node and give the output. So far I'm using the child_process's spawn for spawning commands.
let p = spawn('python',[sourceFilePath]);
Is there a way so that I can limit the access of this child_process to my system so it can't access network,file system,etc ?
Node.js itself does not offer any mechanism to restrict child processes to only a subset of available resources other than setting the child process' UID/GID which might not be sufficient given your goal.
Note that remote code execution as a business model (ie. various *fiddle.org, online playgrounds etc.) are very difficult to make because it's a non-trivial task to protect the host operating system.
I will assume that your program will eventually run on a Linux server as that is the most common type of servers available nowadays for Node.js deployments. Covering this topic for all types of operating systems would be too broad and probably not that helpful.
Your goal
to execute a program with a user-provided input and return the output back to the user
restrict this program from accessing some or all I/O resources (file system, network, host OS information, memory etc.)
tolerate malicious users trying to do harm/extract information/compromise your server etc.
Node.js will not help here, at all. Node can spawn a child process, but that's about it. It does not control any of those I/O resources. The kernel does. We must look to what features the Linux kernel offers in this area.
I found a very detailed article about Linux sandboxing which I will use as a source of inspiration. If you are interested I recommend you read it and search for similar ones.
Low-level: Linux namespaces
Linux kernel offers low-level mechanisms to isolate processes in/from various system resources. You might want to check them out although I feel this is too low-level for your use case.
Firejail
Firejail is a tool to isolate a process for testing purposes from other system resources. I have never used it but it looks like it could be used for your use case.
Containers (ie. Docker)
Containers usually utilise Linux namespaces to create an environment which looks like a full operating system to the process running inside of them, even though they allow complete isolation from the host OS. You can restrict network access, filesystem access and even CPU/memory usage when running a program inside of a container.
Given your use case, I would probably go with container isolation as the community around them is quite huge nowadays which increases the likelihood of you finding the right support/documentation to achieve your goals.
if it's Unix you can provide uid, gid of user/group with lower permissions:
var child = spawn(process, args, {
detached: true,
uid: uid,
gid: gid,
cwd: appDir,
stdio: [ 'ignore', 'pipe', 'pipe']
})
You can use vm2 module for this.I also use that to run untrusted code by users.You can create sandbox to run user code and sandbox can access resources that only you specify to it.
For example
const vm = new VM({
sandbox: {
console:console,
timeout:200,
fileName:fileName,
cmdCommand:cmdCommand,
url:url,
exec:exec,
input:inputs,
imageName:imageName,
reqs:reqs,
resp:resp}
});
This can be your sandbox.User can use only that specific varibale that are listed above.For example if you remove console from above.Then user will not able to log.If they tries to do that code will through error.Late
vmCode = `var child = exec(cmdCommand,function(stderr, result) {
console.log("done in exec")
console.log(fileName)
console.log(result)
if(stderr){
console.log("stderr")
console.log(stderr)
resp.send({"error":"Syntax error"})
}
else {
console.log(result)
}
}
})`
This is place where your child process code will be defined (as string)
Finally
vm.run(vmCode,(err)=>{
console.log("done")
})
This statement will execute code that is as string (vmCode is defined above)
In the documentation it is written that you can use this to run untrusted code.
For more you can read from here https://www.npmjs.com/package/vm2
I have this simple node socker seerver as follows:
var ws = require("nodejs-websocket")
var connectionCount = 0;
console.info("Node websocket started # 8002");
var server = ws.createServer(function (conn) {;
console.log("New connection", ++connectionCount);
conn.on("close", function (code, reason) {
console.log("Connection closed")
});
}).listen(8002);
Now I want to hit this server from machines. So to mimic these machines, I am using docker. I want to create around 10 different docker containers which will hit my server.
I want to hit the server from this docker container by using the load testing tool called thor (https://github.com/observing/thor), which can be run as easily as
thor --amount 1000 --messages 100 ws://localhost:8002
So I want to created 10 different docker container and each container should use this tool called thor and hit my server with
thor --amount 1000 --messages 100 ws://localhost:8002
How can I implement such dockor containers.
PS: I am a novice here.
I believe that it should be possible.
There are images available in the docker hub for node of varying size. Choose the appropriate image.
Here are the pseudo instructions to create an image that you needed.
Get the node image
Install thor from git(which you already have the details)
Run the container with your command(Hoping that your websocket app might already be running)
You can do the above in two ways either doing it manually or using Dockerfile.
I believe that you wanted to run in multiple containers, Dockerfile would be good option.
If you can use docker-compose, since multiple containers, it would even better approach.
Hope this is helpful.
I'm trying to write a test to test a method that connects to mongo, but I don't actually want to have to have mongo running and actually make a connection to it to have my tests pass successfully.
Here's my current test which is successful when my mongo daemon is running.
describe('with a valid mongo string parameter', function() {
it('should return a rejected promise', function(done) {
var con = mongoFactory.getConnection('mongodb://localhost:27017');
expect(con).to.be.fulfilled;
done();
});
});
mongoFactory.getConnection code:
getConnection: function getConnection(connectionString) {
// do stuff here
// Initialize connection once
MongoClient.connect(connectionString, function(err, database) {
if (err) {
def.reject(err);
}
def.resolve(database);
});
return def.promise;
}
There are a couple of SO answers related to unit testing code that uses MongoDB as a data store:
Mocking database in node.js?
Mock/Test Mongodb Database Node.js
Embedded MongoDB when running integration tests
Similar: Unit testing classes that have online functionality
I'll make an attempt at consolidating these solutions.
Preamble
First and foremost, you should want MongoDB to be running while performing your tests. MongoDB's query language is complex, so running legitimate queries against a stable MongoDB instance is required to ensure your queries are running as planned and that your application is responding properly to the results. With this in mind, however, you should never run your tests against a production system, but instead a peripheral system to your integration environment. This can be on the same machine as your CI software, or simply relatively close to it (in terms of process, not necessarily network or geographically speaking).
This ENV could be low-footprint and completely run in memory (resource 1) (resource 2), but would not necessarily require the same performance characteristics as your production ENV. (If you want to performance test, this should be handled in a separate environment from your CI anyway.)
Setup
Install a mongod service specifically for CI. If repl sets and/or sharding are of concern (e.g. write concern, no use of $isolated, etc.), it is possible to mimic a clustered environment by running multiple mongod instances (1 config, 2x2 data for shard+repl) and a mongos instance on the same machine with either some init.d scripts/tweaks or something like docker.
Use environment-specific configurations within your application (either embedded via .json files, or in some place like /etc, /home/user/.your-app or similar). Your application can load these based on a node environment variable like NODE_ENV=int. Within these configurations your db connection strings will differ. If you're not using env-specific configs, start doing this as a means to abstract the application runtime settings (i.e. "local", "dev", "int", "pre", "prod", etc.). I can provide a sample upon request.
Include test-oriented fixtures with your application/testing suite. As mentioned in one of the linked questions, MongoDB's Node.js driver supports some helper libraries: mongodb-fixtures and node-database-cleaner. Fixtures provide a working and consistent data set for testing: think of them as a bootstrap.
Builds/Tests
Clean the associated database using something like node-database-cleaner.
Populate your fixtures into the now empty database with the help of mongodb-fixtures.
Perform your build and test.
Repeat.
On the other hand...
If you still decide that not running MongoDB is the correct approach (and you wouldn't be the only one), then abstracting your data store calls from the driver with an ORM is your best bet (for the entire application, not just testing). For example, something like model claims to be database agnostic, although I've never used it. Utilizing this approach, you would still require fixtures and env configurations, however you would not be required to install MongoDB. The caveat here is that you're at the mercy of the ORM you choose.
You could try tingodb.
TingoDB is an embedded JavaScript in-process filesystem or in-memory database upwards compatible with MongoDB at the API level.