Our team is currently working on an Web API that runs in ExpressJS and is deployed via Google Cloud Functions. We've moved beyond our core MVP deliverable and now have time to beef up more of our security/logging features.
Being paranoid (as you should be when developing security) I would like to protect our api from malicious code introduced from one of our npm packages.
I understand that there are many attacks possible if this were to happen. But I would like to protect against the class that send information to the attackers server.
Is there a way to monitor/whitelist all outbound requests? Perhaps by wrapping the Node process itself or utilizing a feature created by Google.
Thank you very much for reading my question, and stay safe!
👋
This is late, obviously, but I still think it's a relevant question that probably many (should!) think about.
We've had a bit of experience with Puresec Function Shield which works for AWS Lambda and apparently also Google Cloud Functions. One of its features is in fact about blocking outbound network traffic.
You should also be able to put your function in a VPC and set its firewall rules to disallow any traffic that is not conformant to your own known ports and sources.
Hope that helps any late-comers into this question.
Related
When reading blog posts about WAFs and Kubernetes, it seems 90+ % of the posts are written by WAF-providers, while the remaining posts seem to be sceptical. So I would like to hear what your experiences are with WAFs, do they make sense, and if so can you recommend any good open-source WAFs? We are currently not allowed to used American cloud providers, as we work with "person data", and the Schrems II judgement has indicated that unencrypted "person data" is not allowed on their platforms (even if on EU servers).
To my understanding WAF help with the following:
IP-whitelists/blacklists
Rate Limits
Scanning of HTTPS requests for SQLi and XSS
Cookie Poisoning and session-jacking
DDOS (requires a huge WAF cluster)
But I would also think that these problems can be handled elsewhere:
IP-whitelists/blacklists can be handled by the Loadbalancer or NetworkPolicies
Rate Limits can be configured in the Ingress
Handling of SQLi and XSS is done by input sanitization in the application
Server-side sessions bound to IPs can prevent poisoning and jacking
DDOS are hard to absorb, so I have no native solution here (but they are low risk?)
Sure, I can see the advantage in centralizing security at the access gate to the network, but from what I have read WAFs are hard to maintain, they have tons af false positives and most companies mainly use them to be compliant with ISO-standards, and mainly in "monitoring mode". Shouldn't it be secure enough to use SecurityPolicies, NetworkPolicies, Ingress Rules and Loadbalancer Rules rather than a WAF?
A WAF is not strictly necessary on Kubernetes — or on any other deployment platform. Honestly, even after consulting for dozens of companies, I've seldom encountered any site that used a WAF at all.
You're right that you could duplicate the functions of a WAF using other technology. But you're basically reinventing the wheel by doing so, and the programmers you assign to do it are not as expert in those security tasks than the developers of the WAF are. At least they are probably doing it as one of many other tasks they are working on, so they can't devote full-time to implementation and testing of the WAF.
There is also a valid argument that defense in depth in computing is a good thing. Even if you have other security measures in place, they might fail. It's worth creating redundant layers of security defense, to account for that possibility.
There's a tradeoff between implementing security (or any other feature) yourself versus paying someone else for their expert work. This is true for many areas of software development, not only a WAF.
For example, it has become popular to use a web application framework. Is it possible to develop your own framework? Of course it is, and sometimes it's necessary if you want the code to have very specific behavior. But most of the time you can use some third-party framework off the shelf. It saves you a lot of time, and you get the instant benefit from years of development and testing done by someone else.
A good waf does a lot more than that, and it is independent of the deployment model (kubernetes or else).
A waf can
Detect and prevent application level exploits far beyond sqli and xss. Sure, you can make a secure application... but can you actually make a secure application? (A team of sometimes changing developers usually cannot.)
Detect and prevent vulnerabilities in underlying layers, like nginx or the OS - or maybe even kubernetes itself.
Provide hotfixing known vulnerabilities until they are actually fixed in the code or patched in the underlying component (like for example preventing certain values for certain parameters you know are vulnerable and so on).
So in short, yes, a waf does make sense with k8s too, in fact it is not dependent on the deployment model. A waf is just a layer 7 firewall that understands http, and can look into traffic to find flaws and prevent exploits.
Update:
For example a recent vulnerability was log4shell, in log4j. In a request it was possible to run arbitrary stuff on servers due to a framework level (3rd party) vulnerability. A good, regularly updated waf would prevent that probably even before you read about the problem.
Spring4shell was a somewhat similar vulnerability in Spring, that can also be prevented by wafs. So could Heartbleed, a vulnerability in openssl.
There was a php vulnerability quite a while ago that involved a magic number, sent as any parameter.
Command injection vulnerabilities in any application or component follow specific patterns, and so on.
A waf also has more generic patterns for usual application vulnerabilities including (but not limited to) sql injection and xss. Sure, your application could be secure and not have these. But especially over time, it will for sure be vulnerable, even the best team cannot produce bug free code, and that applies to security bugs too.
As a web application is usually only accessible through http, ALL of that is available for capture for a component that understands http. All application layer attacks (and that's a lot) will come through http and a waf at least in theory is capable of preventing them. Surely it will not always recognize everything, it's not magic, and again, you could all implement it yourself. But it would be very difficult and time consuming. The same as you would not implement an API gateway or a network firewall, you would want to use a WAF to provide a layer of protection to your application and it's underlying components.
On the other hand, it's true that it takes some time to configure for your specific scenario and application. At first, it will probably produce false positives. Then you can decide how to manage those, you can disable entire rules, or remove certain pages or parameters from checks and so on. It does involve some work, maybe a lot for a very complex application. But when it's configured, it provides an additional layer of protection against threats you may not even have currently, but will in the future.
WAF suggestions:
If you are running managed kubernetes (AWS EKS, Azure AKS and the like) then probably your cloud provider's waf is the best choice due to easy setup and good integration (though I understand that might not be an option for you). I don't know of a good one apart from modsecurity if you are running your own. Naxsi would come to mind, and while I don't have experience with it, its functionality seems very limited compared to other options and what's described above.
WAF &/or API Gateway you may call, play a very vital role in a web application that many developers fail to understand initially.
First and foremost note that its another "out of process" component of your application that assumes all your attack surface
Least it can provide is to play the role as a "Circuit Breaker". For example your main kubernetes based deployment is down, for multiple reasons, this layer can provide some maintenance pages to your users
Further to that, it can provide caching of response, aggregation of responses from different microservices, buffering, prevention of injection types of attacks, centralized request logging, request analysis, TLS termination, Authentication decoupling, TLS translations, HTTP translations, OWASP protection and the list goes on. See this brief video for one reference implementation: link
There is a reason why a web application like Google Search and all other big similar ones rely on a WAF/API Gateway!
I have a single back-end running node/express providing API endpoints and 2 static (react) front-ends. The front-ends interact with the users and communicate with the back-end.
I need to use https through-out once enter production stage.
The front-ends will require their own domain names.
I’ve been thinking on the simplest way to have these configured and have come up with Option 1 (see diagram). Node.js API server running on one VPS and as the front-ends are static sites these can be loaded on separate servers (UPDATE- Mean't to say hosting providers) hence get their own domains. As an option, and I’m unsure if its needed, add cloudflare to the front-end to provide a layer of security.
This will allow front-ends to have separate domain names.
As this is a start-up project and doubt a large number of visitors, I’m wondering if the above is over-engineered and un-necessarily complicated.
So I’m considering Option 2 of hosting back-end api app and the two front-ends on the same linux vps. As the front-ends are static, I added the front-ends into the public folder of node.js. This allowed me to access the front-ends as http://serverIP:8080/siteA
As I want to access front-end as http://siteA.com I’m assuming I require a reverse proxy (nginX)
The questions to help me decide between the two options are:
For a start-up operation and given he above scenario which option is best ?
I understand that node.js requires a port number regardless to work, for the API I don’t mind having a port number (as its not applicable for end users i.e. http://10.20.30.40:3000), however the two front-ends require their own domain names (www.siteA.xom, www.siteB.com), therefore will I need to employ a reverse proxy (nginX) regardless if they are static sites or not ?
I’m concerned that someone could attack API end-points (http://10.20.30.40:3000). In this case, is it true with Option 2 is safer than 1 - that I could potentially block malicious direct API calls as all sites are hosted on the same VPS and the API can be easily be secured, this is not exposed to the outside world?
My developer once upon a time told me that option 1 is best as nginX adds un-needed complication, but not sure what he meant – hence my confusion, to be honest I don’t think he wanted to add nginX to the server.
I’m looking at a high-level guidance to get me on the right track. Thank
This is - as you have also doubted - unnecessarily complex, and incorrect in some cases. Here's a better (and widely used across the industry) design. I'm strongly recommending to drop the whole VM approach and go for a shared computing unit, unless you are using that machine for some other computation and utilizing it that way is saving your company a lot of money. I strongly doubt this is the case. Otherwise, you're just creating problems for yourself.
One of the most common mistakes that you can make when using Node.js is to host the static content through the public folder (for serious projects) Don't. Use a CDN instead. You'll get better telemetry depending on the CDN, redundancy, faster delivery, etc. If you aren't expecting high volumes of traffic and performance of delivering that static content isn't outrageously important at the moment, you can even go for a regular hosting server. I've done this with namecheap and GoDaddy before.
Use a direct node-js shared - or dedicated depending on size - hosting for your app and use CI/CD to deploy it. You can use CNAMEs to map whatever domain name you want to have your app on (ex: https://something.com) to map to the domain name of the cloud-hosting provider url for your app. I've used multiple things, Azure, Heroku, Namecheap for the apps and primarily Azure DevOps to manage the CI/CD pipeline, although Jenkins is super popular as well. I'd recommend Heroku - since it provides a super easy setup.
When designing any API on HTTP, you should assume people will call your API directly. See this answer for more details: How to prevent non-browser clients from sending requests to my server I'm not suggesting to put something like CloudFlare, but you may be overthinking it, look into your traffic first. Get it when you need it. As long as you have the right authentication / authorization mechanism in place, security of the API shouldn't be a big problem on these platforms. If you deploy it on one of these platforms, you won't have to deal with ports either. Unless you reach absolutely massive scale, it will definitely be cheaper for you operate with high-reliability this way.
You won't need to deal with nginx anymore.
I am working on implementing a microservices-based application using node.js. While searching for examples on how to implement the api gateway, I came across the following article that seems to provide an example on implementing the api gateway: https://memz.co/api-gateway-microservices-docker-node-js/. Though, finding example for implementing the api gateway pattern in node.js seems to be a little hard to come by so far, this article seemed to be a really good example.
There are a few items that are still unclear and I am still have issues finding doc. on.
1) Security is a major item for the app. I am developing, I am having trouble seeing where the authentication should take place (i.e. using passport, should I add the authentication items in the api gateway and pass the jwt token along with the request to the corresponding microservice as the user's logged in information is needed for certain activities? The only issue here seems to be that all of the microservices would need passport in order to decrypt the jwt token to get the user's profile information. Would the microservice be technically, inaccessible to the outside world except through the api gateway as this seems to be the aim?
2) How does this scenario change if I need to scale to multiple servers with docker images on each one? How would this affect load balancing, as it seems like something would have to sit at a higher level to deal with load balancing?
I can tell that much depends on your application requirements. Really.
I'm now past the 5 years of experience in production microservices using several languages going from medium to very large scale system.
None of them shared the same requirements, and without having a deep understanding of what you need and what are your business (product) requirements it would be hard to know what's the right answer, by the way I'll try to share some experience to help you get it right.
Ideally you want the security to be encapsulated in an external service, so that you can update and apply new policies faster. Also you'll be able to deprecate all existing tokens should you find a breach in your system or if someone in your team inadvertedly pushes some secret key (or cert) to an external service.
You could handle authentication on each single service or using an edge newtwork tool (such as the API Gateway). Becareful choosing how to handle it because each one has it's own privileges:
Choosing the API Gateway your services will remain lighter and do not need to know anything about the authentication steps, but surely at some point you'll need to know who the authenticated user is and you need some plain reference to it (a JSON record, a link or ID to a "user profile" service). How you do it it's up to your requirements and we can even go deeper talking about different pros and cons about each possible choice applicable for your case.
Choosing to handle it at the service level requires you (and your teams) to understand better about the security process taking place (you can hide it with a good library) and you'll need to give them support from your security team (it's may also be yourself btw you know the more service implementing security, the more things you'll have to think about to avoid adding unnecessary features). The big problem here is that you'll often end up stopping your tasks to think about what would help you out on this particular service and you'll be tempted to extend your authentication service (and God, unless you really know what you're doing, don't add a single call not needed for authentication purposes).
One thing is easy to be determined: you surely need to think about tokens (jwt, jwe or, again, whatever your requirements impose).
JWT has good benefits, but data is exposed to spoofing, so never put in there sensitive data or things you wouldn't publicly share about your user (e.g. an ID is probably fine, while security questions or resolution to 2FA would not). JWE is an encrypted form of the spec. A common token (with no meaning) would require a backend to get the data, but it works much like cookie-sessions and data is not leaving your servers.
You need to define yourself the boundaries of your services and do yourself a favor: make each service boundaries clean, defined and standard.
Try to define common policies and standardize interactions, I know it may be easier to add a queue here, a REST endpoint there, a RPC there, but you'll soon end up with a bunch of IPC you will not be able to handle anymore and it will soon catch your attention.
Also if your business solution is pretty heavy to do I don't think it's a good idea to do yourself the API Gateway, Security and so on. I'd go with open source, community supported (or even company-backed if you have some budget) and production-tested solutions.
By definition microservice architectures are very dynamic, you'll fight to keep it immutable between each deployment version, but unless you're a big firm you cannot effort keeping live thousands of servers. This means you'll discover bugs that only presents under certain circumstances you cannot spot in other environments (it happens often to not be able to reproduce them).
By choosing to develop the whole stack yourself you agree with having to deal with maintenance and bug-discovery in your whole stack. So when you try to load a page that has 25 services interacting you know it may be failing because of a bug in: your API Gateway, your Security implementation, your token parser, your user account service, your business service A to N, your database service (if any), your database load balance (if any), your database instance.
I know it's tempting to do everything, but try to keep it flat and do what you need to do. By following this path you'll think about your product, which I think is what's the most important think to do now.
To complete my answer, about the scaling issues:
it doesn't matter. Whatever choice you pick it will scale seamlessly:
API Gateway should be able to work on a pool of backends (so from that server you should be able to redirect to N backend machines you can put live when you need to, you can even have some API to support automatic registration of new instances, or even simples put the IP of an Elastic Load Balancer or HAproxy or equivalents, and as you add backends to them it will just work -you have moved the multiple IPs issue from the API Gateway to one layer down).
If you handle authentication at services level (and you have an API Gateway) see #1
If you handle authentication at services level (without an API Gateway) then you need to look at some other level in your stack: load balancing (layer 3 or layer 7), or the DNS level, you can use several features of DNS to put different IPs to answer from, using even advanced features like Anycast if you need latency distribution.
I know this answer introduced a lot of other questions, but I really tried to answer your question. The fact is that you need to understand and evaluate a lot of things when planning a microservice architecture and I'd not write a SLOC without a very-written-plan printed on every wall of my office.
You'll often need to go mental focus and exit from a single service to review the global vision and check everything is going fine.
I don't want to scare you, I'm rather trying to make you think to succeed.
I just want you to make sure you correctly evaluated all of the possibilities before to decide to do everything from scratch.
P.S. Should you choose to act using an API gateway be sure to limit services to only accept requests through it. On the same machine just start listening on localhost, on multiple machines you'll need some advanced networking rule depending on your operating system.
Good Luck!
I am looking to create a simple mobile agent system which will deal with 4 tasks, i.e 4 different mobile agents jobs: Database update, meeting scheduling, network services discovery and kernel update.
I have done my research and have seen different frameworks such as Aglet, Jade, agent builder etc. My question is which one should i use? Also i need to setup the base code for it to work, can someone point me to a site or help me to setup the basic functions of the mobile agent?
I've read about tahiti server for the Aglet model. I'm quite confused about how to set up the mobile agent system. Any help would be much appreciated.
I have also tried to it using RMI. I had created a method of type agent, but i couldn't pass it through remote method implementation. I was reading about tcp and udp socket programming. I was thinking may be it would be more fair to do it using socket programming. In this case, would this be called an agent? I was thinking about the server sending datagram packets to multiple clients.
You need to ask yourself why you want to use mobile agents at all. The notion of a mobile agent was popular in the agent research community in the early 90's, but fell out of favour because (i) it wasn't clear what problem it was solving, (ii) the capability to allow arbitrary code to migrate to a particular computer and execute with enough privileges to access local data and services is very open to abuse, and (iii) all of the claimed benefits of mobile agents can actually be achieved though web services (REST or otherwise) and open data formats such as RDF. Consequently, few, if any, mobile agent platforms have been properly maintained since the early experiments.
It also sounds as though you need to be clear which end-user problem you want to solve. Scheduling a meeting and updating my kernel are very different tasks - I'd be very uncomfortable with a program that claims do both. If your interest is in the automation of system maintenance tasks, such as DB tuning and kernel patching, on large networks you might want to look at the SmartFrog project, or read up on autonomic computing.
I use JADE and I agree with the first guy, agent systems usually take alot of overhead to going so if you can avoid it, please do. If however you choose to proceed choose a platform with alot of support and a big user group.
Jade has some neat features like a directory facilitator DF, which works like a yellow pages so other agents don't have to know what agents are running and what services are supplied they can simply inquire by the DF.
Also JADE ContractNetBehaviours help simplify communication.
We're trying to implement the Gatekeeper Design pattern as recommended in Microsoft Security Best Practices for Azure, but I;m having some trouble determining how to do that.
To give some background on the project, we're taking an already developed website using the traditional layered approach (presentation, business, data, etc.) and converting it over to use Azure. The client would like some added security built around this process since it will now be in the cloud.
The initial suggestion to handle this was to use Queues and have worker roles process requests entered into the queue. Some of the concerns we've come across are how to properly serialize the objects and include what methods we need run on that object as well as the latency inherent in such an approach.
We've also looked setting up some WCF services in the Worker Role, but I'm having a little trouble wrapping my head around how exactly to handle this. (In addition to this being my first Azure project, this would also be my first attempt at WCF.) We'd run into the same issue with object serialization here.
Another thought was to set up some web services in another web role, but that seems to open the same security issue since we won't be able to perform IP-based security on the request.
I've searched and searched but haven't really found any samples that do what we're trying to do (or I didn't recognize them as doing so). Can anyone provide some guidance with code samples? Thanks.
Please do not take this the wrong way, but it sounds like you are in danger of over-engineering a solution based on the "requirement" that 'the client would like some added security'. The gatekeeper pattern that is described on page 13 of the Security Best Practices For Developing Windows Azure Applications document is a very big gun which you should only fire at large targets, i.e., scenarios where you actually need hardened applications storing highly sensitive data. Building something like this will potentially cost a lot of time & performance, so make sure you weigh pro's & con's thoroughly.
Have you considered leveraging SQL Azure firewall as an additional (and possibly acceptable) security measure? You can specify access on an IP address level and even configure it programmatically through stored procedures. You can block all external access to your database, making your Azure application (web/worker roles) the only "client" that is allowed to gain access.
To answer one of your questions specifically, you can secure access to a WCF service using X.509 certificates and implement message security; if you also need an SSL connection to protect data in transit you would need to use both message and transport security. It's not the simplest thing on earth, but it's possible. You can make it so only the servers that have the correct certificate can make the WCF request. Take a look at this thread for more details and a few more pointers: http://social.msdn.microsoft.com/Forums/en-US/windowsazuresecurity/thread/1f77046b-82a1-48c4-bb0d-23993027932a
Also, WCF makes it easy to exchange objects as long as you mark them Serializable. So making WCF calls would dramatically simplify how you exchange objects back and forth with your client(s).