I am trying to build a service where anybody can send an image file from an email address/client and process it. Think about the service a bit like Flickr showing the image in a dashboard that comes via emails
From a usability standpoint this mechanic offers great deal of advantage but I want to understand the security consequences of such an action.Some concerns are:
I need to validate all these files as images
People can probably send a file with an exploit/code that can likely
be a problem. But in my case I am mostly going to do a file open and
save and let the browser show the image
Am I taking the right approach here? Are there serious consequences that I should be of?
Things you should do and take into consideration.
Make sure your mail server is configured for virus scanning, keep it up to date. That'll be the first line of defense.
When the email comes in, attempt to process the image in a known rock solid library.
Be aware that many emails contain multiple images, some of which may have nothing at all to do with the one they are sending. For example, our company emails all include our logo at the bottom. I'm not exactly sure what the solution is here, but you'll want to take it into consideration.
Different email clients handle image attachments, well, differently. Sometimes it's as a normal attachment, sometimes it's embedded in the body. Even within the same client an image might be handled differently depending on if they sent the email as plaint text with attachments or HTML mail.
People will test your system. They'll send .js files, they'll send images whose headers are jacked in order to overflow your image processing library...
Consider enforcing certain email restrictions such as SPF checks.
Be prepared to receive images that are absolutely huge. Today's cameras take very large photos and a lot of people don't know what crop or resize means. You might consider setting a cap of 15MB or larger per email coming into your server. Then, in combination with #2 above, auto resizing images down to something a bit more acceptable.
Determine the mechanism you actually want to use to notify the user of any issues. Bear in mind that this mechanism is subject to abuse. For example, consider a spam message sent to your machine with reply-to headers going to a victim.
If you are using .net, see this for a possible way to confirm a file is an image: How can I determine if a file is an image file in .NET?
I'm not saying this is 100% secure (can you ever be 100% secure?) but here is something that you can try:
Lets say that you have an alias on your postfix (or whatever mail system) that redirects incoming emails to a php/bash/python script for further processing.
The first thing I would do is use an image manipulation library (say imagemagick) and convert all incoming files to a .png format or whatever, and only proceed further with your logic if the conversion is successful.
This way, if someone sends you any malicious attachments (php exploit, jar's, swf's, anything) the conversion will fail, and hence it will be disregarded by your system.
Edit: ImageMagick has the "identify" command which does exactly what you want.
Emails could be easily spoofed as well, which means I can send an email from an email address which doesn't belong to me.
This might help also: Secure way to upload image in PHP ...
Related
I have a bunch of emails which are written using MJML. The output HTML seems to be correct.
As an email sender, I'm using AWS SES. But after sending, all my CSS styles in an email are broken. To send an email, I'm using method ses.sendEmail.
I have tried to use CSS injected in HTML head and inline styles. Does not work.
What could be the problem with it? Thanks!
First, check your HTML. Send it through either of these free sites: https://www.putsmail.com (requires a free Litmus account; they're an established and reputable Email Service Provider, ESP) or https://useparcel.com/ (an editor for email HTML; put your HTML in the editor and use their free send feature). With putsmail, don't accept their offer to inline your CSS for you--that'll change your HTML.
When each email arrives, compare the HTML received to what you sent. Diff tools are helpful.
I just sent a 1300+ line HTML file through each of those. For both, every character arrived unchanged, except for blank lines after the </html>. That's okay with me!
Compare the rendering to what you see in one of the MJML rendering tools (maybe https://mjml.io/try-it-live or the MJML desktop app https://mjmlio.github.io/mjml-app/). Do not depend on the rendering in any one email client as fully representative of the HTML; most of them have quirks. (I'm lookin' at you, all Outlooks except Apple's! And GMail. And Yahoo!. More at https://caniemail.com/ .)
I did a little research on AWS ses.sendmail. There's important advice and information at both of these.
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-email-sendmail.html
https://docs.aws.amazon.com/ses/latest/APIReference/API_SendEmail.html
I found an SO ticket from a little less than a year ago that could be relevant.
AWS Amplify SES sendmail AccessDenied
Recommend you review all that. Maybe something there will improve your experience.
Last, some standard advice from the world of email HTML. If you get sendmail working, perhaps it doesn't apply.
Send your HTML using an ESP that doesn't change your HTML. For people sending lots of email, there are lots of good reasons for using an ESP. Many provide valuable services. By the same token, some ESPs change the HTML before sending; lots of us avoid them.
I hope you get it figured out. Email HTML is, as you're experiencing, anything but easy.
I have an app that sends SMS's out to a bunch of people. Those messages contain links. They are not using any link shorteners or any other service. They link back to my site. The links themselves are randomized strings, which are stored in my db, which are associated with an action. (Click "yes" or "no" link and the db tracks what you chose.) For ALL users, this works perfectly. With one user - and it's always the same user, as soon as the cron job runs, which triggers this event, his "vote" comes in. This is without him clicking or even seeing the message sometimes.
So, the question: has anyone ever seen or heard of a cell provider or a messaging app or similar that "clicks" links as part of some process before sharing the content with the user? I can't see ANYTHING in the code that would single him out so I'm thinking it has to be something in between when the message goes out and he does what he does. Especially because the timestamp is also always within seconds of the cron job running.
Sending an SMS can sometimes go through multiple carriers before reaching an end destination. As such, providers may be "handling" the content in this case.
The best thing to do would for any cases of this in the future would be to write support for further investigation.
We are developing a big data solution in which one requirement is to process incoming emails. The technology stack is not finalized yet but mostly we might go with Sendmail as MTA and Procmail as MDA. We are open to any other very efficient solution.
These emails are essentially carry data in attachments and are not meant for end user, so the email flow ends with Spark processing.
My first thought was it would be great if there was a message queuing system such as Apache-Kafka which could accept emails as messages and then provide them to the client such as Spark on demand but it seems that sort of technology/approach is not available in any of the message brokering systems.
This means we would have to receive emails via SMTP MTA and then extract the information from the MDA.
We could use Procmail to extract the contents of the email and the attachments and put them in a folder per email and then scan the folders and process them in spark.
Alternatively if Spark has any plugins which could pull in emails from an MDA and break it down into it's attachments it would make life much simpler.
If there is any other smarter solution it would be welcome.
So the fundamental question is what technology is available for channelizing emails through Spark for processing. Connectors etc.
Mailgun or Sendgrid incoming email processing is so easy that I could hardly imagine any alternative for a new, especially big, system. I only played with them, but my impression was that my any actual or potential (billions of emails) problem related to emails is solved for good. Not related to Spark, those system just post email content as http POST request to a URL you provide.
Sendgrid used to incorrectly parse encoding, their support ignored my emails and eventually deleted a ticket without solving the problem. Mailgun always returns UTF8 regardless of original encoding. Manual MIME parsing is such a grandiose task itself so it is better to use existing solutions, unless emails are generated by a computer. But even then, IaaS services are so much cheaper than developer time.
Okay maybe iam getting blind or overlooking something.
Since you can set a size limit on the attachments in the Inotes client i thought you could do this for the Notes client as well.
Going through the server settings, policies and so on i came to the conclusion that it is not possible other than adding lotusscript to the querysave of the memo, appointment, (more) forms.
And iam not really a fan of that, it is a domino template, not a custom one. And i foresee some issues. Not that i can think of one right now.
And making a mail rule won't do either.
I could put the code in the onchange or validation event of the rich text elements. Again its the mail template.
So the big question is.....
Is there some way to do this with a policy or any other way than tampering with the mail template.
thx
--->The answer of ken made me realize my question was not complete.
to catch all the big mails we are planning to use the filesendr app an addon you can find them on openntf.
In our test env. it works like a charm. But people can still send large attachments, filesendr will kick in but in the senders his mailfile the memo with attachment will be stored, or in DAOS when installed.
We want to limit the size of the attachments allowed, without hacking the template.
If you insist on doing this on the client-side, you have two choices: modify the template, or write a plug-in that uses the C API Extension Manager feature to intercept NSFNoteUpdate operations and inspect the size of attachments.
You cannot set a limit on the size of an attachment that gets added to a Notes email or document. A user can attach a file of any size to a document, including an email document. Perhaps using database quotas could limit that to a degree, but that's not the granularity you're looking for.
However, at the server you can set a limit on the maximum message size, and the Notes Router will fail a message if it is above that size (i.e. send a non-delivery report). Follow these steps to find the setting:
Go to the Domino Directory on the server,
Click Configuration > Servers > Configurations.
Select the configuration doc for your server.
Click Router/SMTP > Restrictions and Controls > Restrictions
Set the Maximum message size field to some value, say 10,000KBs
That will prevent any message larger than 10MBs from going through the router.
We use Lotusnotes 6.5 as email client. We wil have around 1600+ mails for 9 hrs. If a mail not checked , we have face serious issues with our client. Can any script can be written to check whether all mails are checked and replied?
Update:
We have already tried moving the mails to another folder.But has this mailbox handled by team of persons, we noticed lot of human error happening like moving a unread mail, sometimes they would have read mail but forget to reply it etc.etc.
So I was looking out for a script solution, will your other options. Also one more thing we do is we cc our mailbox mail id for all outgoing emails to have a track of all replied mails, will this could help in any way to find out which mails was missed?
If you need to track unread marks, I second the aforementioned nsftools solution, which works in Domino 7.x too. However, this is very much Notes ID-dependent. A folder would be better.
Note that 6.5x is well out of support, and that Domino 7.x officially died this week: use something at least vaguely modern!
There's an easier non-programmatic way. Just move the email from the inbox into another folder once the email has been responded to. That is more reliable than any programmatic solution, and keeps your inbox tidy (which will certainly be necessary if you get nearly 200 emails per hour!)
That said, here are some other ideas.
Determining if the document was read
Unread marks are not your friend here, unless you'll be accessing the mail file from the same client. Also they tend to get out of sync and would likely prove unreliable at some point, especially given the number of incoming emails. Instead you'll need to have some information that is saved within the individual mail document, such as the last accessed property or a custom item you manage via scripts/formulas.
You can see if an email has been read by checking the Last Accessed property of the mail document. According to IBM's technote (https://www-304.ibm.com/support/docview.wss?uid=swg21086670), the property will be updated when the document is read.
You could write a script in the QueryOpen event that stamps a value on the document and saves it.
Determining if the email was responded to
First off, I'd suggest you save all sent emails in case you need a record of what was sent to the client. That won't give you a way to see which emails have not been responded to, however.
Instead you could add script to the reply action within the memo form. When someone click's reply it could update the current memo, stamping an item on it to say who replied and at what time, for instance. Then you can create a view to show any emails that don't have that item, and another view to show emails that do grouped by who responded. The second view could even show how many emails each person responded to, something that might be used as a measurement of performance perhaps.
"Unread mark" checking is not exposed in the API.
I did find 2 links, this one is a basic implementation, where as this link does have more robust code and is implemented as an object in LotusScript. It should be compatible with Notes 6.5+.
I found the second link through nsftools website which has lots of great snippets that solve various problems. You should at least be able to detect if a mail has been read or not. Note that it requires making API level calls. You should be able to create a new script library and copy/paste the code into it.