Using fuglu plugin and python, trying to parse thru' the suspect, i am trying to get the envelope recipients. eg. Sometimes I can send TO: members#domain.com, xyz#anotherdomain.com
How does fuglu decide who is the envelope recipient when members#domain.com is a group mail id
Postfix has already translated the group to individual ids on incoming relay server before it reaches the outgoing server with this fuglu plugin.
I need to access the envelope recipient address only. How can I get this value in fuglu?
In fuglu, you can get the list of envelope recipients using suspect.recipients.
Now from your question it seems like a message may previously have been sent to group#example.com and this group adress then rewrites to recipient1#example.com, recipient2#example.com, ...
Which one of these adresses fuglu sees in suspect.recipients depends on your mailserver/routing configuration. I understand that fuglu in your case runs on a second mailserver where the alias has already been expanded, so suspect.recipients will likely not contain group#example.com .
It may contain either all or only some of the expanded adresses if the server is configured to handle multiple recipients in the same transaction or if the mailserver is configured to split up the recipients ( destination_recipient_limit in postfix for example).
If your goal is to extract the original recipient address (group#example.com), this is actually quite hard, as this information may not even be available in the SMTP transaction anymore. Some servers write it into a received header:
Received: from [...]
by your.relay.example.com with SMTPS id blabla
for <group#example.com>
and in this case you could extract it from there with some regex magic. But if your inbound server does not add the for <original recipient> part you're out of luck. In this case you could fall back to extracting the To: header from the message using suspect.get_message_rep().get('To') ), which likely contains the original envelope recipient, but unfortunately this is not guaranteed.
Related
I am using a PHP script to send emails using PHPMailer (on an ovh web server).
With the following script :
$mail->isMail();
......
$mail->addReplyTo('postmaster#mydomain.fr', 'Site');
$mail->SetFrom('postmaster#mydomain.fr', 'Site');
Emails get delivered to my gmail account BUT not to other addresses (e.g. my Business email).
With the following script :
$mail->isMail();
......
$mail->addReplyTo('postmaster#mydomain.fr', 'Site');
$mail->Sender = 'postmaster#mydomain.fr';
Emails get delivered to other addresses (e.g. my Business email) BUT not to gmail.
Also the sender appears as
bounce-id=D004=U58180. …… .net
which is not so friendly.
Is there a way to solve those issues ?
Using setFrom by itself will result in Sender being set to the same value as your from address (look at the code in the setFrom method). So in your second example you're setting the envelope sender, but not the from address, which is likely not what you want.
You may find it easier to use SMTP to localhost instead of isMail(); just changing that method to isSMTP() should be enough to do that. That way you can set SMTPDebug = 2 and see exactly what your local mail server is saying.
Separately, you don't need to set a reply-to address if it's the same as your from address – PHPMailer will simply ignore it if you try to set them to the same value anyway.
I've been working on an integration that sends an envelope to 2 signers. I am trying to test and program for the case of 1 of the signers having a bad email address. I would need to:
Be notified of this (webhook?)
Update the email address for the signer
Update the email address in the custom fields as that's printed on the agreement
Resend the envelope
When I enable the delivery failure notification in Connect (this is set up at the account level, for now), I don't see where the failure is noted in the webhook response or for which signer the error is happening.
I've found multiple links via Google talking about "correct and resend" but I haven't been able to get anything definitive in terms of what steps need to be taken or a (PHP) code example. I've also found "resend" in the API reference but still no code examples.
I also haven't seen any code examples (PHP) of how I would update a signer's email address and how that would work via the API.
It'd also be good to know how to update custom field data for both signers so the email address in the contract is updated (it's listed separately as part of the custom form data).
And finally, how would I resend the envelope?
Thank you for your thoughts.
Per DocuSign support (customersupport#docusign.com):
Be notified of this (webhook?)
I have updated your account settings and now you should be able to see AutoResponded in your webhook messages when email invalid or unreachable.
(my comments: so it sounds like you have to ask to be able to see this stuff, it's not part of their standard webhook responses)
Update the email address for the signer
I also haven't seen any code examples of how I would update a signer and how that would work.
Unfortunately we don't have an example for recipient update but the logic is to get list of all recipients, pick the one need to be updated, and update.
You can use below to retrieve all recipients per envelope
https://developers.docusign.com/docs/esign-rest-api/reference/envelopes/enveloperecipients/list/
GET /restapi/v2.1/accounts/accountId/envelopes/envelopeId/recipients
Extract the one you need to update edit email and update using the below
https://developers.docusign.com/docs/esign-rest-api/reference/envelopes/enveloperecipients/update/
PUT /restapi/v2.1/accounts/accountId/envelopes/envelopeId/recipients
You can also see example code in C# how to do this in this thread https://github.com/docusign/docusign-csharp-client/issues/307
Update the email address in the custom fields as that's printed on the agreement
There is bug case created ( above link ) regarding this. At the moment you can not update signer custom fields case EC-2944.
Resend the envelope
You can resend the envelope during recipient update
https://developers.docusign.com/docs/esign-rest-api/reference/envelopes/enveloperecipients/update/
PUT /restapi/v2.1/accounts/accountId/envelopes/envelopeId/recipients
My commentary:
Since custom fields can't be updated, this whole use-case goes out the window. We can't have contracts with incorrect data on it, so my client will have to do these corrections by hand. Kind of a bummer.
At some messages in Gmail, the MESSAGE-ID header is missing and I need to use it as IN-REPLY-TO header when replying so I can keep the replies at the same conversation.
When those emails with missing MESSAGE-ID header are replied from the Gmail's WEB UI, the replies stay with the conversation and the reply in fact acquires a IN-REPLY-TO header something looks like <-4185615914882731559#unknownmsgid> I could not discover how is this generated.
I've been trying to find a solution to this problem quite a while, ended up trying a few desktop email clients, Airmail and Sparrow to see how they behave. If I ever reply a particular email which is missing the MESSAGE-ID header using those email clients, the sent reply also acquires the same IN-REPLY-TO header <-4185615914882731559#unknownmsgid> and the reply indeed stays at the same conversation as expected.
So, I would like to ask how they achieve this, do they somehow generate the IN-REPLY-TO header <-4185615914882731559#unknownmsgid>? Is there a rule or a workaround?
This is a classic problem, every mail reader has it. The classic solution is:
copy the References field from the message you reply to, if any.
make up an In-Reply-To field, perhaps with a made-up message-id, perhaps with the sender's address and the message's date.
Keep the subject, and append "re: ".
You may also copy the Thread-Index field, if there is one.
Gmail's and most other threading algorithms will then connect the dots.
Thread-Index is a Microsoft thing, noone else uses it, as far as I know. The format of the hex blob is not specified, but copying that will help probably help Exchange, if the original message comes from Exchange.
If you just want to send an email and make sure it threads with another email in that user's inbox (and not any other gmail users that may receive the reply) then I believe you can use the Gmail API's messages.send() method and set the message.threadId to match the thread you want it to thread with.
I don't think that e-mails delivered to your gmail account have no message id. My evidence is that I just tried sending one to myself via telnet without a message id, and gmail's mx server added this
Message-Id: <54412844.63bbb40a.0772.ffffcbdcSMTPIN_ADDED_MISSING#mx.google.com>
It may be that copying a message into gmail's imap store can result in a missing message id. Eg see https://productforums.google.com/forum/#!topic/gmail/w5kgRivwbIg (though that is very old, it shows that synchronising mail to a gmail IMAP account can have this result).
The basic rule of e-mail is that if an e-mail doesn't have a message id for whatever reason, one is created for it, which is what has happened here. Hence, when you reply, the made-up message id is used in the in-reply-to header.
I am using the nodemailer SMTP node module, a simple way to send emails through SMTP.
I am using this to send out text messages to various cell carriers, which will allow you to send a text message for free via email. An example email, which will text the Sprint phone (310) 987-6543:
3109876543#messaging.sprintpcs.com
These are all going out through one SMTP email address. This means all text replies from someone's phone will channel back to my email address.
A lot of people send these texts. I want to route the reply text back to the right person, using my own server side code.
The problem is distinguishing where the text came from. Ideally I could hide my own primary key ID for the sender in some sort of SMTP header, which would arrive at the recipients phone and come straight back in their text reply.
I don't really know anything about SMTP or email protocols - is this possible?
edit
As given in each of the answers below, in better email servers you can address to an email address as well as an extension to that address, such as:
// original email
sms-service#mysite.com
// extension
sms-service+a6e1087b#mysite.com
This would be a correct answer and would solve my problem, if only this worked in Exchange Server, which is the email server I am using. I don't know if there is a way to enable this or any other workaround.
Exchange server doesn't support plus addressing natively but it is possible do achieve this with a custom MTA, e.g. http://durdle.com/regexcatchall/
Source: https://serverfault.com/questions/195276/does-exchange-support-plussed-users-e-g-mynamespamexample-com-or-a-similar
Here are a few approaches that you might be able to use:
Unique from addresses
You could you use unique "from" or "reply-to" address for each each message (or for each recipient). For example "sms-service+a6e1087b#mysite.com". In that example, the hex would be a unique identifier that can be mapped to the conversation in your database.
Gmail supports such a scheme: emails targeted to [foo]+[whatever]#gmail.com will be delivered to [foo]#gmail.com and the receiver will know also get the [whatever] back. (more on this http://lifehacker.com/144397/instant-disposable-gmail-addresses)
Identify the user, not the conversation
One option is to not identify the conversation, but identify the phone number the reply comes back from instead. In many cases, the number from where the SMS reply was sent back from is enough to connect that message to the conversation. I'm sure the SMS gateway you're using will provide you with the number from where the reply came from.
Commercial SMS gateway
You can also opt to use a commercial SMS gateway (smsglobal.com comes to mind, though there are many others). These services have well defined APIs for sending a receiving replies. This naturally comes at a price, though. But along with the price, you do get more confidence as to the reliability of the service. A free SMS gateway might get shut down without much warning.
This common problem is traditionally implemented using the semantics of From: and Reply-To: headers of e-mails. This is in fact not (directly) related to SMTP, but rather to ARPA network text messages.
RFC 5322 is the current specification of these messages. It states:
When the "Reply-To:" field is present, it indicates the address(es) to which the author of the message suggests that replies be sent. In the absence of the "Reply-To:" field, replies SHOULD by default be sent to the mailbox(es) specified in the "From:" field unless otherwise specified by the person composing the reply.
So you simply need to encode the information you need into the Reply-To: header. Since you mention your own server-side code, you might be running an SMTP server yourself. In this case, you can put any address that will be routed to your SMTP server and encode the right person's address there.
Alternatively, if you rely on another SMTP server, you might want to use aliases. Postfix, and mail services such as GMail, typically allow address extensions, i.e. addresses of the form user+foo#domain.com, and you can encode the necessary information in the address extension.
For example, let's say that alice#domain.com wants to send a text to Sprint phone (310) 987-6543 through your service, you could send an e-mail to 3109876543#messaging.sprintpcs.com with:
Reply-To: alice%40domain.com#yourservice.com
Alternatively:
Reply-To: yourservice+alice%40domain.com#gmail.com
This will allow you to route the reply properly by looking at the destination of the reply e-mail.
What is the exact difference between the envelope and the email in smtp?
Why does the protocol need the envelope? In snail mail, the letter needs not contain addresses and is not visible to the postman (at least that's what you want to believe), so all the routing is made entirely based on the envelope and that is clear to me.
However in e-mail, the letter itself is parseable and has headers such as From: and To:. Why is that not enough to route email? Why do we need an envelope?
And what is the meaning of a divergence between the envelope and the email itself? Does it necessarily mean that someone is trying to game the system, or is it, under certain circumstances (what circumstances?), legal to have a difference between them.
Your clarifications are more than welcome.
Puzzled email receiver.
The recipient address in the SMTP RCPT TO: command is what mail transports use to determine the actual recipient of an email. The addresses in the To: and Cc: headers are there for mail readers to display to users who the recipients are, but they're not actually used by mail transports.
Most mail clients set the same addresses in SMTP RCPT TO: and MAIL FROM: commands that they insert in To:, Cc:, and From: headers, so the "envelope" addresses will be the same as the addresses in the headers.
Envelope and header addresses will usually agree for most legitimate mail. Notable exceptions are Bcc: addresses and mailing lists.
Spammers often forge header addresses to try to avoid spam filters.
The bottom line is that the overhead would be too great. When mail servers are processing millions of messages per day, it's much faster during the SMTP session (The Envelope) when the mail server can process individual commands.
If you didn't use an Envelope, you would have to push the entire message at the SMTP server, before you could find out if the message needs to be rejected. Some messages are quite large (I've seen 100+Meg messages working with GIS images).
By using an Envelope, we can issue specific 1 line commands against the mail server, and the mail server will tell us if we are successful or not, right then and there.
The envelope is used by the SMTP server and the message headers are used by a mail reader as everyone else has said.
What has not been said is that the RCPT TO: is used to route the message to a specific user regardless of where the user's name appears in the headers. The user does not necessarily need to appear in the To: or even the Cc: headers. Think of a Bcc: where the only thing that the receiver knows is who the message was from. In this case, the To: and CC: headers should be empty - hence the blind part of BCC. In another case, if an email message has the user mentioned in the CC list along with 10 other users, how can the SMTP routing pick the appropriate user. The answer is that it uses the RCPT line to route the message.
This is also used when routing via mailing lists. The To: header will contain the mailing list e-mail address. An SMTP system will generate separate messages for each user in the list each with a specific RCPT TO: user#host... envelope. In this case, the user's name will not even appear in any of the other headers.
An email message is very similar to a physical letter that you would send in the mail. There is an envelope, with To/From information, and there is the actual letter on the inside, with it's own To/From information. The envelope to/from information is the real information that is used for message delivery, for both email servers and post offices.
When an envelope comes into a post office, they inspect the To address on the envelope, and send it to the correct destination. The post office workers have no knowledge of the letter inside the envelope. The letting inside could have completely different To/From information than the envelope says. The Envelope could say the message is to Bob, but the letter inside may say it's for Alice. Or in real world: The envelope could say the message is to SomeCompanyName, and when secretary open envelope, letter inside say, it's is for Mr. Brown which work in SomeCompanyName.
Source: Email Envelope vs Email Header