IMAP APPEND with gmail, treat as new incoming email - node.js

I have successfully got my nodejs app to IMAP "APPEND" a new message to Gmail, with custom date/time. Everything works fine. But, how do I make Gmail treat it as actual new incoming email? (which needs to be sorted based on existing filters, pushed to spam if needed,etc). Basically all the functions carried out on new incoming email by Gmail. Is there any such event that can be raised on the newly created email, maybe something using its UID? Does Gmail support such functionality through IMAP?
P.S: Using "inbox" npm package. Also tried reading IMAP spec, couldn't understand much. (only got to understand APPEND command part).

No. IMAP Append means 'Put this directly in this folder'. You can use the flags argument of APPEND to make sure you don't mark it \Seen, so it'll look like an unread message.
The only way to make it do all the new email steps and filtering is to send it via SMTP.

Related

Cannot keep messages in the same conversation due to missing MESSAGE-ID header at Gmail

At some messages in Gmail, the MESSAGE-ID header is missing and I need to use it as IN-REPLY-TO header when replying so I can keep the replies at the same conversation.
When those emails with missing MESSAGE-ID header are replied from the Gmail's WEB UI, the replies stay with the conversation and the reply in fact acquires a IN-REPLY-TO header something looks like <-4185615914882731559#unknownmsgid> I could not discover how is this generated.
I've been trying to find a solution to this problem quite a while, ended up trying a few desktop email clients, Airmail and Sparrow to see how they behave. If I ever reply a particular email which is missing the MESSAGE-ID header using those email clients, the sent reply also acquires the same IN-REPLY-TO header <-4185615914882731559#unknownmsgid> and the reply indeed stays at the same conversation as expected.
So, I would like to ask how they achieve this, do they somehow generate the IN-REPLY-TO header <-4185615914882731559#unknownmsgid>? Is there a rule or a workaround?
This is a classic problem, every mail reader has it. The classic solution is:
copy the References field from the message you reply to, if any.
make up an In-Reply-To field, perhaps with a made-up message-id, perhaps with the sender's address and the message's date.
Keep the subject, and append "re: ".
You may also copy the Thread-Index field, if there is one.
Gmail's and most other threading algorithms will then connect the dots.
Thread-Index is a Microsoft thing, noone else uses it, as far as I know. The format of the hex blob is not specified, but copying that will help probably help Exchange, if the original message comes from Exchange.
If you just want to send an email and make sure it threads with another email in that user's inbox (and not any other gmail users that may receive the reply) then I believe you can use the Gmail API's messages.send() method and set the message.threadId to match the thread you want it to thread with.
I don't think that e-mails delivered to your gmail account have no message id. My evidence is that I just tried sending one to myself via telnet without a message id, and gmail's mx server added this
Message-Id: <54412844.63bbb40a.0772.ffffcbdcSMTPIN_ADDED_MISSING#mx.google.com>
It may be that copying a message into gmail's imap store can result in a missing message id. Eg see https://productforums.google.com/forum/#!topic/gmail/w5kgRivwbIg (though that is very old, it shows that synchronising mail to a gmail IMAP account can have this result).
The basic rule of e-mail is that if an e-mail doesn't have a message id for whatever reason, one is created for it, which is what has happened here. Hence, when you reply, the made-up message id is used in the in-reply-to header.

trigger a .sh script when a specific subject email is received

Anyway, I have a script that I want to run whenever I receive an email on gmail. And if possible a subject specific email. is such a thing possible and if so, what programs do I need to allow it.
You can't instruct gmail to trigger an external script for you. I think you've got a few basic choices. In order of increasing difficulty and complexity:
1) Configure a gmail filter to deliver your desired messages to a special folder. Write a script to poll that folder, download (or delete or mark as read) messages it finds there, and then launch your local script. Set up a cron on your local machine to run the script every few minutes. You can poll the folder with IMAP or the GMAIL API. IMAP is probably easier. This will be tricky with shell, you're better of with Python, PHP, or similar.
2) Configure a gmail filter to forward your desired messages to an address on a mail server that you control. Use procmail or similar to intercept the incoming messages and launch your script.
3) Set up an account at Mailgun and configure the emails so they get delivered there directly. (Or forward from gmail as in #2.) Configure Mailgun to launch an API request when it receives messages. Build an API handler to receive the request. Launch your process from your API handler.
I have never done it, but I guess the first thing you should do is to take a look at the Google's Gmail API...
What is the Gmail API?
The Gmail API gives you flexible, RESTful access to the user's inbox,
with a natural interface to Threads, Messages, Labels, Drafts, and
History.
It seems to fit what you want - at least, without knowing the details of what you want to do.
The Gmail API can be used in a variety of different applications,
including, typically:
Read-only mail extraction, indexing, and backup
Label management
(add/remove labels)
Automated or programmatic message sending
You can use several programming languages - maybe the trick is using your programming language of choice to write a wrapper for the .sh script... I hope this helps!

GMail email import

I would like to move all of my ARCHIVE incoming and outgoing mails to GMail. I have admin rights on the server, where all my emails are stored, so I may set up Imap (current case), POP3 or even I write an app, if necessary. If it is possible, I would like to preserve also folder sturcture (of course, as labels). Is there any way to solve this problem?
ThX,
Fx
Just a thought: You can use a third party software such as Thunderbird to fetch all the emails and then simply drag and drop into the Gmail account also connected to the same Thunderbird client. I have found that Thunderbird seems to preserve file structure and labels, and auto creates more if they don't exist.
Worth a shot!

Link to individual mails in gmail

I want to download all emails in a gmail account and also want to get the unique url which will open the exact mail in gmail, off course with authentication. I tried using javax.mail imap library but Imap probably doesn't supports anything like it.
I can use "https://mail.google.com/mail/feed/atom" gmail feeds. but won't give me entire email and it only gives unread email and I don't want to miss any email
You can do this if you are using Google Apps for Business/Education. If you are, you can access the Gmail inbox feed (Atom) by using OAuth. OAuth can also be used to access Gmail via IMAP - you can then have complete access to the IMAP server programmatically, see Gmail IMAP and SMTP using OAuth.
Google has extended IMAP to allow developers to provide a more Gmail-like experience via IMAP, see: (Gmail IMAP Extensions, X-GM-EXT-1).
The unique message (X-GM-MSGID) and unique thread (X-GM-THRID) ids can be used to produce links to Gmail messages directly - you just have to hex encode the id long (e.g. Long.toHexString(x_gm_msgId)). Your link will then need need to be in the form of:
http://mail.google.com/mail?account_id=ACCOUNT_ID_HERE&message_id=MESSAGE_ID_HERE&view=conv&extsrc=atom
supplying ACCOUNT_ID_HERE (something like user#someplace.com) and MESSAGE_ID_HERE as appropriate.
I have been working in this area and think you might find my project useful, see: java-gmail-imap.
[NB: URLs formatted as above do not work on Gmail's mobile site (at least on iPhone/Safari).]
https://mail.google.com/mail/#all/HexEncodeMessageID
replace the HexEncodeMessageID part with the ID. You get it, when you open the email in a new window (use the pop out icon in the upper right corner.
The id looks like this: search=inbox&th=1426b8f59e003aa0
I'm fairly confident this is not possible - that there is no reliable way to get the unique URL that'll lead to a single email in Gmail. I'd love to hear otherwise!
I do believe it is possible to get a URL that will lead to the Gmail thread containing the message - but you have no control over which message(s) are "expanded" in this threaded display.

Setting up a MRA (fetchmail/getmail) to pull all folders from Gmail over POP/IMAP

I would like to setup fetchmail or getmail to pull email from Gmail. I would prefer it to be IMAP but I can live with POP as well. The issue I'm running into is that all these seem to want you to specify the exact list of folders you want checked (this is cleaner in IMAP than POP).
This is a hassle since I frequently change my list of folders and when I do, I don't want to have to change my configuration file to add/remove folders. When something changes, I automatically want a mbox or a maildir directory to 'show up' with the contents of my new folder. I would also prefer the mbox to have the same name as the folder on the server
The closest thing I could find was the -r option in fetchmail but that doesn't seem to do this either. I'm sure I must be missing something since every email client is able to do this and I'm hoping that that functionality lives inside the MRA and not in the MUA.
Update: I'd like to move this thread to Serverfault and I'm ok with voting to close this and migrate. Unfortunately, I dont have the rep to do it myself yet
With getmail you can use the following line in your configuration to get all your mail :
mailboxes = ("[Google Mail]/All Mail",)
With IMAP you can list the folders on the server, that way email clients can do whatever they need. In getmail it depends on how the email retrievers are written. If this configuration does not work, you can check the IMAP retriever code and write a specific one that suits your needs ;-)
my 2 cents
PS: I have no getmail installation to check if that do the trick ...
You could look into imapsync. It is designed to recursively sync up IMAP accounts. You will need an IMAP account for the destination. I have not tested it with gmail but if gmail is a well behaved IMAP server it should work.

Resources