Since connect api sends the response to the "listner api" IMMEDIETELY after the last recipient signs the document. If my "listner api" is down at that perticular time. How will I get the response again from connect api?
Does connect api provide any option to send it again?
You're looking for the option to Require Acknowledgement. As per the documentation:
Require Acknowledgement: Select this option to log posting failures.
The acknowledgement failure messages are logged in the Failures page,
which is accessed b clicking Failures on the DocuSign Connect Settings
page. When this option is selected, DocuSign will automatically
attempt to repost any failures. Alternately, you can manually repost
from the Failures page. See the Publish Failures section for more
information
As long as your application can return a 200 status code when it does successfully accept a Connect publish, having Require Acknowledgement on means that failures will be logged and retried.
The answer from Drew is correct, but here's some additional information: the Connect resend algorithm, used if require acknowledgement is on, does not try to resend immediately. And if the Connect Send Individual Messages (SIM) option is off (the default), then a message won't be resent if the message is superseded by another message about the same envelope.
Better is to not have your server (your "listener") ever go down. Best way to do that: have Connect send to AWS or similar, then your app pulls from AWS SQS or similar. See the code examples named connect- here: https://github.com/docusign
Related
Scenario:
I have a remote server which is monitored (via DataDog) and sends out a warning when some anomaly is detected. This warning can be fetched via a webhook. Now I want to connect that webhook (https://docs.datadoghq.com/integrations/webhooks/) with MS Teams (probably via Bot), to receive a warning. Then I want to send a command back to the remote server to resolve the warning.
Technology:
MS Teams, Python flask/Django, remote server
Expected Results:
I can receive a warning from my remote server to MS Teams via a bot. Then send a command back to the remote server. My initial plan is doing this using Python Flask/Django but not tied to a specific language.
Environment:
Remote server is a LINUX based system. we have a internal network that is used within our company, so might need to resolve a firewall problem potentially (idk whole lot about it tho).
Things I have tried:
I just want to see if this is possible or not, so i havent coded up any. But I found some information relevant to our problem:
https://docs.datadoghq.com/integrations/webhooks/
https://learn.microsoft.com/en-us/microsoftteams/platform/bots/how-to/create-a-bot-for-teams
https://learn.microsoft.com/en-us/microsoftteams/platform/webhooks-and-connectors/what-are-webhooks-and-connectors
One last note, I am not also tied down to Teams bot. Our company also uses Azure Devops, so that is another resource I can use to realize the solution.
Btw, this question was posted on here but was told to post on MSDN, but I couldn't find an appropriate forum. Hence, I am posting on stack overflow instead
As you have limited requirements, you could achieve this without a bot.
MS Teams has income and outgoing webhooks. You could create a Incoming webhook inside a Teams channel. It provides an URL which you could use inside the monitoring remote server and POST the message in JSON format to the webhook url. It will be posted in teams channel like below
For sending message back to the server you need to configure the Outgoing webhook in the channel.
I am developing an email sending service, probably for sending bulk emails using sendgrid web API, but I am not able to figure out best practice for scalable system. I wish to keep a record of all those emails which failed to deliver and retry sending to those failed emails after all emails have been sent. I am using NodeJs, so just wanted to know if there is any way to speed up my process(something like sending multiple emails at the same time)
There are multiple ways to handle this, I will suggest two which seems obvious to me.
(Recommended - Easy) Use Async module's control flow option called queue Async Documentation. You can feed in all the request in form of an array of object request and then change concurrency setting to let's say 100, it'll run concurrent 100 workers at one time and to log errors make a separate mechanism and once all the values have been run through handle it separately.
Spawn multiple workers using node.js native approach.
Sendgrid offers an npm package for node.js integration, so you don't have to reinvent the wheel. It accepts messages at a high rate, so you shouldn't have problems delivering yours to sendgrid. You just dump your messages into sendgrid.
Email, being a store-and-forward system, is inherently asynchronous. That means it operates far from real time. Some messages are delivered in a few seconds, and others take hours (when they get soft--"retry later"--rejections from destination servers, for example).
Sendgrid handles this issue with a "bounces" API. (And with "bounces" features in their web back end application). Many bounces are "hard" bounces, meaning you must avoid trying to send messages to that address again. You can use the bounces API to retrieve a list of bounced messages. You should remove those addresses from your email list, and not try to send them again. (Sendgrid bans users who repeatedly send mailings with a high undeliverable rate.)
They also have an "invalid emails" API. This works like "bounces" and returns lists of addresses that are ill-formed or, if sendgrid can tell, not present on the destination server. Again, you should remove these addresses from your email list. If they're invalid now, they will be invalid tomorrow.
Sendgrid offers all sorts of tutorials on this subject.
I'm able to successfully load test my bot server by getting the proper auth token from Microsofts auth URL (basically through this page)
I was wondering if this was a valid test on the service considering that we're not actually hitting the bot frameworks endpoint (which has rate limiting)
Is there another way to load test a bot service wherein i can replicate the bot frameworks throttling/rate limits?
I ended up with using load test with Visual Studio and Visual Studio Team Services.
The reason why I used this approach is that you can setup full path of load tests. Azure Bot Service can be either Web App or Function App with endpoint prepared for receiving messages - using HTTP POST so in the end is just web service.
You can setup load tests for different endpoints including number of hits to selected endpoint. In case of Bots you can for instance setup test with 100 fake messages sent to the bot to see the performance.
You can read more under these two links below:
Load test your app in the cloud using Visual Studio and VSTS
Quickstart: Create a load test project
Unfortunately as stated in the documentation you linked, the rates are not publicly available due to how often they are adjusted.
Regarding user-side throttling- this should not actually have an effect either way as long as you simulate reasonable traffic, but even if you go a bit overboard, an individual user hitting rate-limiting would be functionally equivalent to just having a bit more traffic. The single user sending more messages to the bot is the same as three users sending the same amount of messages slightly slower and there's no limit for your bot in terms of how many customers you might have. That said, a user getting a message, reading it, and typing up a response should not put themselves into a situation where they are rate-limited.
However, regarding bot side throttling it is useful to know if your bot is sending messages too fast for the system. If you are only ever replying directly to messages from users, this will not be an issue, as the system is built with replying to each user message in mind. The only area you might run into trouble is if you are sending additional (or unsolicited) messages, however even here as long as you are within reasonable limits you should be OK. (i.e. if you aren't sending several messages back to a user as fast as possible for each message they send you, you will probably not have problems.) You can set a threshold for bot replies within your channel at some reasonable-sounding limit to test this.
If you would like to see how your bot responds in cases where throttling is occurring (and not necessarily forcing it into tripping the throttling threshold), consider setting your custom channel to send 429 errors to your bot every so often so that it has to retry sending the message.
Thoughts on how you guys would/do handle this... If you need to send a webhook when something happens. Say, when a customer registers.. How would you guarantee that a webhook has been sent for it? You couldn't just have the code send the webhook after the registration happens, as what if the server crashes right after registration but before/during the webhook request.. If you were using MongoDB, I guess you could listen to the oplog and see what a registration record is inserted, but then what if your oplog monitoring script crashes.. I thought about storing locally a file that records the last successfully processed timestamp but then you're really limiting yourself to processing the oplog one at a time, which on an active DB could really get behind.
I thought about using a queuing/task system but then what if the request to store the task in the queue crashes..
Maybe store a field on the registration DB record that says if the webhook has been sent yet, send it once the registration happens and have a script that checks every once in awhile for webhooks that failed to send within a certain amount of time?
Just trying to think of the best way to make sure a webhook is actually sent for a particular event.
Is there a way to peek or see a message before it hits the SMTP on IIS. This is not an Exchange Server, it's just running SMTP. I am trying to see if I can look at the message and then pass it to SMTP?
Thanks
Edit ~ Instead of adding another listner, I am wondering if there is a way to bind to the default SMTP listner and intercept the message then pass it on.
2nd Edit~ Ok, here is my problem. I have a spam filter in front of my exchange box, unfortunately (due to software design) the filter is limited when it comes to "Directory Harvesting Loookup". This is the process where the email addresses are checked if they exists in AD and the mail is dropped if they don't. My current filter drops the mail if one of the addresses does not exists in AD which is not good. I spoke with the vendor and there is nothing they can do at this time. I am looking put an app in front of this filter which would intercept (open, read, parse) the mail, validate the addresses, and then pass on the email to the filter for additional scanning. I'll then trun off this feature in their software. Don't get me wrong, their filter works great with this one exception which I must fix since I have tons of emails send to nonexistent users in my domain.
You can write your own Proxy SMTP service that you connect to to send messages. You can forward all messages directly to your actual SMTP service and pass all responses back. Then you can evesdrop on all these messages and deal with them accordingly.
Might be a bit overkill for what you're after but it's fairly simple to code as you dont need to know anything about the protocol as all you're being is a proxy.
If you're using .NET 2.0 then you can log SMTP sessions to a file:
How do I create a log file of the SMTP session? (System.Net.Mail)
Updated:
Take a look at this question:
Testing SMTP with .net (Stack Overflow)
From your edit:
"I am wondering if there is a way to bind to the default SMTP listener and intercept the message then pass it on?"
...and from your comment below:
"I am looking to inspect the actual message before the SMTP gets it."
I'm not sure if you fully understand the SMTP protocol. SMTP messages aren't just monolithic fire-and-forget entities. SMTP is session based and there is a conversation between client and server, of which, the message is just a part. The tracing method (linked to above) will record the entire exchange between client and server and does intercept the whole message before passing it on. The alternative, a proxy or mock server, will still require your application to engage in the SMTP client/server exchange. The closest solution to your requirement would be to use something like Papercut which is linked to in the answer above.
Kev
In .Net you can tell the SmtpClient to send email to a different folder than the SMTP service is monitoring. That way you could check each message, then move it to the real pickup folder. (See SmtpClient.PickupDirectory)
IIRC, you can still write up event sinks for the IIS SMTP service (even though it's not full blown exchange). It's been many years since I've done this, but you may want to google for "exchange event sink" to see if that helps.
Seems like a something like Ethereal will let you accomplish the sniffing portion of your request. Its not clear to me what you mean by "intercept" and "pass on". Do you want to filter some traffic or just delay traffic long enough for you to inspect before you pass it on, or both?