Am trying to use a custom body template for the alert emails on airflow. Have created a file with the below message for testing purpose and mapped the file path to the variable html_content_template in the airflow.cfg file
Try {{try_number}} out of {{max_tries + 1}}<br>
Exception:<br>{{exception_html}}<br>
Log: Link<br>
Host: ***.***.***.***<br> Use this ip address as hostname instead of localhost to check
for logs and change status.
Mark success: Link<br>
Another issue, from_email config is also not working when have given a custom email address to send.
Airflow Version: 2.2.5
Ubuntu: 20.x
email_backend = airflow.utils.email.send_email_smtp
Related
I try to setup a incoming webhook in a project to trigger a yaml pipeline.
Setup service connection with incoming webhook.
Added yaml snippet to pipeline like this:
resources:
webhooks:
- webhook: testtrigger ### Webhook alias
connection: testconnection ### Incoming webhook service connection
When calling the url via curl : curl -X POST 'https://dev.azure.com/<my org>/_apis/public/distributedtask/webhooks/testtrigger?api-version=6.0-preview' -d '{ "tags": {} }' -H "Content-Type: application/json", I always get the error message :
{"$id":"1","innerException":null,"message":"Cannot find webhook for the given webHookId testtrigger. Try enabling CD trigger for this artifact.","typeName":"Microsoft.TeamFoundation.DistributedTask.Pipelines.Artifacts.WebHooks.WebHookException, Microsoft.TeamFoundation.DistributedTask.Orchestration.Server","typeKey":"WebHookException","errorCode":0,"eventId":3000}
Tried with different names on triggers/connection but nothing worked.
Any ideas what I am missing ?
Echoing #FrankBaumfalk's comment, you need to ensure the pipeline's default branch is the same as the one that youre developing on. (You may be like me, developing and testing this yaml on a branch of the same repo it is built for.)
To indicate your default branch:
Edit yer Pipeline
Choose Triggers from the kabob menu
Choose the YAML tab, see below 👇
Notice the default branch field; save
Also, yes the error referring to webHookId is the WebHook Name in the Incoming WebHook in Services Connections. Yay consistent naming.
Based on my test, I could reproduce this issue.
The root cause of this issue is that you are using the incorrect webhook name.
To find the correct webhook name, you could navigate to Project Settings -> Service Connection -> testconnection incoming webhook.
Or you could go to the target Webhook and check the request url:
https://dev.azure.com/<ADO Organization>/_apis/public/distributedtask/webhooks/<WebHook Name>?api-version=6.0-preview
The testtrigger in yaml pipeline is the name of the webhook resource. This is a custom name instead of the correct webhook name.
I am new to ADF. I created an http linked service and it tested successfully (https://rebrickable.com/downloads/). On the http dataset properties page the same url address is in the Base URL. I then put just the file name that exists on that page (themes.csv.gz) in the Relative URL. When I click next to the File format settings page the Preview pane says:
Failed to read data from http server. Check the error from http
server：The remote server returned an error: (404) Not Found. The
remote server returned an error: (404) Not Found. . Activity ID:
0ca9f9c7-438d-461a-81c1-f81bacace4ca
What am I doing wrong please?
I also tried to create a new linked service and point it to my desktop files. I selected'File', left it at AutoresolveIntegrationruntime, put the path in Host. But don't know what do to with User/password. Please help
I hit the above mentioned end point in by browser : https://rebrickable.com/downloads/themes.csv.gz
Looks like the file is not existent at the mentioned location.
I did a quick scan of the page https://rebrickable.com/downloads
Encountered the below file :
Saw the file hyperlinked to this url (This is different from what you're attempting to in the ADF)
https://cdn.rebrickable.com/media/downloads/themes.csv.gz
I was able to download the file by using the above.
Would suggest you to try the above endpoint.
Update your url to https://cdn.rebrickable.com/media/downloads/ in https connection source
I need to hide a username and password used by nodemailer to be able to push on a public repo.
For now, I am forced to do a push on github without the file containing this info; this is a contact file that creates an object "transporter", which itself contains this private information ...
Then, my app on the server being connected to its github repo for a continuous deployment, it crashes the deployment because this file becomes missing ...
I have to re-do a push, but this time directly to the server, so with this famous contact file to restore the site.
So, the continuous deployment becomes useless !
Besides, as a result, I can't version this file !
Either use a configuration file, add this to your .gitignore and then add the file to the server manually once or use environmental variables/ command line arguments.
You can access environmental variables using node.js in the format:
process.env.ENV_VARIABLE
or
process.argv
is an array of command line arguments in node.js
Check with your hosting service for a way around using the password. For example, with AWS, you register a secure email address (I've used a gmail address), and then you use a security key, which you add the the server file like this:
//Sends mail with submission forms
const transporter = nodemailer.createTransport(sesTransport({
accessKeyId: "",
secretAccessKey: "",
rateLimit: 5
}));
app.get('/send-mail',function(req,res){
const mailOptions = {
to : req.query.to,
from: req.query.to,
subject : "New email subject",
text : "New email text"
}
Note that I am sending from the secure email address to itself. I then forward from that address to wherever I want the email to go. You could instead add another secure address, but AWS will not send to an address that is not registered.
Im having a problem with sending emails using Amazon SES. I have an Amazon EC2 instance.
It worked for the first couple of days but I just noticed last week all emails now fail. I have tried sending using Node and the Amazon SES sdk and from within AWS where you can send a test email. I have the following code in Node:
var aws = require('aws-sdk');
// load aws config
aws.config.loadFromPath('email_config.json');
// load AWS SES
var ses = new aws.SES({
apiVersion: '2010-12-01'
});
ses.sendEmail({
Source: from,
Destination: {
ToAddresses: to
},
Message: {
Subject: {
Data: 'Somebody registered'
},
Body: {
Html: {
Data: body,
}
}
}
}, function(err, data) {
console.log('email err is ', err, ' and data is ', data);
});
The result of the log is:
email err is null and data is { ResponseMetadata: { RequestId: 'ad28f526-0b15-11e6-ad87-1108d652684a' },
MessageId: '010101544ebc41b3-f7bd43dd-0505-4eb2-a056-219ce6180fc5-000000' }
But the email doesnt deliever and I then receive an email from Amazon saying:
An error occurred while trying to deliver the mail to the following recipients: < my email address >
This contains an attachment with the following text:
From: < my email address >
To: < my email address >
Subject: Somebody registered
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
Message-ID: <010101544ebc41b3-f7bd43dd-0505-4eb2-a056-219ce6180fc5-000000#us-west-2.amazonses.com>
Date: Mon, 25 Apr 2016 18:44:01 +0000
X-SES-Outgoing: 2016.04.25-54.240.27.56
Feedback-ID: 1.us-west-2.GkIUmTTEDEIC5VBoooumwcKSnMDcLT8S4Zd3/deS/BU=:AmazonSES
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
s=gdwg2y3kokkkj5a55z2ilkup5wp5hhxx; d=amazonses.com; t=1461609841;
h=From:To:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID:Date:Feedback-ID;
bh=fHqQiK/2DJ+B7zddmElFttCiWFnADDSNj5umLJQCPJs=;
b=ZI/358zmcRHVBKTdA6qbQky5nj5z/YWw215KvkZ+oD73N0booHbl+jx+O05FdcKR
irDjmyEDppGkp7rToZSTt/NHDeRrbERixT/ZCjGo/KOxvShovD7Z5mnDViRmkS5sTz5
qo0oO0NuRz1lGVPkT5ONHNhKhWs7ncC9id0ycm34=
When I actually log into AWS and send a test email through the console, I get the same failure.
I have verified the senders email address and I have an approved sending limit for the region.
Any ideas what this could be?
EDIT
I just noticed in my AWS control panel > SES Home > Domains it says my domain is 'pending verification'. Could this be it? It says I need to add a TXT DNS record with a name of xxx and value of yyy. I already did this on Register365. Maybe I did it wrong? Register365 doesnt provide name and value fields for a TXT record, only a 'result' field. So I added a TXT record with the 'result' field of: xxx=yyy. Is this the correct approach? This was weeks ago though and its still pending verification....
EDIT
I've since added a TXT record to my Register 365 control panel, and still my domain cannot be verified. The record looks like:
Amazon provided me with the following TXT record to verify my domain:
TXT Name*: _amazonses.mydomain.com
TXT Value: u1qHYT6/2KV9Kl1VLKsApXjwcPqVXKJ8KeXj50k=
So in the Register 365 control panel "result" field I've added the record in the form name=value i.e "_amazonses.mydomain.com=u1qHYT6/2KV9Kl1VLKsApXjwcPqVXKJ8KeXj50k="
I then ran nslookup to find the record but got the message:
server can't find _amazonses.mydomain.com: NXDOMAIN
What am I doing wrong?
EDIT
I have now changed the TXT record to:
But after 3 days I have gotten another email from Amazon saying they have failed to verify the domain. Im utterly baffled now, I've been trying to verify it for 6 weeks!
My SES account is not in sandbox mode - i've already been approved to send email via SES. I've also verified my sender email address.
Are there any other options open to me? The Amazon SES service seems absolutely dire.
Also when I run:
nslookup -type=TXT _amazonses.redmatterapp.com ns-478.awsdns-59.com
I'm still seeing:
server can't find _amazonses.redmatterapp.com: NXDOMAIN
When I run:
nslookup -type=TXT redmatterapp.com ns-478.awsdns-59.com
I get:
Can't find redmatterapp.com: No answer
Why is this happening? My DNS is with Register 365
EDIT
Seems like the nameservers I was using with nslookup were wrong. When I run nslookup, i know get:
_amazonses.redmatterapp.com text = "u1qN5cbTEDb/2EV9Bhd67YHT5jjqVXKJ8KeXj50k="
Which looks right. Yet still verification for my domain fails...
As Michael, the SQL Bot pointed out, you need a hostname (_amazonses) on the left, and the value on the right. That will help to validate the domain.
However, there are a number of other possible reasons for failure. Is SES still in sandbox mode? If that's the case, you'll need to verify the TO and the FROM email addresses.
It might be easier to verify individual email addresses if you can't get the domain verification working. So create them in SES, and go through the validation process. Once you create those (or, if you manage to get the domain verified) create an SNS topic that sends you email, and then configure the Bounce, Complaint, and Delivery notifications to that SNS topic - you should end up with an email for every delivery attempt, regardless of whether it succeeds or not.
The last thing to consider is the possibility that your email address has been added to the supression list. If you generate a lot of errors, SES will add you to a "do not email" list. There is an ability to request removal from this list in the SES console.
The hostname part is _amazonses (left column, next to the number 2)
The value is "u1qHY..."
I think you're on the right track in that last image, only I believe the host name is _amazonses, and u1qHYT6/2KV9Kl1VLKsApXjwcPqVXKJ8KeXj50k= is the result, instead of putting everything in the result field in the form "_amazonses.yourdomain.com=u1qHYT6/2KV9Kl1VLKsApXjwcPqVXKJ8KeXj50k=". Iiuc, the idea is that AWS will curl _amazonses.yourdomain.com, expecting your key to be served as a TXT file, but currently you're serving a TXT file with the contents _amazonses.yourdomain.com=u1qHYT6/2KV9Kl1VLKsApXjwcPqVXKJ8KeXj50k= (I can't quite read, as its cut off; pardon my guess) on yourdomain.com instead.
The reason I believe this is that you're getting the error NXDOMAIN, which means the domain _amazonses.yourdomain.com doesn't exist, which makes sense if you hadn't set up a TXT record for _amazonses.yourdomain.com, but instead set up a txt record for http://yourdomain.com instead with the value _amazonses.yourdomain.com=u1qHYT6/2KV9Kl1VLKsApXjwcPqVXKJ8KeXj50k=. Its also what the other two answers seem to suggest, which makes me feel more confident.
I recently verified a domain for the company I work for successfully, it is set as follows in my domain DNS (as a TXT record):
It may be worth you reading Amazon's troubleshooting page if you're still having issues.
Adding to my answer:
I've just checked in my AWS console, if you open up SES > Domains and click on your domain name. Scroll down then click DKIM, I had to verify some more there:
And add them as CNAME records as follows:
One thing people forget with this process is the fact that, Amazon requires you to leave the TXT record in place even after the verification. Otherwise they will revoke the domain.
Hope this helps!
Yet another edit (sorry)
When I run nslookup -type=TXT _amazonses.redmatterapp.com ns-1471.awsdns-55.org to try and find your TXT record, it comes back:
Server: ns-1471.awsdns-55.org
Address: 205.251.197.191#53
** server can't find _amazonses.redmatterapp.com: NXDOMAIN
This shows that the TXT record is not setup correctly.
I have a problem now with Jenkins Email. I tried to attach a file like demo.html in the $WORKSPACE. But it didn't work. I just received an email with build.log attached. Could someone figure out what's the problem?
I am now doubting that the email is processing in Jenkins master, and my job is working in slave. Is that the reason?
Here is Attachments filed in Editable Email Notification. I filled it with "*.html"
============
Attachments *.html
============
I also tried ./*.html in Attachments field, but with no luck.
A part of build log:
completed with result SUCCESS
Checking if email needs to be generated
Email was triggered for: Success
Sending email for trigger: Success
NOT overriding default server settings, using Mailer to create session
messageContentType = text/plain; charset=UTF-8
Request made to attach build log
Adding recipients from recipient list
Adding developers
Successfully created MimeMessage
Sending email to: test999922#hotmail.com
Finished: SUCCESS
I found the reason.
The html is in the salve's workspace not in master's workspace. So jenkins cannot find it.
When I restrict it run on slave, the problem is resolved.