After noticing that our emails are being delivered to the spam folder on Gmail, even though it only happens with gmail, we are investigating any possible issue.
Now, we are looking into the SPF record, which look like this after a cleanup:
v=spf1 a include:sendgrid.com include:_spf.google.com -all
Tests we run on http://www.kitterman.com/spf/validate.html result in a "Too many DNS lookups in an SPF record" error.
As far as I know, the SPF record there has only 2 dns lookups... and I checked other SPF records, such as Sendgrid's and they have far more lookups and still their test return "Passed".
My SPF record is published both as an SPF and TXT. Am I missing something?
Domain is fulltraffic.net
The sample SPF record has three mechanisms which may require DNS lookups: the "a" mechanism and two "include" mechanisms. This is well below the limit of 10 defined by the latest RFC7208 version (see https://www.rfc-editor.org/rfc/rfc7208#section-4.6.4 and this limitation was already present in RFC4408).
The culprit here is that "include" triggers a recursive evaluation of the included policy, but the limit is for the complete evaluation (as opposed to per policy). In your case, the included policies define 7 further includes, which in turn define further includes, in overall exceeding the 10 DNS-based term limit.
You will need to find a way to reduce the number of DNS-based terms, like replacing your "a" mechanism with an "ip4" mechanism.
Related
Say a company successfully applied to IANA to make .bob a Top-level domain and the company now operates the registry of every domain with .bob as the TLD. If the comany is under an authoritarian government with a track record of manipulating the Internet infrastructure, can a domain target.bob be hijacked so that it gets resolved to a server the government owns, instead of going through the name server the domain owner specified? Will DNSSEC help?
Yes, technically any node in the DNS tree can pervert everything below. But, especially at the TLD level it will be akin to a move with a nuke, it will be seen quickly and draw lots of actions and consequences.
You may want to go back at the Verisign Sitefinder fiasco. Not exactly the case you describe but very similar. It generated two consequences:
contractually, gTLD registries at least because under contract with ICANN are now prevented to do things like that
technically, there are safeguards, see for example root-delegation-only in bind that means exactly that (for both root and TLDs) to ensure there is no "hijacks".
And DNSSEC can help, but in a limited way. Because in your case even if the domain is properly delegated and DNSSEC secured before the hijack, it means the registry has and published the DS record, so once the hijack happens those DS records can be hijacked as well, or just removed, and end resolvers will see the change but just with that can not detect really a problem nor work around it.
Note that in your case of "authoritarian government" they don't even need to go to the registry and the authoritative nameservers. They can force things at the recursive state also, forcing ISPs. And it completely happens today, and even in non authoritarian government: various names are, by law or judge, forbidden to be resolved. Sometimes it happens at registry side (see Microsoft seizing domains recently - and regularly - at https://arstechnica.com/information-technology/2021/12/microsoft-seizes-domains-used-by-highly-sophisticated-hackers-in-china/), and sometimes at resolver (this the recent judgment against Quad9 a big public recursive nameserver service: https://www.quad9.net/news/blog/quad9-and-sony-music-german-injunction-status/)
Also, side technical note: in the DNS a dot does not mean necessarily a delegation, and both sides can be administratively and technically handled by a single party. For example gouv.fr and fr are technically and administratively handled by the same entity, there is no delegation nor hijack.
I am looking for away on the linux command line to list all the websites associated with 1 IP. I know web hosting companies can have hundreds of websites on a single IP and I am looking at how I can query that list. I know some website can provide this information but I don't think the list is always complete.
I have tried dig, and whois and a few other commands but none seem to give me more than a single dns lookup.
Any ideas? All the articles I can find talk about finding the single ip for a website. I know how to do that.
I am looking for away on the linux command line to list all the websites associated with 1 IP
There is no such command. And you may want to start your question in fact by specifying WHY you need that information from the first place.
I know some website can provide this information but I don't think the list is always complete.
It is never complete because it can't be.
How everyone is doing it is basically starting with a list of names, resolving them, hence getting IP addresses, and storing the results. Once you did that with enough names, you can now do inverse searches because you stored all the data. If you are at appropriate places on the Internet, you can also passively monitor recursive DNS data, showing what people are searching for - in terms of names resolution - and hence capture replies which will have the name to IP mapping, which you can then trivially reverse.
Obviously you will never get all the data that way, and name/IP association can change in both time and space (depending from where you ask, because of geolocation, etc.), so the results are far from exhaustive and precise.
I have tried dig, and whois and a few other commands but none seem to give me more than a single dns lookup.
As they are designed to do... (except that whois does not do DNS queries by the way).
Some people will tell you about DNS PTR records that supposedly maps IP addresses back to names, but:
they are not needed for day to day operations (they are vaguely needed for email operations, absolutely not for web operations)
hence very little IP addresses have those records
and even if a webhosting company uses them, for an IP address hosting hundreds of names (as mass virtual hosting is possible nowadays for both HTTP and HTTPS), the PTR record will probably just show one name, not all of them (few clients would cope with PTR records with multiple names anyway)
I purchased my domain through Google Domains. It's awesome because you get up and running with almost everything pre-configured for you.
Until you want to change a few things!
I'm using AWS SES to send emails from my platform through this domain. This means I need to tell DNS to allow SES to send emails on behalf of this domain by adding an SPF record.
According to RFC4408 a domain cannot have multiple SPF records. And this is where things get complicated.
Out of the box, Google Domains gives you pre-configured Synthetic Records, SPF being one of them:
There is no way to edit these pre-configured Synthetic Records and clicking on the Delete button feels intimidating.
There is a section to add Customer DNS Records but that would mean that RFC4408 would be violated.
How do you resolve this?
I don't think you have any choice; you need to scrap what they provide and build your own set manually. It shouldn't take you long – there are not many records there. You can probably drop the SPF-type record safely, leaving just the TXT one.
The RFC violation is a factor, but the practical reality is that no receiver will check for more than one SPF record (because of the RFC), so it wouldn't help even if you did add multiple SPF records.
Most important though – keep an accurate record of exactly how it was originally configured so that you can get back to it easily!
I train customers how to use Emarketing tools. One important thing I tell them they have to do is have their IT set their Email domain DNS to allow the emarketing domains to send on their behalf via an SPF record and I give them most all the settings. Until this one current client where their IT is refusing to create one. My understanding is if they don’t, at minimum, all their Emarketing emails will basically all go to junk/spam or even get them blacklisted in places. So... is there a potential downside to making an SPF record that would make this person so adamantly against creating one?
The main downside of making an SPF record that actually helps deliverability (by confining mail sources) is that you have to know and limit the sources from which email may be sent, and that's not necessarily easy.
Say your domain is example.com and you want to send email from that domain using your gmail account (though gmail is not used to handle your domains email); your SPF would need to include gmail's SPF, and it therefore means that anyone on gmail gets to be able to send in your domain's name from gmail (other measures notwithstanding). Now imagine you're a big company and you have a long history of having employees routinely send email from company addresses using their local ISP. Your SPF needs to become very large, or very dumb, and includes so many email sources it becomes ineffective as a countermeasure.
Reluctance to have to deal with user complaints when you say that all outbound mail must go through company mail servers (when you have a long history of not requiring that) would be painful for a lazy IT support department. I have no sympathy for this policy - you reap what you sow.
You can of course have an SPF record "in name only" by having it set to something useless like +all - that would get a technical SPF "pass" for all sources, but in all other respects it's completely useless, marking fraudulent, spam & phishing messages as equally valid as legit email.
Any IT department that wilfully ignores the benefits of SPF is eventually going to have a major problem with deliverability and phishing, so you can reasonably argue they are failing to do their job, and I'd recommend reporting them to higher management.
I'm using node.js's dns.resolve function to find NS and CNAME records for a given domain. I also would like to get TXT and SPF records now. If I do that all separately - it's four requests. Is there any way to get all four record types in one request?
Also, is the SPF information always in TXT record type or it can have its own?
I'm no Nodejs expert, but just from looking, it appears you're going to have to do four different requests.
Also, is there any way to query SPF record with this function at all
or should I use something else?
Well according to Wikipedia, SPF is:
Sender Policy Framework (SPF) is an email validation system designed
to prevent email spam by detecting email spoofing, a common
vulnerability, by verifying sender IP addresses. SPF allows
administrators to specify which hosts are allowed to send mail from a
given domain by creating a specific SPF record (or TXT record) in the
Domain Name System (DNS). Mail exchangers use the DNS to check that
mail from a given domain is being sent by a host sanctioned by that
domain's administrators.
You could code your own, probably by querying the TXT record and then doing a check to see whether it is valid.
Again, I have very minimal Nodejs knowledge.