UDP Packet forwarding getting EINVAL from sendmsg - linux

Question

I'm writing my own DNS black hole to block advertising and malware on my home network. I realize that this type of program already exists, but I would like to learn the process and write my own. From the Linux documentation, it appears that sendmsg() can be told to use a different return address so that a UDP packet can be forwarded and the receiving server will send the response to the original requester instead of my server. From the sparse documentation, I set up my socket bound to port 53 (DNS). I'm receiving DNS requests, interpreting them and responding when the site is blacklisted. For 'good' domain names, I'm getting 2 different results. On MacOS my forwarding request is sent, but the response comes back to my program instead of the original requester. On Linux (Armbian 4.13 kernel), I get EINVAL from the sendmsg() call and nothing is sent. Can anyone shed some light on what I'm doing wrong? (all error checking has been removed for brevity)
Additional info... a wrinkle in this scheme is that the original DNS request is bound to a socket port other than 53 (obviously). How do I tell sendmsg() to return the response to the correct port number of the original request?
My 'proxy' version of this works. I store the transaction ID and requesting port number of the original request and am able to return the response successfully
Latest news - I was able to successfully accomplish what I wanted, but not with sendmsg(). By using RAW sockets and 'spoofing' the return address in the IP header, I was able to get the packet sent back to a different return address and port number (using sendto). nslookup sees this as a problem, but browsers are okay with it. I guess we can call this 'case closed'
struct sockaddr_in addr, addrfrom, fwaddr;
socklen_t addrLen = sizeof(struct sockaddr_in);
int rc, listen_sock;
listen_sock = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP);
rc = 1;
setsockopt(listen_sock, IPPROTO_IP, IP_PKTINFO, &rc, sizeof(rc));
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(53); // port address of DNS
addr.sin_addr.s_addr = INADDR_ANY;
bind(listen_sock, (struct sockaddr*)&addr, sizeof(addr));
memset(&fwaddr, 0, sizeof(fwaddr));
fwaddr.sin_family = AF_INET;
fwaddr.sin_port = htons(53);
fwaddr.sin_addr.s_addr = 0x08080808; // Google DNS server
<... Receive DNS request from client...>
if (bForward)
{
struct msghdr msg;
struct iovec iov[1];
struct {
struct cmsghdr cm; /* this ensures alignment */
struct in_pktinfo ipi;
} cmsg;
memset(&cmsg, 0, sizeof(cmsg));
memset(&msg, 0, sizeof(msg));
iov[0].iov_base = request_bufffer;
iov[0].iov_len = request_len;
msg.msg_flags = 0;
msg.msg_name = &fwaddr; // dest address of packet
msg.msg_namelen = addrLen;
msg.msg_iov = &iov[0];
msg.msg_iovlen = 1;
msg.msg_control = &cmsg;
msg.msg_controllen = sizeof(cmsg); //sizeof(struct in_pktinfo);
cmsg.cm.cmsg_len = sizeof(cmsg);
cmsg.cm.cmsg_level = IPPROTO_IP;
cmsg.cm.cmsg_type = IP_PKTINFO;
cmsg.ipi.ipi_ifindex = 0;
cmsg.ipi.ipi_spec_dst = addrfrom.sin_addr; // original source address
rc= sendmsg(listen_sock, &msg, 0);
} // bForward

Answer 1

Issues
I see 2 potential issues in this code.
You are not using the CMSG macros to allocate and manipulate your ancillary data (i.e. your cmsg structure).
Even if that is fixed, you are trying to spoof the source IP address for the sent packet - i.e. you are trying to declare it is from an address not owned by this box. I have never tried to do this but wouldn't be surprised if you find the kernel security prevents you from doing that.
So what to do?
First I would fix up your meassage structures, using code like this SO answer. I suspect you should look closely at the offset of in_pktinfo and the length you're passing in to cmsg_len as these could cause EINVAL errors. A possible strategy might be to cut the code back to the point it's just sending packets normally to see which data structures are being rejected.
If that doesn't do it, I guess you could then consider re-writing your send logic to use raw sockets to bypass any IP validation your kernel might be doing. For example, this code looks like it should be roughly what you need.
If that's still not enough, I suspect you need to route around in the kernel security systems.