The alookup module returns duplicate IPs for Akamai edge dns CNAME domains

[domderen]

Hey,

I ran this command with zdns:

 $ echo 'www.walmart.com' | ./zdns alookup --name-servers 8.8.8.8

And I got the following result:

{
    "data": {
        "ipv4_addresses": [
            "23.74.172.136",
            "23.74.172.136"
        ]
    },
    "name": "www.walmart.com",
    "status": "NOERROR",
    "timestamp": "2022-11-17T21:47:05+01:00"
}

It looks like the zdns alookup module is returning duplicate ip addresses in some cases. When I try it with the a module, result looks correct:

 $ echo 'www.walmart.com' | ./zdns a --name-servers 8.8.8.8 

{
    "data": {
        "answers": [
            {
                "answer": "www.walmart.com.edgekey.net.",
                "class": "IN",
                "name": "www.walmart.com",
                "ttl": 74,
                "type": "CNAME"
            },
            {
                "answer": "e4373.x.akamaiedge.net.",
                "class": "IN",
                "name": "www.walmart.com.edgekey.net",
                "ttl": 21427,
                "type": "CNAME"
            },
            {
                "answer": "23.74.172.136",
                "class": "IN",
                "name": "e4373.x.akamaiedge.net",
                "ttl": 20,
                "type": "A"
            }
        ],
        "protocol": "udp",
        "resolver": "8.8.8.8:53"
    },
    "name": "www.walmart.com",
    "status": "NOERROR",
    "timestamp": "2022-11-17T21:48:39+01:00"
}

After some additional tests, I think this issue is somehow related to the fact that Walmart is using Akamai Edge DNS system. When I tried the same test on a CNAME record that doesn't use Akamai, the alookup module returned correct result without duplicated IPs.

Could someone point me in the right direction, why this might be happening? I'm happy to help fix the issue, but I'm quite new to zdns and I could use a helping hand pinning this issue.

Thanks in advance for your help!

[zakird]

Thanks for filing this issue. This sounds like an edge case where there are multiple CNAMES that point to the same IP address. I think that we need to add some logic in https://github.com/zmap/zdns/blob/master/pkg/miekg/miekg.go#L811 to dedup IPs before handing them back. I looked at what calls this function, and it's pretty limited, so I think we're safely just always deduping IPv4 and IPv6 addresses in that function before returning.

[domderen]

Hey @zakird thanks for the input, I created a PR to fix this.