0xdade/sephiroth

support for compacting resulting CIDR

Closed this issue ยท 3 comments

roycewilliams commented

The resulting CIDR can be significantly compacted - consolidating blocks into contiguous CIDR with exactly the same coverage, but fewer subnets.

The Perl way would be to use something like NetAddr::IP::compact (wrapper example here); I'm not sure about the best way to do this in Python.

Example savings - literally an order of magnitude when using the superset of all currently supported cloud providers (IPv4 only):

$ wc -l output/nginx_aws_azure_gcp_oci_2020-02-18_050910.conf.ipv4-only
   20682 output/nginx_aws_azure_gcp_oci_2020-02-18_050910.conf

$ wc -l output/nginx_aws_azure_gcp_oci_2020-02-18_050910.conf.compact
    2047 output/nginx_aws_azure_gcp_oci_2020-02-18_050910.conf.compact
0xdade commented

This will be a good idea for an optional flag. I want to avoid it by default because by compacting contiguous blocks you are likely to lose some level of detail about those addresses (which are only used in the form of comments right now, but hey). I think a --compact can be added which will attempt to reduce it to the smallest number of cidr ranges that is an exact match. It's pretty easy to do in python as well.

  • ๐Ÿ‘1
roycewilliams commented

Great! Yep, totally concur that it should be optional. I can definitely think of some use cases where more detail would be beneficial for many users.

0xdade commented 
sephiroth on ๎‚  master [!] on ๐Ÿณ v19.03.12 via ๐Ÿ 3.8.3
โžœ wc -l output/2020-08-05_001815_nginx_aws_gcp_oci_azure.conf output/2020-08-05_001827_nginx_aws_gcp_oci_azure.conf
   3037 output/2020-08-05_001815_nginx_aws_gcp_oci_azure.conf
  32069 output/2020-08-05_001827_nginx_aws_gcp_oci_azure.conf

Idk why it took me 5 months to get around to implementing this, blame COVID, but here we can see a similar scale of magnitude in size reduction.

(We can also see that there are about 12000 new CIDR ranges between Feb 18 and Aug 4, heh)

  • ๐ŸŽ‰1