Optimus Sitemap Generator (OSG) is a universal XML sitemap generator that works by crawling your website, avoiding excessive bandwidth overhead by only scanning the contents of pages that have changed since the last time the sitemap was generated. == Installation Download OSG from http://patrickmylund.com/projects/osg/ If you have Go installed, you can run: go get github.com/pmylund/osg (an osg binary will be added to your GOPATH/bin folder) Note: You do not need to have Go installed to run the stand-alone version. == Usage OSG takes the name of a sitemap, and a list of the pages from which crawling should begin. Any files linked to will be included in the sitemap, and pages' links will be followed automatically. OSG will not crawl outside the domain of a given link (to do this, list a page on each domain to include in the sitemap.) ./osg sitemap.xml example.com ./osg sitemap.xml example.com/category1 example.com/category2 ./osg -v sitemap.xml example.com Run osg without any arguments for details, like how to ignore robots directives like nofollow, or how to exclude particular URL patterns. See http://patrickmylund.com/projects/osg/ for more information.