[Feature Request] handle i18n URLs

[capi1O]

Currently my robots.txt is

User-agent: *
Disallow: /oauth/*
Disallow: /api-auth/*
Disallow: /en/login/*
Disallow: /en/admin/*
Disallow: /en/page-1/*

Host: my.website.com
Sitemap: http://my.website.com/sitemap.xml

with url.py

urlpatterns = [
	url(r'^oauth/', include('social_django.urls', namespace='social'), robots_allow=False),
	url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework'), robots_allow=False),
	url(r'^sitemap\.xml$', sitemap),
	url(r'^robots\.txt$', url_robots_views.robots_txt)
]

# i18n URL patterns
urlpatterns += i18n_patterns(
	url(r'^admin/', include(admin_urlpatterns), robots_allow=False),
	url(r'^login/', some_view), robots_allow=False),
	url(r'^page-1/', some_view), robots_allow=False),
	url(r'^page-2/', some_view)),
	url(r'^page-3/', some_view))
)

better output would be :

User-agent: *
Disallow: /oauth/*
Disallow: /api-auth/*
Disallow: /*/login/*
Disallow: /*/admin/*
Disallow: /*/page-1/*

Host: my.website.com
Sitemap: http://my.website.com/sitemap.xml

in order to handle all i18n URLs