Investigate failures in twbs/bootstrap

Starting with recentish release, linkinator started failing on a perfectly valid link in the bootstrap docs. The issue was raised here:
twbs/bootstrap#32678 (comment)

This link shows a 404:
https://getbootstrap.com/docs/5.0/

I debugged a little and it turns out to be an issue with the https://github.com/vercel/serve-handler transition, which was done for directory listings. More research needed.

cc @XhmikosR

Not sure if it will help, but after reading the source code a bit I think it'd be better if

this would be moved to a separate function which would return the expandedPaths and use Promise.all maybe?

linkinator/src/options.ts

Lines 78 to 107 in cb1d808

    
           const paths: string[] = []; 
        
           for (const filePath of options.path) { 
        
             // The glob path provided is relative to the serverRoot. For example, 
        
             // if the serverRoot is test/fixtures/nested, and the glob is "*/*.html", 
        
             // The glob needs to be calculated from the serverRoot directory. 
        
             const fullPath = options.serverRoot 
        
               ? path.join(options.serverRoot, filePath) 
        
               : filePath; 
        
             const expandedPaths = await glob(fullPath); 
        
             if (expandedPaths.length === 0) { 
        
               throw new Error( 
        
                 `The provided glob "${filePath}" returned 0 results. The current working directory is "${process.cwd()}".` 
        
               ); 
        
             } 
        
             // After resolving the globs, the paths need to be returned to their 
        
             // original form, without the serverRoot included in the path. 
        
             for (let p of expandedPaths) { 
        
               p = path.normalize(p); 
        
               if (options.serverRoot) { 
        
                 const contractedPath = p 
        
                   .split(path.sep) 
        
                   .slice(options.serverRoot.split(path.sep).length) 
        
                   .join(path.sep); 
        
                 paths.push(contractedPath); 
        
               } else { 
        
                 paths.push(p); 
        
               } 
        
             } 
        
           } 
        
           options.path = paths;

perhaps use more path API methods in options.ts, like basename, parse, join etc instead of this manual handling of paths?

In fact, it seem any directory with periods will return a 404 wrongfully.

	const paths: string[] = [];
	for (const filePath of options.path) {
	// The glob path provided is relative to the serverRoot. For example,
	// if the serverRoot is test/fixtures/nested, and the glob is "/.html",
	// The glob needs to be calculated from the serverRoot directory.
	const fullPath = options.serverRoot
	? path.join(options.serverRoot, filePath)
	: filePath;
	const expandedPaths = await glob(fullPath);
	if (expandedPaths.length === 0) {
	throw new Error(
	`The provided glob "${filePath}" returned 0 results. The current working directory is "${process.cwd()}".`
	);
	}
	// After resolving the globs, the paths need to be returned to their
	// original form, without the serverRoot included in the path.
	for (let p of expandedPaths) {
	p = path.normalize(p);
	if (options.serverRoot) {
	const contractedPath = p
	.split(path.sep)
	.slice(options.serverRoot.split(path.sep).length)
	.join(path.sep);
	paths.push(contractedPath);
	} else {
	paths.push(p);
	}
	}
	}
	options.path = paths;