apify/fingerprint-suite

Random useragent

Closed this issue · 1 comments

    for (let i = 0; i < 100; i++) {
        var browserFingerprintWithHeaders = fpGenerator.getFingerprint({
                'locales': ["en-US", "en"],           // setup your desired fingerprint features
                'operatingSystems': ["macos","windows","linux"], // windows, macos, linux, android and ios.
                'devices': ['desktop'], // desktop and mobile                
        });        
                  
        var { fingerprint, headers } = browserFingerprintWithHeaders;
        var enhancedFingerprint = fpInjector._enhanceFingerprint(fingerprint);
        var { screen, userAgent } = enhancedFingerprint;
        console.debug(userAgent); 

}

Result:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.183
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.183
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36

There are so much redundant user agents. How to generate much more non-redundant user agent??

barjin commented

Hi @tenkuken, the fingerprint-suite tools are based on a random generator - therefore, there is no straightforward way how to generate non-redundant data. This, however, doesn't mean you cannot get a list of non-duplicate user-agents - you just have to deal with it in your own code. Here is an example of how you can do this:

const { HeaderGenerator } = require('header-generator');
const headerGenerator = new HeaderGenerator();

const userAgents = {};

for (let i = 0; i < 1e4; i++) {
    const userAgent = headerGenerator.getHeaders()['user-agent'];
    userAgents[userAgent] = true;
}

console.log(Object.keys(userAgents)); 

In this snippet, we generate 10000 header sets and deduplicate the user agents by using them as keys in a JavaScript object.

Please note that the looser constraints you set for the generator (by removing the 'locales', 'operatingSystems' or 'devices' options from the getFingerprint() call), the more distinct user-agent strings the generator will be able to generate.


A very important footnote:

If you really need only the user-agent strings (or some other HTTP headers) you'll be much better off with using HeaderGenerator instead of FingerprintGenerator. Here is a graph of run times for the script above using different tools to generate user-agent strings:

obrazek

You see that the header-generator (in my snippet above) is the fastest option with an average run time of just under 2 seconds, and the fingerprint-generator option (from your snippet) is the slowest one with 8s. This is because the fingerprint-generator uses a much larger model for generating not only the user-agent string but also screen dimensions, different Web API values, etc.

Feel free to use the script I provided above :)

I'll close this issue as wontfix, but feel free to ask additional questions if you get any.