themm1/procyclingstats

Max number of races is limited to 100?

Closed this issue · 1 comments

Hi!

I have created a small custom function

def fetch_rider_results(rider_url):
    rider_results = RiderResults(rider_url + "/results")
    rider_results_JSON = rider_results.parse()
    return rider_results_JSON

When I call this function, for instance

st.write(fetch_rider_results("rider/jonas-vingegaard"))

I get a JSON object, but the number of races and stages only go from 0-99. Wouldn't it be possible instead to fetch for instance the results for the last 5 years?

Best regards
Kasper

Unfortunately it's not possible to get more than 200 results per request, becasue PCS won't show more on a rider results page.
It's possible to get 200 results by changing the Results per page field on this page for example: https://www.procyclingstats.com/rider/jonas-vingegaard/results , using the filter, copying URL and using it as URL for the RiderResults scraping class.
The new URL will look like that: https://www.procyclingstats.com/rider.php?xseason=&zxseason=&pxseason=equal&sort=date&race=&km1=&zkm1=&pkm1=equal&limit=200&topx=&ztopx=&ptopx=smallerorequal&znation=&type=&continent=&pnts=&zpnts=&ppnts=largerorequal&level=&rnk=&zrnk=&prnk=equal&exclude_tt=0&racedate=&zracedate=&pracedate=equal&name=&pname=contains&category=&profile_score=&zprofile_score=&pprofile_score=largerorequal&exclude_gcs=0&vert_meters=&zvert_meters=&pvert_meters=largerorequal&uci_pnt=&zuci_pnt=&puci_pnt=largerorequal&filter=Filter&id=jonas-vingegaard&p=results

It can be shortened if you play with it a little, but this one should work too. In general it's possible to obtain some results in one request only if it is possible to see them on one PCS page with some URL. So if you want to get more you need to play a little with filters on the page, get some URL that these filters produce and use it in code.