/python_http_requests_perfs

A repo to demonstrate how to reduce processing time of HTTP call in python.

Primary LanguagePython

Http call performence

Description

This repo demonstrate how to reduce processing time of HTTP call in python.

As the requests module won't allow full performances, I used the httpx module.

This material is highly inspired from this video. I encourage you to see it for more explainations!

Other resources

Installation

To run this code you will need the httpx and requests module.

In your projects, your should choose one of them.

You can install it with:

  • pip install httpx
  • pip install requests

Code

You can find the here in main.py with some comments.

Perf report

Those are the result of GET requests made on https://quotable.io/random with a Macbook Pro 16" i9 processor on the 08/08/2022.

The results could change depending on the API but the gap should be more or less the same.

requests.get for loop

  • 50 requests = 24 sec
  • 150 requests = 54 sec

Httpx.get for loop

  • 50 requests = 24 sec
  • 150 requests = 54 sec

requests.Session.get SYNC for loop

  • 50 requests = 6 sec
  • 150 requests = 19 sec

Httpx.Client.get SYNC for loop

  • 50 requests = 6 sec
  • 150 requests = 19 sec

Httpx.AsyncClient.get ASYNC for loop

  • 50 requests = 6 sec
  • 150 requests = 19 sec

Httpx.AsyncClient.get ASYNC tasks

  • 50 requests = 0.8 sec
  • 150 requests = 1 sec
  • 1000 requests = 9 sec