How to use Python Requests Module Like a Pro

by

How to use Python Requests Module Like a Pro

In this tutorial, we will learn about how to use Python requests module like a pro. In Python, requests module is widely used crucial tool to send HTTP methods (GET, POST, PUT, PATCH and DELETE). It returns a response object which includes the response status along with the content. We will learn more about this module in the upcoming section using some examples. So, let us begin the tutorial.

 

Python Modules used in this Tutorial

  • time Module
  • json Module
  • requests Module (Show’s Topper)
  • colorama Module

 

NOTE:

To install any Python module use the command pip install <module-name> or pipenv install <module-name>.

 

Python requests Module Overview

In Python, requests module is a popular library which is widely used for making HTTP requests. This module helps the developer to retrieve data from a RESTful API, submit a form on a website or interact with any web service in  easy way. It provides different functions to call HTTP methods. There are different HTTP methods available. These are:

  • GET 
  • POST
  • PUT
  • PATCH
  • DELETE

For each HTTP method there is corresponding function provided by requests module like requests.get(), requests.post(), requests.put(), requests.patch() and requests.delete(). For each HTTP request sent, a response object is returned which contains the status code and the content requested during HTTP request call. The status code determines if the request is successful or not. Please note that it is beyond the scope of this tutorial to cover each HTTP request method. So, we will only talk about requests.get() method which is used to send HTTP GET requests.

 

Prerequisites

  • Basic Knowledge on Python
  • Previously mentioned Python Modules are installed

 

How to use Python Requests Module Like a Pro

Also Read: How to Install python-socketio Stable Diffusion [5 Easy Steps]

I will first give you the overview of the use case which we will implement using all the modules mentioned earlier in this post. We will write a code to make multiple API calls at the same time. We will see the code below then will look at its execution flow.

Example: Make multiple API calls at the same time

import time
import requests, json
from colorama import Fore, Style

def callApi(url):

    response = requests.get(url)
    response_data = response.json()
    output = json.dumps(response_data, indent=2)
    return output

if __name__ == "__main__":

    start_time = time.time()

    urls = {
        "https://reqres.in/api/users/1",
        "https://reqres.in/api/users/2",
        "https://reqres.in/api/users/3",
        "https://reqres.in/api/users/4"
    }

    for url in urls:
        result = callApi(url)
        print(f" {Fore.GREEN}Response from '{url}': {Fore.WHITE}{result}\n")

    elapsed = time.time() - start_time
    print(f"Elapsed: {Fore.YELLOW}{elapsed:.2f}s")
    print(f"{Style.RESET_ALL}")

 

Code Explanation

  • Import time module to store the code start and end execution time. The difference of two will output the total time taken by the code to complete its execution.
  • Import requests module to make HTTP GET API call.
  • Import json module to convert the HTTP GET  response object into json format.
  • Import colorama module to color the text.
  • In the main function, store current time using time.time() function in variable “start_time”.
  • Create a list type variable called  “urls” which will store 5 url.
  • For each url in the list urls, make a call to the callApi() function. Store the function return value in the variable called “result“.
  • callAPI() function takes url as command line argument.
  • requests.get(url) method will send the HTTP GET request  fetch the response object. It will store the response object in variable called “response”.
  • json.dumps() function will convert the response_data into json format and store it in variable called “output”.
  • callAPI() will return “output” value  when this function is called.
  • Use print() function to print the result value on the standard output (console).
  • Use Fore.GREEN function to color the text. Similarly use different colors from Fore function to color the text (RED, WHITE etc.)
  • Below is the output returned by above code.

 

OUTPUT

Response from 'https://reqres.in/api/users/4': {
"data": {
"id": 4,
"email": "[email protected]",
"first_name": "Eve",
"last_name": "Holt",
"avatar": "https://reqres.in/img/faces/4-image.jpg"
},
"support": {
"url": "https://reqres.in/#support-heading",
"text": "To keep ReqRes free, contributions towards server costs are appreciated!"
}
}

Response from 'https://reqres.in/api/users/3': {
"data": {
"id": 3,
"email": "[email protected]",
"first_name": "Emma",
"last_name": "Wong",
"avatar": "https://reqres.in/img/faces/3-image.jpg"
},
"support": {
"url": "https://reqres.in/#support-heading",
"text": "To keep ReqRes free, contributions towards server costs are appreciated!"
}
}

Response from 'https://reqres.in/api/users/1': {
"data": {
"id": 1,
"email": "[email protected]",
"first_name": "George",
"last_name": "Bluth",
"avatar": "https://reqres.in/img/faces/1-image.jpg"
},
"support": {
"url": "https://reqres.in/#support-heading",
"text": "To keep ReqRes free, contributions towards server costs are appreciated!"
}
}

Response from 'https://reqres.in/api/users/2': {
"data": {
"id": 2,
"email": "[email protected]",
"first_name": "Janet",
"last_name": "Weaver",
"avatar": "https://reqres.in/img/faces/2-image.jpg"
},
"support": {
"url": "https://reqres.in/#support-heading",
"text": "To keep ReqRes free, contributions towards server costs are appreciated!"
}
}

Elapsed: 0.79s

 

Summary

Python’s requests module is quite versatile and widely used module for web scraping , interacting with APIs and also for web application development. It is developers go-to tool if dealing with any of the mentioned tasks in Python.

 

 

 

Leave a Comment