Welcome to our lesson on Using Query Parameters with requests
and Working with REST! In this lesson, we will explore how to use query parameters with Python's requests
library and extract data from REST APIs. By the end of this lesson, you will have solidified your knowledge in data retrieval and will be able to effectively use query parameters and REST APIs to fetch data, laying the foundation for your future web-scraping projects.
Let's first talk about what query parameters are. Query parameters, also known as query strings, are used to send data to the server in the form of key-value pairs. They are attached to the end of a URL after a '?' character and separated by '&' for multiple parameters. For example, if you ever filtered a search result on a website and noticed your URL change to something like this http://website.com/search?param1=value1¶m2=value2
, those are query parameters in action!
Python's requests
library offers a simple way to pass those query parameters. The requests.get()
method accepts a parameter params
that can be used to specify these. Let's illustrate this in the code we have:
Python1import requests 2 3url_action_api = 'https://en.wikipedia.org/w/api.php' 4params_action_api = { 5 'action': 'query', 6 'prop': 'info', 7 'titles': 'Earth', 8 'format': 'json' 9} 10 11response_action_api = requests.get(url_action_api, params=params_action_api)
Here, params_action_api
is a dictionary of key-value pairs, which specify the parameters to be included in the query string. requests.get()
then constructs the URL with these parameters.
When we fetch data from the server, it often comes back in JSON (JavaScript Object Notation) format, which is a lightweight data-interchange format that is easy to read and write. We can use the response.json()
function to convert this JSON response into a Python dictionary for us to easily manipulate the data:
Python1if response_action_api.ok: 2 print("Content from Wikipedia's action API fetched successfully!") 3 print(response_action_api.json()) 4else: 5 print("Failed to fetch content from Wikipedia's action API.")
REST APIs are a type of web service that allows communication between different systems over the internet. They are based on the principles of REST, which stands for Representational State Transfer. REST APIs use standard HTTP methods like GET, POST, PUT, DELETE, etc., to perform operations on the server. REST APIs return data in JSON format, which can be easily parsed using Python's requests
library.
REST APIs usually support CRUD (Create, Retrieve, Update, Delete) operations. For example, you can use a REST API to fetch data from a server, update data, or delete data. REST APIs are stateless, meaning each request from a client to the server must contain all the information needed to understand the request. This makes REST APIs easy to use and understand.
Here are some common HTTP methods used in REST APIs:
In the next section, we will explore how to interact with REST APIs using Python's requests
library.
The Python requests
library simplifies the process of working with REST APIs. To send a GET request and fetch data from a REST API, we use the requests.get()
method. Take a look at the second part of the example code where we interact with Wikipedia's REST API:
Python1url_rest_api = 'https://en.wikipedia.org/api/rest_v1/page/title/Earth' 2response_rest_api = requests.get(url_rest_api)
We can then parse the response JSON data the same way as before:
Python1if response_rest_api.ok: 2 print("\nContent from Wikipedia's REST API fetched successfully!") 3 print(response_rest_api.json()) 4else: 5 print("Failed to fetch content from Wikipedia's REST API.")
The output of the above code will be:
1Content from Wikipedia's REST API fetched successfully! 2{'items': [{'title': 'Earth', 'page_id': 9228, 'rev': 1222764383, 'tid': 'fb092700-0d3a-11ef-ba41-6f469669db51', 'namespace': 0, 'user_id': 47778926, 'user_text': 'Editor314159', 'timestamp': '2024-05-07T19:42:03Z', 'comment': '', 'tags': ['visualeditor'], 'restrictions': [], 'page_language': 'en', 'redirect': False}]}
This output summarizes the REST API's response for fetching details about the "Earth" page on Wikipedia. It shows various pieces of information including the page ID, revision ID, user ID of the last editor, and more in a dictionary format.
Let's explore another example where we delete a resource using the DELETE method:
Python1# DELETE request to delete a resource using an experimental API 2url_delete_api = 'https://jsonplaceholder.typicode.com/posts/1' 3response_delete_api = requests.delete(url_delete_api) 4 5print(response_delete_api.status_code) # Output: 200
In this example, we are using the requests.delete()
method to delete the resource with ID 1 from the JSONPlaceholder API. The response will indicate whether the deletion was successful or not. In this case, the status code 200
indicates that the deletion was successful.
In this lesson, we covered how to use Python's requests
library to efficiently work with query parameters and fetch data from REST APIs. These skills are crucial when you need to retrieve data from the web for analysis or web-scraping projects.
Now that we've walked through how to structure requests and handle the responses, it's time to apply these skills. In the following practice exercises, you will be given opportunities to work with different APIs and requests configurations, which will strengthen your understanding of these topics. Remember, practical application is key in accelerating your learning journey. So let's get started!