Welcome to our lesson on Using Query Parameters with requests and Working with REST! In this lesson, we will explore how to use query parameters with Python's requests library and extract data from REST APIs. By the end of this lesson, you will have solidified your knowledge in data retrieval and will be able to effectively use query parameters and REST APIs to fetch data, laying the foundation for your future web-scraping projects.

Let's first talk about what query parameters are. Query parameters, also known as query strings, are used to send data to the server in the form of key-value pairs. They are attached to the end of a URL after a '?' character and separated by '&' for multiple parameters. For example, if you ever filtered a search result on a website and noticed your URL change to something like this http://website.com/search?param1=value1&param2=value2, those are query parameters in action!

Python's requests library offers a simple way to pass those query parameters. The requests.get() method accepts a parameter params that can be used to specify these. Let's illustrate this in the code we have:

Python
Copy to clipboard
1import requests
2
3url_action_api = 'https://en.wikipedia.org/w/api.php'
4params_action_api = {
5    'action': 'query',
6    'prop': 'info',
7    'titles': 'Earth',
8    'format': 'json'
9}
10
11response_action_api = requests.get(url_action_api, params=params_action_api)

Here, params_action_api is a dictionary of key-value pairs, which specify the parameters to be included in the query string. requests.get() then constructs the URL with these parameters.

When we fetch data from the server, it often comes back in JSON (JavaScript Object Notation) format, which is a lightweight data-interchange format that is easy to read and write. We can use the response.json() function to convert this JSON response into a Python dictionary for us to easily manipulate the data:

Python
Copy to clipboard
1if response_action_api.ok:
2    print("Content from Wikipedia's action API fetched successfully!")
3    print(response_action_api.json())
4else:
5    print("Failed to fetch content from Wikipedia's action API.")

REST APIs are a type of web service that allows communication between different systems over the internet. They are based on the principles of REST, which stands for Representational State Transfer. REST APIs use standard HTTP methods like GET, POST, PUT, DELETE, etc., to perform operations on the server. REST APIs return data in JSON format, which can be easily parsed using Python's requests library.

REST APIs usually support CRUD (Create, Retrieve, Update, Delete) operations. For example, you can use a REST API to fetch data from a server, update data, or delete data. REST APIs are stateless, meaning each request from a client to the server must contain all the information needed to understand the request. This makes REST APIs easy to use and understand.

Here are some common HTTP methods used in REST APIs:

• GET: Used to retrieve data from the server.
• POST: Used to send data to the server to create a new resource.
• PUT: Used to send data to the server to update an existing resource.
• DELETE: Used to delete a resource on the server.

In the next section, we will explore how to interact with REST APIs using Python's requests library.

The Python requests library simplifies the process of working with REST APIs. To send a GET request and fetch data from a REST API, we use the requests.get() method. Take a look at the second part of the example code where we interact with Wikipedia's REST API:

Python
Copy to clipboard
1url_rest_api = 'https://en.wikipedia.org/api/rest_v1/page/title/Earth'
2response_rest_api = requests.get(url_rest_api)

We can then parse the response JSON data the same way as before:

Python
Copy to clipboard
1if response_rest_api.ok:
2    print("\nContent from Wikipedia's REST API fetched successfully!")
3    print(response_rest_api.json())
4else:
5    print("Failed to fetch content from Wikipedia's REST API.")

The output of the above code will be:

Copy to clipboard
1Content from Wikipedia's REST API fetched successfully!
2{'items': [{'title': 'Earth', 'page_id': 9228, 'rev': 1222764383, 'tid': 'fb092700-0d3a-11ef-ba41-6f469669db51', 'namespace': 0, 'user_id': 47778926, 'user_text': 'Editor314159', 'timestamp': '2024-05-07T19:42:03Z', 'comment': '', 'tags': ['visualeditor'], 'restrictions': [], 'page_language': 'en', 'redirect': False}]}

This output summarizes the REST API's response for fetching details about the "Earth" page on Wikipedia. It shows various pieces of information including the page ID, revision ID, user ID of the last editor, and more in a dictionary format.

Let's explore another example where we delete a resource using the DELETE method:

Python
Copy to clipboard
1# DELETE request to delete a resource using an experimental API
2url_delete_api = 'https://jsonplaceholder.typicode.com/posts/1'
3response_delete_api = requests.delete(url_delete_api)
4
5print(response_delete_api.status_code) # Output: 200

In this example, we are using the requests.delete() method to delete the resource with ID 1 from the JSONPlaceholder API. The response will indicate whether the deletion was successful or not. In this case, the status code 200 indicates that the deletion was successful.

In this lesson, we covered how to use Python's requests library to efficiently work with query parameters and fetch data from REST APIs. These skills are crucial when you need to retrieve data from the web for analysis or web-scraping projects.

Now that we've walked through how to structure requests and handle the responses, it's time to apply these skills. In the following practice exercises, you will be given opportunities to work with different APIs and requests configurations, which will strengthen your understanding of these topics. Remember, practical application is key in accelerating your learning journey. So let's get started!