Classwork 8

Collecting Data with Python requests and APIs

Question 1

Provide a Python code to get New York City air quality surveillance data using the following API endpoint:

https://data.cityofnewyork.us/resource/c3uy-2p5r.json

• Use the following query parameter dictionary in your request, which sets the “$limit” to control how many rows are returned:

# query parameters (e.g., limit rows)
param_dict = {
    "$limit": 100   # You can increase this to get more observations
}

Show answer

import requests
import pandas as pd

endpoint = 'https://data.cityofnewyork.us/resource/c3uy-2p5r.json'  ## API endpoint

# query parameters (e.g., limit rows)
param_dict = {
    "$limit": 100   # You can increase this to get more observations
}

response = requests.get(endpoint,
                        params= param_dict)

content = response.json() # to convert JSON response data to a dictionary/ a list of dicts


df = pd.DataFrame(content)

Question 2

Use a for-loop, requests, and the FRED API to retrieve two macroeconomic indicators:

1. U.S. Real GDP (series_id = "GDPC1")
2. U.S. Unemployment Rate (series_id = "UNRATE")

Store them into one single DataFrame called df_all.

The df_all DataFrame contains both FRED series ids for (1) U.S. Real GDP (series_id = "GDPC1") and (2) U.S. Unemployment Rate (series_id = "UNRATE"), and has three columns as shown below:

Note

df_all = df_all[ ['date', 'value', 'series'] ] means:

• “keep only these columns, and relocate them in this order.”

Show answer

import requests  # to handle API requests
import pandas as pd
param_dicts = {
  'api_key': 'YOUR_API_KEY', ## Change to your own key
  'file_type': 'json',
  'series_id': 'GDPC1'    ## ID for US real GDP
}

url = "https://api.stlouisfed.org/"
endpoint = "series/observations"
api_endpoint = url + "fred/" + endpoint   # sum of strings

lst_series = ['GDPC1', 'UNRATE']
df_all = pd.DataFrame()
for val in lst_series:
    
    param_dicts['series_id'] = val
    response = requests.get(api_endpoint, 
                            params = param_dicts)

    # Convert JSON response to Python dictionary.
    content = response.json() 

    # Extract the "observations" list element.
    df = pd.DataFrame( content['observations'] )
    df['series'] = val
    
    df_all = pd.concat([df_all, df], ignore_index=True)

df_all.columns

# df[ LIST ]
df_all = df_all[ ['date', 'value', 'series'] ]

Discussion

Welcome to our Classwork 8 Discussion Board! 👋

This space is designed for you to engage with your classmates about the material covered in Classwork 8.

Whether you are looking to delve deeper into the content, share insights, or have questions about the content, this is the perfect place for you.

If you have any specific questions for Byeong-Hak (@bcdanl) regarding the Classwork 8 materials or need clarification on any points, don’t hesitate to ask here.

All comments will be stored here.

Let’s collaborate and learn from each other!