TDM 10100: Project 5 Question 4 examples

In Python, we can read in data from Parquet format (a compressed format) into a data frame as follows:

import pandas as pd
myDF = pd.read_parquet("/anvil/projects/tdm/data/taxi/yellow_2025/yellow_tripdata_2025-09.parquet")

This data is from Yellow Taxi Cabs from September 2025.

We can consider the head of the data:

myDF.head()

For instance, the Airport_fee column is sometimes 0.00 and is sometimes non-zero. There are altogether 4251015 rows and 20 columns:

myDF.shape

Exactly 1369541 rows have nonzero values for the Airport_fee column:

myDF[myDF["Airport_fee"] != 0].shape

and the other 2881474 rows have 0.00 values for the Airport_fee column:

myDF[myDF["Airport_fee"] == 0].shape

We can use the value_counts and the head to find the 10 values of the DOLocationID column that have more than 100000 rows each. For instance, the DOLocationID 237 has occurs on 168597 rows:

myDF['DOLocationID'].value_counts().head(n=20)