TDM 10100: Project 3 — Fall 2026

  • Exploratory Data Analysis and Data Types (categorical data, numerical, strings, geospatial/temporal data, indexing)

Motivation: In Project 3, we learn about various types of data that we encounter in real-world data sets.

Context: We will continue to explore data with both R and Python.

Scope: exploratory data analysis, data types, R, Python

Learning Objectives:

  • Demonstrate how to perform exploratory data analysis

  • Learn about various Data Types (categorical data, numerical, strings, geospatial/temporal data, indexing)

Again, please use the template found here, and the important information about project submissions here, when you are ready to submit your work.

Dataset(s)

This project will use the following dataset:

  • /anvil/projects/tdm/data/icecream/combined/reviews.csv

  • /anvil/projects/tdm/data/flights/subset/airports.csv

  • /anvil/projects/tdm/data/zillow/cities_crosswalk.csv

  • /anvil/projects/tdm/data/starwars/characters.csv

  • /anvil/projects/tdm/data/grouping/googleplaystore.csv

  • /anvil/projects/tdm/data/formula_1/circuits.csv

  • /anvil/projects/tdm/data/bay_area_bike_share/baywheels/202507-baywheels-tripdata.csv

In this project, please only use 2 cores in your Jupyter Lab session:

notebook.anvilcloud.rcac.purdue.edu/

(do not use 4 cores or 16 cores for this project)

Questions

Question 1 (2 pts)

Data is often given as a string. For example, we provide some examples here:

the-examples-book.com/projects/fall2026/10100/project3examples1

in R, about the ice cream combined reviews data, stored here:

/anvil/projects/tdm/data/icecream/combined/reviews.csv

and in Python, about the airports data set

/anvil/projects/tdm/data/flights/subset/airports.csv

For Question 1, use the Zillow data stored here:

/anvil/projects/tdm/data/zillow/cities_crosswalk.csv

In R, use the table, sort, and tail on the state column myDF$State to find the top 10 states in the Zillow cities crosswalk data.

Similarly, in Python, use value_counts and head on the state column myDF['State'] to find the top 10 states in the Zillow cities crosswalk data.

Deliverables

  • Use R to find the top 10 states in the Zillow cities crosswalk data.

  • Then use Python to again find the top 10 states in the Zillow cities crosswalk data.

  • As always, be sure to document your work from Question 1 (and from all of the questions!), using some comments and insights about your work. We will stop adding this note to document your work, but please remember, we always assume that you will document every single question with your comments and your insights.

Question 2 (2 pts)

In the Star Wars data:

/anvil/projects/tdm/data/starwars/characters.csv

The height and weight are numerical data, given as real numbers.

Show the head of the height and the weight columns in Python. Then use mean in Python to find the average height and also the average weight of the Star Wars characters.

Then do the same in R, namely, show the head of the height and the weight columns in R. Then use mean in R to find the average height and also the average weight of the Star Wars characters. When you do this in R, you need to use na.rm=TRUE as discussed here, for example:

www.datacamp.com/tutorial/na-rm-in-r

As a side note: coincidentally, the 4 droids are the ones who do not have their weight given! BB-8 does not have a given height either, as you can see here:

`myDF[is.na(myDF$weight), ]`

The hair_color and eye_color are categorical variables, in other words, they specify a category.

Show the head of the hair_color and the eye_color columns in R. Then use table in R to find the number of Star Wars characters with each hair_color and with each eye_color.

Then do the same in Python, namely, show the head of the hair_color and the eye_color columns in Python. Then use value_counts in Python to find the number of Star Wars characters with each hair_color and with each eye_color.

Deliverables

  • Show the head of the height and the weight columns in Python and R

  • Then show the average values of the height and the weight columns in Python and R

  • Show the head of the hair_color and the eye_color columns in R and Python

  • Then find the number of Star Wars characters with each hair_color and with each eye_color in R and Python.

  • It does not matter which order that you do these, but please organize your work in such a way that you include some comments (as always!) and the TA can understand what you did.

Question 3 (2 pts)

In the Google Play Store data:

/anvil/projects/tdm/data/grouping/googleplaystore.csv

the column Last.Updated stores the date as a string. It is common to need to know how to convert data from strings to dates, and there are many ways to do this, in both Python and R.

For instance, in R, if you look at the head of the data after you load it into a data frame called myDF:

head(myDF)

then you will see that the first few dates are stored as "January 7, 2018", "January 15, 2018", "August 1, 2018", "June 8, 2018", "June 20, 2018", "March 26, 2017", etc.

head(myDF$Last.Updated)

You may not see the double-quotes, but we write double-quotes when data is stored as a string. It is necessary to transform the strings into dates.

In R, we can use a method like this:

as.Date(myDF$Last.Updated, format = "%B %d, %Y")

and if you load the lubridate package:

library(lubridate)

then you can use the year function to extract the year from the dates. Then you can use the table function to see how many entries there are per year.

In Python, you will notice some differences. For instance, the column of data was called myDF$Last.Updated in R, but it will be called myDF['Last Updated'] (with a space instead of a period) in Python. You can see the first several values as follows:

myDF['Last Updated'].head()

Notice also that Python starts numbering the rows from 0, and for comparison, R starts numbering the rows from 1. This is simply a difference in how Python and R handle their indices.

If you have Pandas loaded:

import pandas as pd

Then we can use a method like this:

pd.to_datetime(myDF['Last Updated'], format="%B %d, %Y")

and we notice that the format is very similar to R, but also there is an error.

In this case, R dealt with the error automatically, and Python did not. We can add the option errors="coerce" after the format, and then Python will convert the strings to dates properly.

then you can use the dt.year function to extract the year from the dates. Then you can use the value_counts() function to see how many entries there are per year. The output will look strange, because Python is trying to deal with the value that was not formatted correctly, so it does not know that you want integers as output. Instead of using:

.dt.year.value_counts()

you might try something like:

dt.year.astype("Int64").value_counts()

to force the output to be integers for the years.

Deliverables

  • Use R to find the number of entries from the Google Play Store data for each year

  • Then use Python to find the number of entries from the Google Play Store data for each year

Question 4 (2 pts)

Geospatial data can be easily mapped in Python and R.

For example, in this data set:

/anvil/projects/tdm/data/formula_1/circuits.csv

we can see the latitude and longitude for the locations of F1 racetracks around the world.

To plot the locations of these racetracks in R, we can use the sf and leaflet libraries. We convert the longitudes and latitudes to WGS 84 format (you can read about that here)

en.wikipedia.org/wiki/World_Geodetic_System

and then display those locations with Leaflet:

library(sf)
library(leaflet)

myDF <- read.csv("/anvil/projects/tdm/data/formula_1/circuits.csv")

points <- st_as_sf(myDF, coords=c("lng", "lat"), crs=4326)

leaflet(width = "500px", height = "500px") %>%
      addTiles() %>%
      addMarkers(data = points)

Similarly, in Python, we can plot the locations of these racetracks, using geopandas and folium.

import pandas as pd
import geopandas as gpd
import folium
myDF = pd.read_csv('/anvil/projects/tdm/data/formula_1/circuits.csv')
my_map = folium.Map(location=[39.77, -86.29], zoom_start=1)
points = gpd.GeoDataFrame(myDF, geometry=gpd.points_from_xy(myDF.lng, myDF.lat), crs="EPSG:4326")
for index, row in points.iterrows():
    folium.Marker(
        location=[row["lat"], row["lng"]]
    ).add_to(my_map)
my_map
Deliverables

  • Render the map of the F1 racetracks in R.

  • Then render the map of the F1 racetracks in Python.

Question 5 (2 pts)

Classify each of the columns from this data set:

/anvil/projects/tdm/data/bay_area_bike_share/baywheels/202507-baywheels-tripdata.csv

namely:

"ride_id"
"rideable_type"
"started_at"
"ended_at"
"start_station_name"
"start_station_id"
"end_station_name"
"end_station_id"
"start_lat"
"start_lng"
"end_lat"
"end_lng"
"member_casual"

as a string, categorical variable, geospatial data, or date/timestamp (i.e., temporal data). Be sure to briefly justify/explain your answers.

Deliverables

  • Classify each of the types of data from the Bay Area Bike Share Baywheels data, as a string, categorical variable, geospatial data, or date/timestamp (i.e., temporal data). Be sure to briefly justify/explain your answers.

Submitting your Work

Please make sure that you added comments for each question, which explain your thinking about your method of solving each question. Please also make sure that your work is your own work, and that any outside sources (people, internet pages, generative AI, etc.) are cited properly in the project template.

Prior to submitting your work, you need to put your work into the project template, and re-run all of the code in Jupyter Lab and make sure that the results of running that code is visible in your template. Please check the detailed instructions on how to ensure that your submission is formatted correctly. To download your completed project, you can right-click on the file in the file explorer and click 'download'.

Once you upload your submission to Gradescope, make sure that everything appears as you would expect to ensure that you don’t lose any points. We hope your first project with us went well, and we look forward to continuing to learn with you on future projects!!

Items to submit

  • firstname_lastname_project3.ipynb

It is necessary to document your work, with comments about each solution. All of your work needs to be your own work, with citations to any source that you used. Please make sure that your work is your own work, and that any outside sources (people, internet pages, generative AI, etc.) are cited properly in the project template.

You must double check your .ipynb after submitting it in gradescope. A very common mistake is to assume that your .ipynb file has been rendered properly and contains your code, markdown, and code output even though it may not.

Please take the time to double check your work. See here for instructions on how to double check this.

You will not receive full credit if your .ipynb file does not contain all of the information you expect it to, or if it does not render properly in Gradescope. Please ask a TA if you need help with this.