TDM 10200: Python Project 1 — Spring 2025

Motivation: Happy New Year! In this project, we will get comfortable loading some data sets in Python and making some simple visualizations.

Context: No background knowledge in Python is needed. It is OK if you are totally new to Python. We will learn together, step by step, working with data in a practical way.

Scope: Anvil, Jupyter Lab, Python

Learning Objectives:

  • Learn how to load data sets in Python using Pandas

  • Make just a few plots using data in Python using matplotlib

Make sure to read about, and use the template found here, and the important information about project submissions here.

Dataset(s)

This project will use the following dataset(s):

  • /anvil/projects/tdm/data/flights/subset/airports.csv

  • /anvil/projects/tdm/data/icecream/breyers/reviews.csv

  • /anvil/projects/tdm/data/icecream/bj/products.csv

Questions

Question 1 (2 pts)

If you have been in The Data Mine in fall 2024, you can scan quickly through this question, and simply load the head of the airports subset data, as described in the videos. If you are new in The Data Mine this spring, it should be helpful to read carefully through this question.

First and foremost, welcome to The Data Mine! We hope that throughout your journey with us, you learn a lot, make new friends, and develop skills that will help you with your future career. Throughout your time with The Data Mine, you will have plenty of resources available should you need help. By coming to weekly seminar, posting on the class Piazza page, and joining Dr. Ward and the TA team’s office hours, you can ensure that you always have the support you need to succeed in this course.

The links to Piazza are:

TDM 10200 Piazza link

TDM 20200 Piazza link

TDM 30200 Piazza link

TDM 40200 Piazza link

Dr Ward is also available on Monday mornings in the Hillenbrand dining court from 8:30 AM to 11:20 AM (Eastern time zone). He is also available on Monday afternoons during 4:30 PM to 5:20 PM on Zoom at https://purdue-edu.zoom.us/my/mdward/

If you did not (yet) set up your 2-factor authentication credentials with Duo, you can set up the credentials here: https://the-examples-book.com/setup If you are still having issues with your ACCESS ID, please send an email containing as much information as possible about your issue to [email protected]

Let’s start off by starting up our first Jupyter session on Anvil! We always use the URL https://ondemand.anvil.rcac.purdue.edu and the ACCESS username that you were assigned (when you setup your account) and the ACCESS password that you chose. These are NOT the same as your Purdue account!

These credentials are not the same as your Purdue account.

In the upper-middle part of your screen, you should see a dropdown button labeled The Data Mine. Click on it, then select Jupyter Notebook from the options that appear. If you followed all the previous steps correctly, you should now be on a screen that looks like this:

OnDemand Jupyter Lab
Figure 1. OnDemand Jupyter Lab

If your screen doesn’t look like this, please try and select the correct dropdown option again or ask one of the TAs during seminar for more assistance.

Jupyter Lab and Jupyter Notebook are technically different things, but in the context of seminar we will refer to them interchangeably to represent the tools you’ll be using to work on your projects.

There are a few key parts of this screen to note:

  • Allocation: this should always be cis220051 for The Data Mine

  • Queue: again, this should stay on the default option shared unless otherwise noted.

  • Time in Hours: The amount of time your Jupyter session will last. When this runs out, you’ll need to start a new session. It may be tempting to set it to the maximum, but our computing cluster is a shared resource. This means every hour you use is an hour someone else can’t use, so please ONLY reserve it for 1-2 hours at a time.

  • CPU cores: CPU cores do the computation for the programs we run. It may be tempting to request a large number of CPU cores and set it to the maximum, but our computing cluster is a shared resource. This means every computational core that you use is one that someone else can’t use. We only have a limited number of cores assigned to our team, so please ONLY reserve 1 or 2 cores at a time, unless the project needs more cores.

The Jupyter Lab environment will save your work at regular intervals, so that at the end of a session, your work should be automatically saved. Nonetheless, you can select File from the menu and Save Notebook any time that you want. (It is not necessarily, because the notebooks save automatically, but you can still save anytime if you want to.)

With the key parts of this screen explained, go ahead and select 1 hour of time with 1 CPU cores and click Launch! After a bit of waiting, you should see something like below. Click connect to Jupyter and proceed to the next question!

Launch Jupyter Lab
Figure 2. Launch Jupyter Lab

You likely noticed a short wait before your Jupyter session launched. This happens while Anvil finds and allocates space for you to work. The more students are working on Anvil, the longer this will take, so it is our suggesting to start your projects early during the week to avoid any last-minute hiccups causing a missed deadline. Please do not wait until Fridays to complete and submit your work!

In Spring 2025, we are trying something new. The projects are due on Wednesdays (instead of Fridays!). You can see the schedule here: the-examples-book.com/projects/spring2025/10200/projects Please do not wait until Wednesday to complete and submit your work!

Download the project template, as described here: https://the-examples-book.com/projects/templates

When you first open the template, you may get a pop-up asking you to select what kernel you’ll be using. Select the seminar kernel (not the seminar-r kernel). If you do not get this pop-up, you can also select a kernel by clicking on the upper right part of your screen that likely says something similar to No Kernel, and then selecting the kernel you want to use.

We give some information about kernels here: the-examples-book.com/projects/kernels

As you continue to get comfortable with Jupyter Lab, you might want to read more about Jupyter Lab (this is optional).

For the first question in this project, let’s try the first example from the kernel page: We will load the airports data set in Python and will display the head of the airports data set.

import pandas as pd
myDF = pd.read_csv("/anvil/projects/tdm/data/flights/subset/airports.csv")
myDF.head()

Just try this Python code using the seminar kernel (not the seminar-r kernel) and make sure that you can see the first five rows of the airports data frame.

Deliverables

  • Use Python to show the output with the first five rows of the airports data frame.

  • Be sure to document your work from Question 1, using some comments and insights about your work.

Question 2 (2 pts)

In Python, we often use the Pandas library for loading DataFrames. Pandas allows us to check some properties of our data frame. For instance, we can use the shape property to see how many rows and columns our DataFrame has:

myDF.shape

Notice that Python starts counting from 0 (as opposed to R, which starts counting from 1). So the initial row of the Pandas DataFrame is row 0. In the head of the DataFrame, as we saw in Question 1, we see rows 0, 1, 2, 3, 4.

Now load the tail of the DataFrame. It displays rows 3371, 3372, 3373, 3374, 3375. As indicated by the shape parameter, this DataFrame has 3376 rows altogether, so this makes sense.

myDF.tail()

We can select rows of the data frame that meet certain conditions. For instance, we can extract the airports located in New York City as follows:

myDF[(myDF['city'] == 'New York') & (myDF['state'] == 'NY')]

Now try it yourself! After you display the airports in New York City, then please display the airports from Indianapolis, IN, and also from Houston, TX. Please note that you need to specify the city and the state for this to work. If you forget the state on the Indianapolis query, it will be OK, because no other city in the country is named Indianapolis. BUT if you forget the state for Houston, you will get some airports that are not in Houston, TX, but instead, are from other states. For this reason, you need to always include the conditions on the city and the state of the desired location.

For each question in The Data Mine, please always be sure to put some comments after your cells, which describe all of the work that you are doing in the cells, as well as your thinking and insights about the results.

Some common Jupyter notebooks shortcuts:

  • Instead of clicking the play button, you can press ctrl+enter (or cmd+enter on Mac) to run a cell.

  • If you want to run a cell and then move immediately to the next cell, you can use shift+enter. This is oftentimes more useful than ctrl+enter

  • If you want to run the current cell and then immediately create a new code cell below it, you can press alt+enter (or option+enter on Mac) to do so.

  • When a cell is selected (this means you clicked next to it, and it should show a blue bar to its left to signify this), pressing the d key twice will delete that cell.

  • When a cell is selected, pressing the a key will create a new code cell `a`bove the currently selected cell.

  • When a cell is selected, pressing the b key will create a new code cell `b`elow the selected cell
Deliverables

  • Use the shape property to see how many rows and columns are in the data frame with the airports data.

  • Display the tail of the DataFrame.

  • Display the airports located in New York, NY.

  • Display the airports located in Indianapolis, IN.

  • Display the airports located in Houston, TX.

  • Be sure to document your work from Question 2, using some comments and insights about your work.

Question 3 (2 pts)

For this question, we only pay attention to the state (not the city) for each airport. Which state has the largest number of airports? How many airports are located in that state? We can extract (only) the states from each airport by writing:

myDF['state']

and then the value_counts function gives the number of airports in each state:

myDF['state'].value_counts()

You do not need to make a plot of the value counts for the airports, but you are welcome to try this, if you want to!
Deliverables

  • Use the value_counts function to find the number of airports in each state.

  • Be sure to document your work from Question 3, using some comments and insights about your work.

Question 4 (2 pts)

Now load the ice cream products data set, stored here:

/anvil/projects/tdm/data/icecream/combined/products.csv

Using the value_counts function with the brand column from the ice cream products data set, how many rows are associated with each brand? In other words, how many times does each brand occur in the brand column?

Now solve the same question with a different data set. First load the head of the ice cream reviews data set, and then find how many times each brand occurs in the ice cream reviews data set:

/anvil/projects/tdm/data/icecream/combined/reviews.csv

How many times does each brand occur in the reviews.csv data set?

You work will be similar to the work from Question 3. Be sure to document each question with comments about your work.

Deliverables

  • Use the value_counts function to find the number of times that each brand occurs in the ice cream products data set.

  • Then use the value_counts function again, to find the number of times that each brand occurs in the ice cream reviews data set.

  • Be sure to document your work from Question 4, using some comments and insights about your work.

Question 5 (2 pts)

Now we use matplotlib to display the value_counts from Question 4. To accomplish this, we first load matplotlib as follows:

import matplotlib.pyplot as plt

Then we can save our counts into a variable, which we choose to call mycounts like this:

mycounts = myDF['brand'].value_counts()

and finally we can plot the names of the brands (which are the index values in mycounts) on the x-axis, and the values on the y-axis, as follows:

plt.bar(mycounts.index, mycounts)

Give this a try yourself, with BOTH of the data sets from Question 4.

In other words, make one plot that shows the number of occurrences of the brand in the ice cream products data set, and then make another plot that shows the number of occurrences of the brand in the ice cream reviews data set.

Deliverables

  • Make one plot that shows the number of occurrences of the brand in the ice cream products data set, and then make another plot that shows the number of occurrences of the brand in the ice cream reviews data set.

  • Be sure to document your work from Question 5, using some comments and insights about your work.

Submitting your Work

Please make sure that you added comments for each question, which explain your thinking about your method of solving each question. Please also make sure that your work is your own work, and that any outside sources (people, internet pages, generating AI, etc.) are cited properly in the project template.

Congratulations! Assuming you’ve completed all the above questions, you’ve just finished your first project for TDM 10200! If you have any questions or issues regarding this project, please feel free to ask in seminar, over Piazza, or during office hours.

Prior to submitting your work, you need to put your work into the project template, and re-run all of the code in your Jupyter notebook and make sure that the results of running that code is visible in your template. Please check the detailed instructions on how to ensure that your submission is formatted correctly. To download your completed project, you can right-click on the file in the file explorer and click 'download'.

Once you upload your submission to Gradescope, make sure that everything appears as you would expect to ensure that you don’t lose any points. We hope your first project with us went well, and we look forward to continuing to learn with you on future projects!!

Items to submit

  • firstname_lastname_project1.ipynb

It is necessary to document your work, with comments about each solution. All of your work needs to be your own work, with citations to any source that you used. Please make sure that your work is your own work, and that any outside sources (people, internet pages, generating AI, etc.) are cited properly in the project template.

You must double check your .ipynb after submitting it in gradescope. A very common mistake is to assume that your .ipynb file has been rendered properly and contains your code, markdown, and code output even though it may not.

Please take the time to double check your work. See here for instructions on how to double check this.

You will not receive full credit if your .ipynb file does not contain all of the information you expect it to, or if it does not render properly in Gradescope. Please ask a TA if you need help with this.