TDM 10100: Project 6 Question 4 examples
Now we can load the Zillow data into Pandas using pd.read_csv but we need to first load Pandas with import pandas as pd
import pandas as pd
myDF = pd.read_csv("/anvil/projects/tdm/data/zillow/State_time_series.csv")
There is a column called Date:
myDF.head()
and we can extract the months from these dates. For instance, the first six values are all from April, the fourth month:
pd.to_datetime(myDF['Date']).dt.month.head()
We can also see how many entries occurred per year:
pd.to_datetime(myDF['Date']).dt.month.value_counts()
The Iowa Liquor Sales data is too large to read the entire data set. If you try to read it all, your kernel will crash. (If you try, and you crash your kernel, then you need to re-load the data.table library again, from the start of the examples.)
myDF = pd.read_csv("/anvil/projects/tdm/data/iowa_liquor_sales/iowa_liquor_sales.csv")
So we can load just a few of the columns, for instance, the Date, Store Number, and Store Name:
myDF = pd.read_csv("/anvil/projects/tdm/data/iowa_liquor_sales/iowa_liquor_sales.csv", usecols=["Date", "Store Number", "Store Name"])
and you can see the first few entries:
myDF.head()
This time, if we try to extract the months or years with pd.to_datetime, it works in a straightforward way in Pandas.
pd.to_datetime(myDF['Date']).dt.year.value_counts()
We can even extract the months and the years together,
by using dt.to_period("M")
pd.to_datetime(myDF['Date']).dt.to_period("M").value_counts()
In the year 2021, there were 2622712 rows of data:
myDF2021 = myDF[pd.to_datetime(myDF['Date']).dt.year == 2021]
and the store called HY-VEE #3 / BDI / DES MOINES had 19168 lines of data corresponding to 2021:
myDF2021["Store Name"].value_counts()
If we focus only on June 2021:
myDFJune2021 = myDF[ (pd.to_datetime(myDF['Date']).dt.year == 2021) & (pd.to_datetime(myDF['Date']).dt.month == 6)]
we can plot the number of rows of data per day:
import matplotlib.pyplot as plt
myDFJune2021["Date"].value_counts().sort_index().plot()