TDM 10200: Project 10 - Visualizations

Each year, the Centers for Disease Control and Prevention (CDC) publishes the most comprehensive data on mortality in the United States through the National Vital Statistics System. The dataset contains records of all deaths nationwide between 2005 and 2015, with detailed information on causes of death and demographic characteristics of the individuals.

The dataset used for this project contains information from the year 2014 about the mortality data records released by the CDC. This dataset is thorough and contains 38 columns and 2631171 rows of death record data. The columns cover everything from date of death to personal details (age, sex, marital status, etc.) to manner of death and more. Some of the columns may not make a lot of sense and that is OK. Feel free to explore this dataset as much as you would like to gain a better understanding.

Many of the columns only contain encoded values. There are accompanying datasets that will help you understand what is actually being shown. Some examples of these decoder files are provided in Question 1.

Some of the columns from myDF that we will work with include:

An example of what the decoder files may contain is:

Read in the Death Records dataset as myDF from /anvil/projects/tdm/data/death_records/DeathRecords.csv. This dataset is the main source of the death records data, but there are a few different dataset files that go along with myDF that contain the decoder data keys for what the values in some of the columns mean.

Read in the following datasets to ensure correct interpretation of the values from the specific columns Race, and MaritalStatus:

These datasets contain descriptions for what each of the code values of the columns of myDF represent. In the value counts of the column Race in myDF, there are relatively few unique values representing different races. For reference, use the Race.csv dataset to find out what the numerical values of this Race column mean. Also, the number of occurrences of the different values in the Race column varies greatly. Some categories have only a few hundred entries, while others have tens of thousands or more.

Make a plot of these counts to help visualize this. To make a very basic plot in Python, use .plot(). You can specify what kind of plot you want this to be (i.e. .plot(kind='bar')), but that is near the limit of what customization is useful without additional libraries.

Import matplotlib, and add a title, x- and y-axis labels. Finally, do plt.show() to display the finished plot.

The Race category coded as '1' appears 2241510 times. Compared to other categories with counts such as 700, 4913, or even 13297, this value is disproportionately high, making it difficult to visualize alongside the others on the same scale.

Create a new dataset containing only the entries from the Race column that are not equal to '1':

Use .plot() to show this series as either a line plot or a bar plot. Which method do you find more insightful for this data comparison?

Think about your new visualization. How many additional values would you exclude to make the visualization more interpretable? The '2' race values? '3'? More? Less? Try some of these and explain your final decision.

It is almost always a good idea to include as much data as was provided when making a comparison. This can be done, for example, by comparing how many death records of each race there were. But sometimes it is the case that removing data actually helps. If you look at your first plot from Question 1, it is not hard to see that you cannot really tell what is going on with the lower values because the '1' race has extended the upper y-limit so high that everything else is small by comparison.

When you do end up removing the '1' and other values to limit what races are shown, be sure to document this. Write a comment in your notebook about "I removed this value because…​.." and just explain it a little bit.

When you are finishing these plots, it can also be good to include what values you excluded in the title or labels, like "Death Record Counts by Race (Excluding …​.) ". If these plots are just for you, then this gives you a quick reference in some time when you come back and wonder what you plotted. If these plots are for sharing data findings, even better, because these labels help the plot to be interpreted by others who are not in your head.

The MaritalStatus column tracks what relationship stage each person was in at the time of death. Make sure you understand what the letters in this column represent before continuing. Additionally, the values in the Age column range from 1 to 999. Obviously, this 999 is an error code number, not a person who was actually that age. For now, we will keep this value.

The seaborn library is very similar to matplotlib in python. Matplotlib is what is commonly used as default; it gets most things done, and is fairly easy to customize. Seaborn is a higher-level library used for plotting. It is built on matplotlib, and is thus able to provide more options for making appealing visualizations while not over-complicating things.

sns.boxplot is well suited for exploring the relationship between one categorical and one numerical variable. Create a boxplot that shows how the different marital statuses compare to the ages of the people who have died:

It is easy to see where the outliers are in this plot. With the 999 values being so much higher than the other (actual) ages, the rest of the plot gets squished down, so it is not very useful.

Filter out the ages of myDF where they are 999, and save this as cleanDF. With this new cleanDF, make a boxplot to show the reasonable ages and the marital status of the people in the death records.

Take a specific age range (including at least 40 years of ages within the range) from the actual ages of the people who have died, and make a boxplot to show this against the marital statuses.

Make a boxplot that is very similar to that which you just made, except only for the people whose marital status is "M" (married) OR "W" (widowed). Your plot should have two "boxes", with clearly distinguishable boxes for each marital status.

For this boxplot, add proper title, axis labels, and colors. Any additional customizations you want to add are welcome.

People can die at any age but it is more likely for someone older to be widowed in their death record than it is for a younger person to be. The same is likely true for any status besides single, depending on how young or old of people you are looking at.

So, how do you compare the marital statuses across a certain age? You can plot it.

Take your subset of cleanDF (which should only include the people who were married or widowed). Filter subset to take only the data entries where the peoples' ages were exactly 60. Save this as subset2 to avoid overwriting subset.

Find the value counts of subset2 to see how the marital statuses of the 60-year-olds were distributed between the people that were married vs widowed. Call this my_counts, and put it into a barplot.

Filter the data of cleanDF to only include the people who are age 60, across all marital statuses. Call this my_first_counts - or whatever name you find fitting.

Do the same for where the people are age 70, and age 80, respectively:

With each subset, get the value counts AND sort by the index. The counts of people of each marital status are different depending on the age, so sorting by the index allows us to keep these three subsets' marital statuses ordered the same.

You have hopefully learned a bit about using Pandas vs matplotlib vs seaborn for when you are wanting to plot.

Plot the value counts of my_first_counts, my_second_counts, and my_third_counts as their own barplot, using whatever method you like. For these barplots, include at least: