TDM 10100: Project 2 - Indexing in R

This dataset is an ideal starting point for learning R because:

What kinds of movies usually get this Certified Fresh rating? Are PG movies more likely to get Fresh reviews than R-rated ones? In this project, you’ll explore questions like these, while also building fluency with R tools like sort(), table(), grep(), sum(), and more.

As you work through each question, you’ll sharpen your skills in indexing, pattern matching, and plotting - all while discovering trends behind the movies we love (and hate).

Import the Rotten Tomatoes Movies and TV Shows dataset from the file /anvil/projects/tdm/data/movies_and_tv/rotten_tomatoes_movies.csv into a data frame called myDF. Check out what data it contains by using the head(), tail(), and dim() functions.

What are some of the columns? What are the dimensions of this dataset (without the two columns)?

Now we will look at the rating column. It would be interesting to see how evenly these different rating levels are distributed across the 16638 rows of movie data. Making a table of the rating column will show us this!

Another way to become familiar with the data is by visualizing it. We will start with a basic R visualization function, plot(). Let’s use plot() to visualize the table we just created. Do the resulting lines look reasonable?

Take another look at the table() of the rating column. How many unique categories are there?

It is easy to see that there are 8 different film rating types. But what about when there are many more than 8 entries in a table? Let us try to see the same for another variable, directors.

It is not always easy to view all levels at a glance using the table function. In fact, printing all of them might even crash the kernel. Fortunately, there is a simpler and safer alternative: checking the length of the level vectors instead of listing them.

This is good, but we may also want to see which item from the table is the most/least populated. Wrapping our current table() statement in a sort() function, and then looking at the head() and the tail() will show us these values.

The movies (and tv shows!) in this dataset each are listed with an entry in the genre column. How would we go about finding the total number of entries in this dataset for a specific genre?

To start, let’s find all of the entries that are specifically 'Comedy'.

Using the '==' operator shows the results for when it is true that 'a' == 'b'. But trying myDF$genre == "Comedy", and looking at the head of this shows confusing results full of FALSE and TRUE values. Each of these are related to a binary value:

Please go through and get the sum() of where the genre being 'Comedy' is TRUE - this should give us what we are looking for.

But what about when the genre is not just listed as a singular category? The sum we just found is only a part of the movies in this dataset that have this genre. It is important that we know how to find the entire count rather than just where it is an exclusive listing. We need some specific functions for this aim in R. Please go through and read about grep and grepl here. (Feel free to read other sources as well!)

Run grepl("Comedy", myDF$genre). This still outputs in the TRUE and FALSE results for each row of the dataset (as before), but these results are different. Why? If we look back at the head() of the dataset, we can see that the first entry’s genre is 'Action & Adventure, Comedy, Drama, Science Fiction & Fantasy', so this was not counted as TRUE before when we were not using grepl.

To see the results of which rows are TRUE only, run grep("Comedy", myDF$genre).

To print the total count of these results, use

While grepl returns logical values indicating whether a pattern is found (TRUE or FALSE), grep returns the positions (indices) of the matches.

We could find the number of rows of specifically not exclusively 'Comedy' listings, OR we could try a really cool grepl statement.
Try showing a few rows of sort(table(myDF$genre[grepl("Comedy", myDF$genre) & (myDF$genre != "Comedy")])). After looking at this table, remove the sort() and table() and get the length() for the count of these rows.

Let’s go back to our table of the rating column.
This table shows us how many times each of the rating types corresponds to a row of the dataset.

This is not the most lovely table. Why? Look at the contents. There is 'G', 'NC17', 'NR', 'PG', 'PG-13', 'PG-13)', 'R', and 'R)'. The value counts of 'PG-13)' and 'R)' are not high because these are clearly not meant to be their own rating type. But this is just a case of where humans have been humans and have made a typo. This happens a lot in real-world data. Later in the semester, we will go on to show how to clean datasets by merging values like this with their true values, but in this project, please run myDF$rating[myDF$rating == "R)"] <- "R" and myDF$rating[myDF$rating == "PG-13)"] <- "PG-13" to clean up these values. Essentially, we are correcting typos by replacing them with the accurate versions.

We will also take a look at the table of the tomatometer_status column. There are just three categories here! What could we possibly find from this?

Let’s make a new table. This table should contain BOTH the rating column and the tomatometer_status column. We will save this table as the variable 'table_data'.

Read about creating tables in R here. Run a page search for the Two-Way Tables section.

Taking our 'table_data', we will now create a plot. Here, we will use a plotting method that is not as common as a typical line or scatter plot.

We are going to be using the mosaicplot() function to create our visualization. Read about mosaic plots in R here.

We will continue to work with these two columns (rating and tomatometer_status). In Question 4, we showed the table of these columns put together. Now we will use the paste() function to merge them. This allows us to see the numerical values for the combined columns. Please display the sorted table of these two columns once they have been pasted together.

From the table, choose one combination you would like to continue learning about. The following examples will use the 'PG' and 'Certified Fresh' rating combination.
Let’s create a subset of our data. This subset will contain only the data where the rating column shows 'PG' and the tomatometer_status is Certified Fresh.

Now that we know a bit about subsetting, let’s do one last thing. Please subset the original myDF data again twice to find where the movie_title entry is listed as each of your two favorite movies (or shows).