I taught two sections of Critical Reasoning at Yonsei University UIC for the past three semesters. Due to the pandemic, the first of the three was fully online. The second was a blended format, with lectures delivered online and discussion held in-person. The third was fully in-person. Each discussion meetings had as its topics an article related to the week’s content from a popular website. Throughout my time teaching this course, I’ve been interested in two questions:

Now, with the help of data analysis, I’m able to answer these questions. Below, you’ll find the visualizations I generated and my answers, followed by links to the scripts I used to process the data and create the visualizations.

To summarize, I found that:

  • Online discussion had the least amount of contributions.
  • The longest and most difficult articles had the least amount of discussion.
  • The straightforward and provocative articles had the most discussion.

Read on for more details about how I arrived at these conclusions.

Which learning format had the most participation?

The following bar chart shows the average number of discussion contributions per class for each learning format:

Figure 1:

According to the chart above:

  • Online classes had the least amount of class contributions on-average, with 8.5 contributions per class.
  • Blended and In-person classes had exactly the same amount, with 9.06 contributions per class.
  • Online class participation had the highest standard deviation (3.33), meaning that its weekly participation numbers swung the most.
  • Blended classes had a slightly lower standard deviation (2.38) than In-person classes (2.67).

The first result I found surprising because my impression had always been that Online discussions generated the most discussion, but this clearly was incorrect on my part. Why did I have this impression? I think a major factor was that the students would often prepare written remarks before class and then read them out for the discussion. This had two main effects:

  • The quality of discussion contributions was quite high, since students prepared ahead of time.
  • Students spoke for longer periods of time, so there were fewer contributions on average

I was probably so impressed by the written remarks that I confused contribution quality with quantity. Thankfully, I have the data so now set things right!

I was so surprised by the second conclusion that I assumed that I must have made a mistake in my Python script to parse the CSV file and create the database. To test, I added fake values to the see if they would affect the final tally, and indeed, they did. Another way I tested was to calculate the standard deviation of the participation set, which confirmed different values for online and in-person learning. Somehow, I managed to have exactly the same number of contributions for the semester I taught blended and in-person, though the standard deviation in each case was slightly different.

Although it is surprising that these two values are exactly the same, it’s not surprising that they were very close, since the discussions for Blended learning classes were conducted in-person. Nevertheless, the data does suggest that whether lectures were conducted online or in-person did not affect the amount of in-person discussion. This result is interesting because it suggests that at least one learning outcome variable–amount of class discussion–is not affected by the method of lecture delivery. If this result scales with more data, it could be an argument in favor of Blended classes.

The standard deviation results are also interesting. The Blended and In-person values were quite similar to one another, and both were much lower than the Online discussion. So Online discussion not only had fewer contributions on average, but also had the largest range of weekly discussion values. This becomes clear when we look ahead to Figure 2. Although Online discussion of the article by Charlie Warzel had the highest average number of contributions, for 5 out of 8 discussions, the Online format had the lowest average discussion.

From this, we can conclude that on average, Online discussions generate fewer contributions than in-person discussions in fully In-person or Blended learning formats.

Which articles generated the most discussion?

The topic of discussions in my Critical Reasoning classes were a series of popular articles from places like the Chronicle of Higher Education, the New York Times, and Aeon. The complete list is as follows:

The chart below shows a grouped bar chart representing the average number of contributions per class with each learning format color-coded. The x-axis refers to the article by the author’s last name. The numerical value represents the average number of contributions across all learning formats.

Figure 2:

  • The articles by Warzel and Labaree generated the most discussion, with 11.17 and 12.67 contributions per class on average.
  • The articles by Sober and Williamson generated the least amount of discussion, with only 6.83 and 5.67 contributions per class on average.
  • The total average number of contributions per article was 8.87.

This finding seemed to agree with my general recollection. Students really seemed to enjoy the articles by Warzel and Labaree, and I think part of the reason is that they’re a bit provocative. Warzel argues against critical thinking in favor of media literacy techniques, for example, and Labaree pillories the “Five Paragraph Method” for writing essays. I would highly recommend both articles in critical reasoning classes.

The articles by Sober and Williamson don’t usually generate much discussion owing I believe to their difficulty. Sober’s article is about the theoretical foundations for Ockham’s Razor, which is fabulously interesting, but demanding. It’s also a longer article, so in addition to its difficulty it’s a bit tougher for students to skim through in class if they haven’t read it.

Williamson’s article is shorter, but even more demanding in so far as it discusses logic itself at an extremely high level. Students really struggle with it, but I’ve yet to find a better replacement for an article written by a philosopher for the public about logic. For both articles, I usually spend a lot of time just explaining the main ideas in the articles before then anxiously waiting for students to speak up, so it’s not surprising to me that the data shows they generate the least discussion.

Resources