"Six points for six posts?" Facebook increases student's cognitive; affective learning in college mass lectures

I've been running an early data analysis on Facebook comments from my Fall 2011 "Introduction to the Mass Media" course at West Virginia U, and the results are very promising. In short, we find that students participating in a voluntary Facebook group as a supplement to the mass lecture attendance scored nearly six percentage points higher on their comprehensive exams (a significant effect), and enjoyed the class more. Early results of course, but their implications for college instruction are apparent.

477 Facebook posts from our supplemental Facebook group 

to WVU COM105 "Introduction to the Mass Media"

It's no secret that the mass lecture-format of college lecture has some shortcomings. They are designed for efficiency of education - to teach the greatest number of students in a specified time and space requirement. Unfortunately (as colleague and coauthor Meagan Bryand points out) the immense size of a mass lecture can make it difficult for all students to decode message perfectly as they were intended. As well, students in mass lectures often feel detached from their professors and from each other - and they often report feeling intimidated by the mass lectures as a "public space" where they don't want to share their opinions and questions in front of peers. In general, we recognize that the mass lecture can be an isolating environment, but we also recognize that is an efficient and effective way to give students a broad base of knowledge before directing them into smaller, upper-level courses. While a larger discussion of the situation is beyond the scope of this one post, for most universities it is simply impractical to remove the mass lecture format from course offerings. Not only impractical, but undesired.

To help address some of these points, our research team of myself, Bryand and WVU undergraduate Lindsey Carr wondered using Facebook as supplemental space to the mass lecture might be a useful and cost-effective way to enhance the mass lecture experience. Thus, in Fall 2011 students (N = 321, 195 male and 126 female) enrolled in a mass lecture were invited to join a closed Facebook group devoted to their class. Participation in the group was strictly voluntary and students were assured that their profile information would remain private and closed to other members (or at least, they would not be required to friend anyone to use the page). We wondered if involvement in the group might influence learning and attendance in class, and the results are quite interesting. [NOTE: Data analysis still in progress, particularly with affective reactions to class]

DESCRIPTIVES:

Course grade. Three exams were given in COM105 to provide an assessment of student’s basic understanding of course concepts; our cognitive assessment. These exams were 50-item, four-response options closed-ended format, with slightly heterogenous test items (average Kuder-Richardson 20 = .76) falling just below the KR = .80 cut-off suggested by (Lewis & Ortiz, 1988). All three were weighted equally; these grades were added together and transformed into a simple percentage, with students scoring an average grade of M = 75.27%, SD = 12.57.

Attendance. Attendance in COM105 was measured using a random assignment of seven different attendance activities given throughout the semester; this was considered a measure of student engagement. These activities were coded “1” (completed) or “0” (not completed), and students completed an average of M = 4.80 of these assignments, SD = 1.98. Research activity (extra credit). Students were provided with three opportunities to earn six total extra credit points in COM105, and these scores were tracked as an indicator of increased engagement in the course. Students completed an average of M = 3.55 of these six points, SD = 1.61.

Use of Facebook group. In all, n = 142 students (44%) used the Facebook group, posting an average of M = 6.88 posts, SD = 9.50. Students posted an average of Knowing that a significant bias exists such that females are more likely to use social media than males (Bowman,Westerman, & Claus, 2012), we examined the distribution of Facebook members and non-members by gender; no significant difference was found, χ2 (1) = .043, p = .835. For both males and females, distribution to Facebook members and non-members was nearly 50% each (males: 89 members, 106 non-members; females: 59 members, 67 non-members). A total of 477 posts were made to the Facebook students, with 283 of these (~ 60%) posted by students. Cluster analysis by three expert coders (i.e., authors of this report) found nine distinct categories of post, see Figure below.

	Exam Review	Admin	Class-Related External Links	Peer Support	Humor	Affect for Instructor/Class	Unrelated External Links	Instructor Support	Unclassified
# posts	201	119	61	17	16	15	13	8	27
Avg # comments per post	3.60	2.47	1.46	3.41	2.74	.292	.288	.375	1.64
# posts initiated by instructor	64	60	39	0	3	1	5	8	13
Avg # of comments per post	3.66	1.22	.923	0	4.33	0	0	.375	2.92
# posts initiated by students	137	59	22	17	13	14	8	0	14
Avg # of comments per post	3.54	3.71	2.00	3.41	1.15	4.39	3.75	0	.357

Unsurprisingly given visual inspection, a chi-square goodness-of-fit test reveals a significantly different distribution of comments by comment type, χ2 (8) = .655.3, p < .001. Comparing to an expected distribution of 53 comments per category (i.e., a null effect of comment category on observed comments) we see that exam review and administrative questions were by far the most prominently-posted comments, with the other seven categories falling under what might be expected by chance. Comparing these two categories against each other, we see a significant difference in that exam review questions occurred far greater than change, χ2 (1) = 21.01, p < .001). A few patterns in here are interesting, and deserve some further investigation, mainly (and we note the purely descriptive nature of this analysis; in fact we are open to analysis suggestions as we work this data set:

• Students initiated exam review questions at a nearly 2:1 ratio as compared to instructor-initiated exam review posts, yet the number of comments generated did not differ from each other. This suggests students on the Facebook page to be proactive in their studying. 
• Students engaged in posts related to general humor as well as expressing support for each other and the professor of the class. of the 477 posts, only one was found that expressed dissent with the course (a student upset about an exam being perceived as too difficult). 
• What else are you seeing in here? 

RESULTS: 

Course engagement. Students who were members of the class Facebook group submitted more attendance assignments (M = 5.23, SD = 1.79) than students who were not in the group (M = 4.43, SD = 2.07), t(319) = -3.69, p < .001. No significant difference was found between the groups for completing extra credit assignments, t(319) = 1.70, p = .571; both groups completed an average of M = 3.55 extra points, SD = 1.61. For students in the Facebook group, they posted an average of M = 6.88 post, SD = 9.05.

Cognitive assessment. Students who were members of the class Facebook group scored significantly higher in the class (M = 78.55%, SD = 8.54) than students who were not in the group (M = 72.64%, SD = 13.60), t(319) = -4.71, p < 001. For students in the Facebook group, there was no significant association between the number of times an individual posted in the Facebook group and their attendance (r = .124, p = .144) or their course grade (r = .158, p = .061). This data suggest that membership in the group was the primary influence on the witnessed effects rather than increased active involvement in the group. Some support for this finding can be found in looking at the distribution analyses of Facebook posts per user, which showed a heavy positive and leptokurdic skew (skewness = 4.09, kurtosis = 26.23), suggesting a few super-users providing the majority of Facebook content with many users lurking from this.

So, there you have it. Early, quick and dirty analysis -but something to get the conversation going! Some very early and preliminary data to suggest that encouraging students to use Facebook to supplement the mass lecture can be an effective way to get them engaged in class and, if you can get them posting, to enhance their learning of course material. We of course acknowledge limitations of the current study, including a rather base approach to cognitive learning qua exam grades and a potential self-selection bias such that engaged students are more likely to join a class-related Facebook than non-engaged students - both should be investigated further in replication. Yet, we feel strongly that our data suggest that using Facebook as a persistent and cost-effective supplemental space for which to continue discussions in the university mass lecture can significantly enhance student learning. And we look forward to your comments!