I would like to report that the inclusion of audio in the online version of "Writing, Style, and Technology" in Fall 2005 was a grand and clear success, and that my use of audio files was overwhelmingly enlightening for all of my students. I would like to make this claim, but I can't. Instead, I would characterize my results as mixed, but promising, for two reasons. First, I believe it's possible to see some optimism in the results of a simple survey I conducted with my Fall 2005 students. Second, based on what I will admit are anecdotal and intangible gut feelings, I believe that students in the Winter 2006 and Spring/Summer 2006 versions of the course are making better use of the audio files.
At the outset, let me point out that I am well aware of the limitations of my survey. It was a small sample; only 11 out of the 19 students who finished the course participated. I conducted the survey in March and April of 2006; this was about two and a half months after the course ended, and I am sure that my students' (and my own) memories of the events of the course were no longer as sharp. I didn't conduct any sort of survey at the beginning of the term, meaning there was no point of comparison or way to test my "treatment" of adding audio to the online course. I am sure there are many other problems with this work that will be identified by others. But even with these and other qualifications in mind, I think the results were informative. I offer them here as a starting point for further research.
I conducted my survey (which was reviewed exempted by the Eastern Michigan University Human Subject Review Board) with Fall 2005 students via email. See the survey itself (pdf). Seven of the eleven students were female and all but one (who was a graduate student) were juniors and seniors. Roughly speaking, these basic demographics reflect the overall make up of the course.
My first set of questions explored the previous experience students had with both online classes and computer technology. Five of the students had previously taken online courses; two of the students indicated that they had taken conventional classes where there was a significant online participation component to the course.
Students who had had previous online experiences had mixed feelings about those classes. Things that students valued and liked about previous online courses included clear, straight-forward, and easy to understand instructions; threaded discussions; freedom to work and attend class on an individual schedule; and regularly available instructors. Things students did not like about previous online courses included: threaded discussions; inattentive or unavailable instructors; down web sites or other computer connection problems; and a general lack of contact with other students in the class.
In response to the question regarding their experiences and knowledge of computer technology, seven out of eleven students rated themselves as computer comfortable --or, in terms of the survey instrument, they gave themselves a "B." Two students rated themselves as computer savvy ("A"), and one selected computer competent ("C"). Based on my experiences in working with these students in the course, these results strike me as reasonably valid. However, I think that this is an especially problematic question, in part because students seemed to give themselves "grades" that corresponded to the grades they received in the course itself. And on the whole, I think the problem of students' perceived and actual computer/technical literacy is much more rich and complex than these simple questions can explore. But that is a different project.
Of the eleven participants in this survey, four indicated that they listed to all or more than half of the audio files for the class. Four students said they listened to some of the recordings (which I defined as more than five of them, but less than half of the available files), and three students said they didn't listen to any of the files. With the exception of one student, all who indicated listening to some or none of the audio files had had previous experiences with online classes.
The students who listened to all or most of the audio commentaries consistently found them helpful, which they noted by making comparisons to onsite, face-to-face class settings:
"Listening to the audio recordings felt like you were actually in the classroom."
"I could sit back and listen as I would in a regular classroom setting."
"I was very impressed ... because I felt like I was still in a classroom, it was just in my room."
These students also said that the audio recordings helped them study and understand the reading material:
"It was nice to play the audio because it gave me a chance to look away from the monitor for periods of time and jot down notes or simply reference the reading."
"I think it gave a me a better feel for the topics..."
Students who listened to only some or none of the audio recordings gave a variety of reasons for not listening to them. Some said that they didn't listen to recordings because of their own learning styles and preferences:
"I have a better time understanding directions if they are written out for me. I can print them and go step by step. I learn better that way."
"The main reason I didn't listen to all of the audio files was not because I thought they would be useless, but because of my study habits. I'm pretty much a visual learner so I don't retain much from 'lectures' in general."
Some students didn't listen to the audio files because, for good or bad reasons, they lacked the time:
"I enjoyed the files when I listened to them. I found it really helpful to listen. But sometimes, unfortunately, I just wouldn't have the time to get through it all (I'm a procrastinator)."
"Mostly, I just didn't have time. Between the readings and papers, and my other classes, I didn't have time to do everything. Something had to be cut out."
Despite these different listening habits, almost all of the students who responded to my survey thought the audio class supplements were a good idea. Nine out of the eleven students thought that the recordings helped in making a connection with me as the instructor. Consider these two quotes, the first from a student who listened to some of the recordings, and the second from a student who listened to none of them:
"I definitely think that it helped me feel as though I knew my professor better. It's hard enough not meeting someone face to face a few times a week but then to also not hear a voice makes it much less personable."
"I could see where it would make students feel more connected, to be able to put a voice (a personal thing) with the words that are on the page. That sometimes helps people become more familiar with a personality."
Further, ten out of the eleven students responded positively to the last question on the survey about suggestions for including audio files in future online classes. Their suggestions included keeping the audio files brief, keeping them optional/supplemental material, and keeping them interesting.
So, as I said at the beginning of this section, the results were mixed.
On the one hand, I was disappointed that the majority of survey respondents listened to some or none of the audio files, and I was frustrated by some of the reasons these students gave. In my experience, "I don't have time" is code for "I didn't want to do it" or "I didn't see the point," neither of which strike me as compelling explanations in and of themselves for not engaging in a learning activity. And furthermore, I have no data that indicates that students who listened to only some or none of the audio files did not grasp the material as well as those who did listen to them. All of the students who responded to my survey--listeners and non-listeners alike--ended up with B's or A's in the class.
On the other and more positive hand, I think these results are promising for at least three reasons. First, the four students in this survey who listened to all or most of the recordings were very enthusiastic about them. Second, a clear majority of the students (including those who listened to none or only some of the recordings) more or less confirmed my reasons why I included audio files in the first place: the recordings helped build a connection between the students and the instructor.
Third, a clear majority of students also suggested that the inclusion of some audio files to help explain readings and exercises simply made sense. So while I am disappointed that more students in the survey did not report either listening to all or most of the audio files I recorded, I take comfort in the fact that I did not have to spend a tremendous amount of time and resources to create teaching materials that were used only by a few students and that were seen as not helpful. And, in most cases, I can use the same audio files for these lessons each time I teach the course.
I also believe that the students in my Winter 2006 (from January until April) and Spring/Summer 2006 (from May until July) versions of "Writing, Style, and Technology" have been more regularly listening to the audio files. As I said earlier, this is based entirely on my, perhaps, faulty perceptions. Still, I have had a fair number of students during the Spring/Summer 2006 term (which is when I am writing this text) who have made references to my audio files in their comments on the threaded discussions. I wouldn't want to draw any firm conclusions about this without surveying these students. However, I think these perceptions are based on better instruction from me on how to use the audio file and also the introduction of podcasting as a separate audio enhancement to the course. I discuss both of these issues in more detail in the "What I've learned, and where I'm going" and "Podcasting on the cheap and beyond" sections of this site.