» Other methods
So far I've only examined the assessment tools already used by
the Rutgers Writing Program; however, there are other tools writing
programs should consider when considering the question of web site
assessment. We plan on implementing some of these assessments at
Rutgers in the future.
Course Evaluations
One obvious assessment tool for a program web site is the course
evaluation form. At Rutgers, course evaluations are processed by
the Teaching Excellence Center. Though the form is standardized
and scannable, several questions on the form can be customized to
a particular program or department. These include questions with
a numerical response on the front of our forms and questions with
a written response on the back.
In our program, we've discussed the possibility of adding questions
about the web site onto the evaluation forms for all writing program
classes, but such a change would require printing totally new forms.
Given current budget conditions at Rutgers, new evaluation forms
cannot be ordered until the current supply is used. Thus, while
this is an assessment we look forward to using, we have not been
able to implement it yet.
Such an assessment would combine the advantages of print surveys
with the added advantages of a reliable channel for distribution
and processing. At most institutions, mechanisms are already in
place to handle such course evaluations, and students are already
used to completing such forms. Using course evaluation tools as
an assessment would reach, presumably, the entire student population;
however, we must also keep in mind that students don't always take
the time to complete these forms carefully. At Rutgers at least,
course evaluations are usually handled at the end of the last class,
and students frequently rush through the form just to get out of
the room that much earlier.
Still, this method promises to be a valuable assessment tool, particularly
when combined with the other tools we currently use.
Usability Testing
Usability testing is another option. Not only is it a standard method
of evaluating digital compositions ranging from web sites to software,
but it provides a different kind of assessment, one more directly
focused on a user's ability to locate needed information or to complete
anticipated tasks.
We had initially planned on performing such a test, as designed by
the consultant for our web site redesign, Todd Reichart (see sidebar).
But at the time we lacked the resources, ranging from funds to space,
to perform such a study properly.
For while usability testing can function as part of an overall
process of participatory design and evaluation, proper usability
testing requires particular an unique resources. For example, in
Designing Effective Web Sites (2002), Johndan Johnson-Eilola
notes some of the needed resources, including an area where users
can work and be observed non-intrusively as well as audiotaping
or videotaping equipment (p.54). The lack of these resources may
hamper a program's ability to perform such a study
Assessment Heuristic
Returning to Yancey's (2004) suggestions, we might consider too
the heuristic that she suggests for assessing digital media. She
specifically prompts us to ask
- What arrangements are possible? At this time,
only one--the arrangement we created in the navigational structures
of the site.
- Who arranges? Both the program and the users
of its site. While we have created the arrangement of links, users
can, as Yancey does with one student portfolio, "make them
work as [they] choose," which is confirmed by the online
survey data on students using teacher resources as well as he
print survey data on teachers using the Gradatorium, found in
the student resources (2004, p.98).
- What is the intent? To create a pedagogically
useful, resource-rich web site that's valuable to all populations
that participate in the program.
- What is the fit between the intent and the effect?
Here, the heuristic threatens to break down. In Yancey's analysis,
the heuristic is applied by the audience of the digital composition:
she is the one using it to evaluate email and digital portfolios.
As a program, we can't determine the fit between intent and effect--not,
at least, with the heuristic alone. Again, assessments must be
combined, interweaved, to begin formulating this answer.
conclusions »
|
