Digital Contexts: Studies of Online Research and Citation

 

Introduction

section one
Spaces for Digital Scholarship: Creating, Using, and Improving

section two
Teaching, Training, Investigating: Understanding Student Research Behaviors

section three
Beyond the Academy: Understanding Digital Research Behaviors

SECTION ONE
Spaces for Digital Scholarship: Creating, Using, and Improving

The four articles in this section consider (from several different perspectives) the digital spaces/locations that are shaping scholarly practices, as well as the patterns of use that determine the value of digital scholarship. For example, Rudy McDaniel discusses the creation of an online undergraduate research journal, both in terms of its practical development and its intended purposes as a venue for digital scholarship. Both the second and third articles in this section discuss the use of digital journals for scholarly publication and research. Peta Mitchell provides an in-depth analysis of one particular journal – the online journal M/C – to discuss (and critique) the valuing of digital scholarship in the humanities. Doug Eyman and Colleen Reilly use a range of digital journals to make the argument that we must continue to develop new ways of presenting, disseminating, and assessing digital scholarship for the pursposes of tenure and promotion. The work of Anne Beaulieu is also connected to the study of venues for scholarly research, as she examines the use of a scholarly listservs by digital researchers in the field of women’s studies. Beaulieu’s article illustrates the innovative ways that humanities scholars may make use of internet technologies. Together, these four articles examine range of different venues that scholars can use to disseminate and discuss their research, as well as illustrating different scholarly methods for examining research practices in digital spaces.

CHAPTER ONE
TITLE: Building an Online Undergraduate Research Journal: A Case Study
AUTHOR: Rudy McDaniel, Assistant Professor, University of Central Florida

ABSTRACT

Chapter one is a case study that details the development and publication of the University of Central Florida’s cross-disciplinary Undergraduate Research Journal, an online and faculty-reviewed collection of student research manuscripts. The article presents the experience of developing, designing, and publishing the journal through more than a year’s worth of editorial meetings in which various issues concerning online scholarship and citation were discussed, poked, prodded, and finally assembled into a sequence of metrics useful for constructing and maintaining an online journal. The study discusses the relative successes and failures of this particular project, and specifically outlines the various technologies essential for an online research community. These issues include a discussion of server-side programming languages and database systems, the selection of an adequate web server with backup support, and the use of Web forms for the submission of student manuscripts and the dissemination of editorial comments.

From a theoretical perspective, the case study considers the rhetorical implications of this type of online research journal. For example, how is ethos influenced by the journey of research from a paper-bound to a digital environment? In this regard, both authorial and editorial ethos are manifested through a new and slightly unusual channel that features graphical complexity and animation, various agents of advertising mayhem, and numerous other technological distractions and unforeseen juxtapositions of media. The exploratory portion of the essay considers the ways in which rhetoric can be used not only to evaluate and assess, but also to model the construction of digital communities and assist with the spread of cross-disciplinary knowledge.

________________________________________

CHAPTER TWO
TITLE: Endorsing the Digital: Online Scholarship, Public Intellectualism, and Discourses of Academic Legitimacy
AUTHOR: Peta Mitchell, School of English, Media Studies, & Art History, The University of Queensland Brisbane.

ABSTRACT:
Chapter two investigates the discourses of academic legitimacy that surround the production, consumption, and accreditation of online scholarship. Using the web-based media and cultural studies journal M/C Journal (http://journal.media-culture.org.au) as a case study, it examines how open-access scholarly journals in the humanities often position themselves as occupying a space between the academic and the popular and as having a functional advantage over print-based media in promoting a spirit of public intellectualism. The current research agenda of both government and the academy prioritizes academic research that is efficient, self-promoting, and relevant to the public. Yet, although the cost-effectiveness and public-intellectual focus of open-access scholarship in the humanities speak to these research priorities, online journals such as M/C Journal have occupied, and continue to occupy, an unstable position in relation to the perceived academic legitimacy of their content.

The paper charts the debate surrounding online, open-access scholarship from the early 1990s, when resistance to academic work online focussed on concerns over ensuring the integrity of the peer-review process in an environment that privileged speed. In recent years, this original debate has shifted focus, towards a privileging of open-access scholarship. Indeed, a quantitative study, completed by Thomson ISI, indicates open-access journals (particularly those in the sciences) are on a par with print-based journals in relation to how academics use and cite them as authoritative sources. However, despite various moves by both publishers and academic institutions towards new media use, studies conducted in the past five years also indicate that publishing online in fact adversely affects the chances of achieving promotion and tenure for early career academics. These studies reveal the core of the problem facing open-access journals: while it is now deemed safe to use online scholarship, it is still not safe to produce it.

These developments illustrate that despite positive developments in the debate over open-access scholarship, open-access scholarly journals in the humanities must still work to carve out a new space for academic discourse that maintains the principles of academic rigor and that can immediately be recognized as “legitimate” scholarly work, but that also does not neutralize the online medium’s potential for public intellectualism or public cultural work.

________________________________________

CHAPTER THREE
TITLE: Beyond Citation: Evaluating the Significance and Impact of Electronic Publications.
AUTHORS: Doug Eyman, George Mason University and Colleen Reilly, University of North Carolina at Wilmington.

ABSTRACT:
The process of evaluating scholarly journals in print is generally well-established. Peer-reviewed, print journals can point to their impact factors, circulation numbers, and the prestige of their editorial board members and authors, which is often based on citation data of the publications of these individuals, to demonstrate their importance and influence in a particular field. While electronic peer-reviewed journals certainly have significant board members and authors, the traditional measures in the form of citation and circulation data are relatively ineffectual in determining the significance and impact of electronic publications (Harter & Kim, 1997; Herring, 2002; Herring, 1999; Zhang, 1998; Zhao, 2005). To address the inadequacy of traditional methods of evaluation, our chapter provides evaluative strategies for electronic publications that can be used by scholars and students to assess the value of electronic, peer-reviewed journals as possible publication venues and research tools. As a case study, we will provide example evaluations of three well-established peer-reviewed electronic publications in rhetoric and writing (Kairos, Across the Disciplines, and Journal of Computer-Mediated Communication) using the strategies we present to determine the degree to which these strategies produce usable results.


Arguably, the publication of independent, peer-reviewed, electronic journals (those not replicating the content of print counterparts online) related to rhetoric and composition, technology enhanced education, and the teaching of writing is at critical point. An examination of the list of twenty such journals linked to Writing Across the Disciplines’ website (http://wac.colostate.edu/journals/) reveals that nine of these online-only journals are no longer active (publishing no new content since Fall 2005 or before) and two of the nine, Academic.Writing and Language and Learning Across the Disciplines merged in 2004 to become Writing Across the Disciplines. Significantly, the editors of IMEJ of Computer-Enhanced Learning (http://imej.wfu.edu/) have explained that they decided to discontinue publication largely due to the lack of high quality submissions and surmised that this deficit might be attributable to the uncertain prestige of electronic publications.

As has been noted for some time, scholars and the university administrators who assess their scholarship continue to question the legitimacy of electronic publications. In order to address their concerns, we need to provide clear methods and measures that scholars can use to evaluate peer-reviewed, electronic journals as potential publication venues and subsequently to support the validity of their publications to administrators and retention, promotion, and tenure committees. Furthermore, educators also require evaluation strategies that they can impart to their students, who, more than seasoned scholars, often approach the Web as the starting point for all research and have difficulty in distinguishing between scholarship and other types of web-based resources.


As we have noted elsewhere (Reilly & Eyman, forthcoming), many quantitative and qualitative methods for evaluating electronic scholarship exist, although they have not been codified and tested. To examine the available methods and demonstrate their application, our chapter will provide an evaluative case study of the three aforementioned peer-reviewed, electronic journals. The quantitative methods we will examine include web impact factors and retrieval analysis. In terms of qualitative methods, we will use hyperlink mapping, analysis of citation practices, and an evaluation of whole journals based on a heuristic continuum of electronic publications that categorizes electronic texts based on the degree to which they differ in format, structure, and content from print publications. Through the application of these methods to the journals in question, we will provide a model of evaluation to scholars and students alike and simultaneously demonstrate the legitimacy of electronic scholarship.


Works cited

Harter, Stephen P., & Kim, Hak Joon. (1997). Electronic journals and scholarly communication: A citation and reference study. The Journal of Electronic Publishing, 3(2). Retrieved Oct. 28, 2006, from http://www.press.umich.edu/jep/archive/harter.html.
Herring, Susan Davis. (2002). Use of electronic resources in scholarly electronic journals: A citation analysis. College and Research Libraries, 63(4), 334-340.
Herring, Susan Davis. (1999). The value of interdisciplinarity: A study based on the design of Internet search engines. Journal of the American Society for Information Science, 50(4), 358-65.
Reilly, Colleen A., & Eyman, Douglas. (forthcoming). Multifaceted methods for multimodal texts: Alternate approaches to citation analysis for electronic sources. In Dànielle Nicole DeVoss and Heidi A. McKee (Eds.). Digital Writing Research: Technologies, Methodologies, and Ethical Issues. Creskill, NJ: Hampton Press.
Zhang, Yin. (1998). The impact of Internet-based electronic resources on formal scholarly communication in the area of library and information science: A citation analysis. Journal of Information Science, 24(4), 241-54.
Zhao, Dangzhi. (2005). Challenges of scholarly publications on the Web to the evaluation of science—A comparison of author visibility on the Web and in print journals. Information Processing and Management, 41, 1403-1418.


________________________________________


CHAPTER FOUR
TITLE: What’s Needed for a Next Generation of E-science? An Exploration of Location and Consolidation of Knowledge around a Women’s Studies Discussion List
AUTHOR: Anne Beaulieu, The Virtual Knowledge Studio for the Humanities and Social Sciences (VKS) Royal Netherlands Academy of Arts and Sciences

ABSTRACT:
Computation and networked data-sharing are the dominant activities pursued in ‘e-science’ initiatives (Wouters and Beaulieu, in press). As a result, fields where computation is not a core activity often do not consider that e-science initiatives have much to offer. In turn, the needs of researchers in humanities and social sciences are not usually taken into account when imagining new e-science applications, even through scholars in many other fields are currently making extensive use of internet technologies and appropriating them in innovative ways.

This article seeks to amend this lack of correspondence through an examination of scholarship in the field of women’s studies, which highlights ways the aspirations and scope of e-science might be reformulated. The article reports on an ongoing research project on the use of digital and electronic technologies in the field of women’s studies, in which the main approach is virtual ethnography (Hine 2000). Theoretically, the framework of epistemic cultures (Knorr-Cetina 1999) is used, and has been adapted from science studies for analysis of a field in the humanities.

The main empirical focus in this paper is a US-based discussion list for women’s studies with a large international user-base. The material presented was gathered through my reading of the list for 16 months (June 2004-present), in-depth interviews, and a detailed content analysis for specific periods. My analysis reveals the extent and style of the list’s embedding in other digital and networked scholarly practices in women’s studies (websites, search engines, etc). The paper also introduces labels for three recurring patterns of interactions on this list (‘pointers’, ‘stacks’ and ‘waves’). Each is associated with important functions of this form of scholarly communication, and point to the list as a conversation, in relation to the archive of the list, helping bring the contents of the archive to the attention of the list members; a kind of short-term memory to supplement the long-term memory provided by the archives; a place of consolidation of multiple and intersecting resources, usually according to dynamics of canonization and institutionalization; a space in which to enact the location of knowledge, not only in a geographical sense, but also in a personal, political, and institutional one.

The way technologies are used and valued by researchers is therefore shown to be linked to the practical needs and intellectual tenets of this field. These functions (and especially the latter, which is demonstrably not very well supported by the list), highlight the importance of location and the ‘performance of place’ in feminist scholarship. If, for women’s studies, knowledge is best treated as locatable and situated, this notion must be reconciled with the usual characterization of the email and the web as global and virtual, and of discussion lists as a level conversational playing field.

Performance, display, accountability, and embedding in particular contexts are all activities that could be enhanced and supported by novel applications for discussion lists. The paper suggests a number of ways in which current practices could be enhanced by new applications, and how they might inspire a ‘next generation’ of non-computational e-science.

references

Hine, Christine. 2000. Virtual ethnography. London: Sage.
Knorr-Cetina, Karin. 1999. Epistemic Cultures: How the Sciences Make Knowledge. Cambridge, Massachusetts: Harvard University Press.
Wouters, Paul, and Anne Beaulieu. in press. Imagining e-science beyond computation. In New Infrastructures for Knowledge Production: Understanding E-Science, edited by C. Hine: Idea Group.

______