Science is, both as an institution and as an ongoing method of inquiry, a continual work in progress. There was no particular moment when the practice of natural philosophy evolved into the methods of the modern sciences, although the roots of empirical method run as far back as Aristotle and the ancient Greeks. But empiricism alone does not reveal truths about nature. The ideal of science is that of an ongoing process, a dialogical refinement over time of theory by confirming or contradicting evidence as applied to hypotheses. The infrastructure for that sort of ideal dialogue has never been fully perfected. Many have traced modern science back to the emergence of the various scientific societies of Enlightenment Europe, and indeed we can picture the salons of the time, abuzz and with nascent natural philosophers engaging in the rudiments of idealized scientific method. With a small enough community of researchers, no great infrastructure is necessary. But as empirical research spread, and science became de-localized, international, and pursued a broader scope of subject matter, the possibility for scientific method to develop close to its ideal diminished due to practical impediments. Now, with the emergence of new media for communication, and new methods of publication, the possibility of finally meeting the requirements of a ideal method are real, but we now face the significant challenge of restructuring other parts of the institutions of the sciences, both academic and commercial-technological, in order to fully conceive the promise.

The Ideal of Science

Let’s look first at what I mean by ideal method in science, and examine why the current and historical institutions of science have prevented its realization. The ultimate goal of the methods of science is the search for nature’s truths. It relies on the interplay of observation, experiment, and hypotheses. Over time, through the successful application of its methods, we slowly come to have confidence in the models scientists develop. But as all students of the philosophy of science, and as all scientists know, the “conclusions” of science are always provisional and contingent. The best theory, supported by the longest string of confirming experiments and observations, must still always be subject to revision or abandonment in the face of new evidence. As scientific institutions began to form, parlors would not long suffice for disseminating the results of experiments, sharing hypotheses, and devising theories. A growing and geographically diffuse group of peers needed a means of review, of ongoing dissemination and discussion of research programs. Scientific publications began to emerge, beginning in 1665 with the French Journal Des Sçavans and the English Philosophical Transactions of the Royal Society. Publications afforded scientific researchers a means of disseminating results and inviting further testing, and thus either confirmation or falsification of hypotheses. By expanding the audience to all potential readers of a journal, rather than a researcher’s immediate community, science began to become an international enterprise. But the process of research/peer review/publication/confirmation-falsification remained far from ideal in practice given the nature of hard-copy publication and dissemination as it has existed since the mid 17th century. In an idealized method, scientific texts would necessarily be hyper-textual. The nature of hard-copy publication, however, makes this only virtually possible. One could, for instance, extract the current state of a scientific hypothesis by finding a seminal text summarizing that hypothesis, then following up with every published subsequent text criticizing, providing counter-examples, or outright falsifying the hypothesis with contrary evidence. While this process is aided by electronic research tools and hyperlinked references, each text, once published, remains intact. None of this would pose much of a problem for the ultimate pursuit of truths, the underlying goal of science, but for the fact that it is the goal of publication, rather than the underlying process of scientific inquiry, which has become most important in the current academic-scientific enterprise. Thus, the text itself has been emphasized, rather than the process of which it is meant to be a part, and the conclusions that emerge from numerous related texts focused on a particular area of inquiry. Because of the emergence of publication as the primary means of academic and scholarly measurement, published texts have come to be iconized, and the potential for an idealized method has been subsumed to the demands of promotion and tenure. These demands stress the fact of publication, and the status of the journals in which those publications occur, rather than the long-term implications and effect of a researcher’s inquiries on a particular scientific discipline. In idealized method, the provisional conclusions of a particular research program, constantly shifting with new investigation, experiments, and observations, ought to be paramount. A particular text ought to mean little, except inasmuch as it serves as a temporary piece, marking provisional conclusions and a new point of departure for further study.

Less Than Ideal

The nature of academic science, which has evolved on top of the pre-existing infrastructure of scientific publishing, has helped to solidify an outdated preference for hard-copy, printed journal articles. These are the currency of academic science after all. They make or break promotion and tenure decisions. Yet the relative value of journal articles changes over time. While their value to science may slowly climb over time, given a steady accumulation of confirming evidence, their value to other scientists and the evolving institutions of science may also suddenly plummet to near zero in the face of a single instance of strong counter-evidence. Because of the value of journal articles as professional currency, moreover, and because of the relative dearth of reviewers in comparison to articles needing review, the process now is admittedly stressed. Scientists whose careers depend upon publication, and journals wishing to scoop each other, lend to a climate where articles are regularly accepted, published, and retracted. Because of the time lag involved in print media, this is a wasteful and even dangerous process that also lends to an apparently rising public misperception of the nature of scientific inquiry, rather than a merely healthy skepticism of scientific integrity. Public misperception of the nature of scientific inquiry and truth is fueled by the less-than idealized process of scientific publication. The stuttering progress of scientific inquiry necessitated by even electronic journal publishing, overwhelmed by articles and understaffed by reviewers, inspires little public understanding of the incremental and steady progress of science overall. The relation of article publishing to career promotion in academic science has also promoted “salami science,” in which a single scientific study may generate more papers than would practically be necessary to disseminate the results of a single study. This is often done by “slicing” up the study in an unnecessary way, splitting apart either a single research program into more parts than is logical or necessary, in order to generate interim publications (padding the CVs of the researchers), or even splitting single experiments into more than one published piece. Frequent, incremental publications may do little to promote scientific ideals, and may hinder those ideals if the result is that real breakthroughs are revealed only in parts, and time must be wasted re-piecing together a single study that has been incoherently sliced for the sake of lengthier CVs.

A New Paradigm?

An idealized model would open this steady, incremental growth of scientific knowledge and inquiry to constant public and scientific scrutiny. This is the model afforded real possibility only recently with the emergence of wikis. The Wikipedia model is a reflection of the idealized scientific model, and should replace the time-worn and flawed model of scientific publishing we have grown accustomed to over the past few centuries. In idealized science, authorship is subsumed to the more important qualification of usefulness in discovering and explaining natural truths. Original authorship is less important than the impact of an idea on future experiments and developing theories, and confirmation by repeated experiment. In idealized science, the search for the laws of nature and explanations of phenomena is collaborative and ongoing, and does not become fixed in a final text, but is always subject to revision and refinement in the face of new evidence. The wiki model is the natural extension of the initial model of science employed in the early salons and meetings of the first scientific societies, before publishing overtook the processes employed by natural philosophers as the be-all and end-all of science itself. Wiki technology, first developed by Ward Cunningham in 1995 and improved by countless others since then in true wiki fashion, allows for anyone to contribute to, or author, a hypertext on the World-Wide-Web. Its most famous implementation is in Wikipedia, which now spans continents and encompasses all subject matters and numerous languages in an attempt to build the largest, most comprehensive encyclopedia ever. It has succeeded brilliantly. According to Nature, as of 2005, Wikipedia had grown from nothing to 1.8 million articles (now 2,540,416) in 200 languages, with nearly 15,000 contributors. More importantly, although there is no vetting process for authors, and anyone may create, edit, delete or comment on any article, the Nature study found that Wikipedia’s accuracy rivaled that of the Encyclopedia Britannica, at least in articles regarding scientific subjects. While the openness of the wiki technology presents the possibility of real mischief, and allows utterly un-vetted information to be briefly introduced, the reality of a peer-review process thrown wide-open has resulted in speedy corrections and the development of new community standards for cooperation and steady improvement. Wiki technology could overcome the temporal and pragmatic limitations of scientific journal publication, but only if certain long-held assumptions about the roles of publishing in academic science, and deeply ingrained prejudices are overturned. Let us consider what changes would be necessary in the institutions of science and academia were the wiki model of publishing adopted before considering the net benefits that might be realized.

Overcoming The “Publish-or-Perish” Mentality

The greatest threat to full-scale adoption of wiki-science rather than journal publishing is the role that journal articles now serve in academic science, particularly in the promotion and tenure process. There is simply no greater factor weighed in most promotion and tenure decisions for academic scientists, and most science is still done in an academic setting. When being considered for promotion and tenure, publications are judged for two qualities: number or articles, and “impact factor” of journals. While there are certainly other important achievements weighed in promotion and tenure decisions, including grants attained and to some extent teaching and guidance of students, the weight afforded to journal publishing has had some unwelcome and unintended consequences for science as an institution. One notable consequence of the demand for journal articles on scientific resumes is the hyper-abundance of journals. If Science and Nature were all that were available, then they would not only be flooded with an untenable number of submissions, but few scientists would be able to publish. The result would be bare CVs, and few tenured scientists. Moreover, the progress of science would then be hindered as results would be delayed or withheld from publication due to overly narrow channels. So publications have necessarily increased in number, with increasingly specialized journals proliferating. Of course new ones, untested by years of community acceptance, maintain low “impact factors,” but still they serve as available channels for publishing results, and help build young researcher’s CVs. The proliferation of journals has not solved the problems inherent in the journal-publishing paradigm of science. Most journals will not become widely-read, and most have very low impact-factors, meaning that publications in them, even if groundbreaking, risk going largely unnoticed in the broader scientific community. This risk also impacts science. Real breakthroughs or even important new studies may not receive the attention of researchers in appropriate or related fields given the abundance of articles and limited time available to researchers to simply read new articles. Researchers must try to balance their need to do bench work, write, and publish, against an ongoing duty to remain abreast of new knowledge. Ultimately, scientific inquiry itself likely suffers, and the promise afforded by new technologies does little to enable better communication, dissemination, and collegial testing and criticism. Another casualty of the pressure for publications overshadowing the scientific goal of filtering out bad science from good is scientific integrity. Some researchers impelled by career pressures may rush to publish results before they are properly confirmed, leading to inevitable and apparently increasing numbers of wholesale retractions of papers. In some cases, fraud rather than carelessness is the problem. In either case, scientific integrity is the victim, and public perception of the characters of scientists as well as the integrity of the institutions and processes of science suffer as a result. Of course, none of this squares well with the idealized vision of science, though arguably, in the long run, truth does emerge from the whole messy process. Is it possible to envision restructuring academic science so that a more idealized version both of scientific progress and of recognition of academic and scientific merit prevailed? Can the wiki model be used as a backbone of a revamped form of scientific publishing that could both help ensure greater, swifter, fairer and more collegial scientific progress, and help ensure that greater scientific integrity prevails?

Speedier, More Transparent, and Democratic

One of the first responses to significant lapses of professional integrity, whether in science or in other realms, is developing greater transparency and thus accountability. Time and again, when faced with scientific fraud, publishing misconduct, and conflicts of interest in research, new mechanisms and regulatory institutions have been developed with the intent of creating greater transparency and improving accountability through voluntary disclosures. The technology of wikis has both transparency and accountability built-in, and naturally encourages disclosure. Wikis only work if authors are trusted. Because of their openness, wikis are naturally susceptible to editing by anyone, regardless of his or her expertise on a particular subject. Yet, because of the ongoing attention of peers, errors tend to be rapidly corrected, and editors who contribute poor or false entries become less trusted. The result, as borne out by experience, is a robust and trustworthy repository of quickly updated knowledge at low cost. Wiki technology democratizes peer review, and by spreading the workload among any self-identified peer, speeds up the process as well. Scientists are already discovering the practical uses of wikis in ongoing research, using it as a collaborative workspace for specific research projects. Because of its open and immediate nature, wiki technology could help alleviate some of those factors which help cause scientific and publishing misconduct. Increased transparency, and the threat of immediate editing, correction, censure, or ostracism by peers might overcome the temptation created by the lag-time and lack of immediate interactivity in other published media. The problem remains, however, as to how to develop means of academic recognition in a wiki-based environment. Let’s conclude by exploring some possible ways that a universal “science-wiki” could supplant traditional scientific publishing and methods of professional evaluation might yet be maintained in academia. Wiki technology could help create a universal repository for scientific knowledge, free from the bureaucratic and pragmatic obstructions involved in traditional journal publishing. In so doing, it would democratize the scientific process, create greater transparency and accountability, and hopefully speed up the dissemination of experimental results, criticisms, propagation and refinement of hypotheses, and development of robust theories. Unfortunately, it would eliminate, or at least complicate, an age-old manner of evaluating researchers for promotion and tenure decisions. Or perhaps this would be a fortunate consequence forcing us to rethink how academic scientists are evaluated, and reemphasize the collective goal of discovery and invention rather than individual careers. All of which presupposes a number of things, including:

- science as an institution must value the pursuit of truths above individual careers,

- scientists who make positive contributions to the pursuit of truths ought to be recognized,

- a CV listing publications is not necessarily an accurate reflection of a scientist’s contributions to the institutions of science,

- wiki technology can capture individual contributions of authors and researchers in some fair manner.

- and, career decision can be made based upon real contributions to scientific institutions.

All of these assumptions ought to be generally acceptable, at least in the context of idealized scientific method. Can we devise new methods for professional evaluation and recognition of individual achievement and still improve the methods of the sciences with wikis? As long as wiki entries are properly referenced, ascribing original research and findings to the actual contributors of knowledge, then a scientist’s professional dossier can be tracked with wiki technology. It will be possible to see not only whether a particular researcher contributed to new knowledge, but to what degree that contribution has influenced other scientific developments. Of course, none of the technologies and methods proposed for this version of “Science 3.0” precludes disseminating knowledge in more traditional ways as well. Presumably, scientists will still co-author papers with colleagues, present and debate findings in public forums, and write books. The Wikipedia model and its community have already devised means of vetting, appealing, flagging, and otherwise retaining means of tracking both users and entries in a way that ought to help avoid mischief and cheating in a centralized WikiScience. A recent article in The Economist discusses the merits of science “blogging” as a way to open up research, avoid some of the pitfalls to traditional publishing, and work collaboratively. (Sept. 20, 2008, p. 85). This too could work hand-in-hand with wiki-science, providing references and greater depth to entries, and serving as a parallel channel of communication among scientists and the general public. As the Economist article recognizes, any significant change in the way that the institutions of science work (not to mention academia) is a “chicken and egg” problem. But as scientists themselves begin to embrace these tools for the sake of scientific communication, collaboration, and record-keeping, barriers to adoption should begin to fall, as they have in the realm of electronic journals. At one time not so long ago, electronic journals were viewed as substandard and unworthy, but now no such knee-jerk dismissal of the medium exists, and excellent electronic journals are universally acknowledged. The same will be true for WikiScience. The promise and usefulness of the technology itself will outweigh institutional prejudices, and Vannevar Bush’s universal “memex” machine, envisioned in his 1945 Atlantic Monthly article “As We May Think” (and an inspiration for wikis) will finally be realized.

References:

2007 Maxwell J. W. “Using wiki as a multi-mode publishing platform,” SIGDOC ’07, ACM 978-1-59593-588-5/07/0010.

2005 Editorial, “The cost of salami slicing,” Nature Materials, Vol. 4, No. 1, Jan, p. 1.

2005 Maier H., Dohr S., Grote K., O’Keefe S., Werner T., Hrabe de Angelis M., Schneider R., “LitMiner and WikiGene: identifying problem-related key players of gene regulation using publication abstracts,” Nucleic Acids Research, Vol. 33, web server issue W779-W782.

2007 Tumlin M., Harris S.R., Buchanan H, Schmidt K., “Collectivism vs. individualism in a wiki world: librarians respond to Jaron Lanier’s essay ‘digital Maoism: the hazards of the new online collectivism,” (Johnson K., ed.) Serials Review, 0098-7913.

2008 “User-generated science,” The Economist Sept. 20, p. 85.