Vitruvian design for scholarship in the humanities

A checklist for writers

2012-01-31T06:48:00.001-08:00

Technology both shapes and reflects our values. What do we value in scholarly writing, and how well do our technological choices match those values?

I look for software that supports four necessary or possible qualities of good scholarly writing:

expository writing should be explicit and unambiguous
the writing process is iterative: good writing only comes from rewriting
academic writing in the natural sciences is often collaborative; this is becoming less rare in the humanities (although not necessarily in the cargo-cult humanities )
born-digital writing should be reusable

In a digital enviroment, to write explicitly and unambiguously means more than choosing our words well: it also means expressing the structure and contents of our writing explicitly and unambiguously. Our writing should embody the fundamental principle of separating concerns in our digital work: our first goal is to express our ideas clearly, not to exercise our typesetting skills, so we need a format that that can explicitly and unambiguously express structure. We might choose an XML-based semantic markup system, or some semantically classed “markdown” system such as markdown or textile. What we should not choose is a “word processor.” Even if you can approximate a semantic structure using a carefully chosen set of “styles” (a tell-tale term!), you will be planting your semantic hints in a thick forest of code focused on the particulars of displaying your text visually. Note that it’s perfectly possible to express this irrelevant information using XML formats like OpenDocument. Our question is not “is this an XML format?” but “does this format express the semantics of my document?”

In considering how to support the remaining items in our list, we should look for examples beyond the humanities, since expository prose is not the only form of writing that shares these qualities. In particular, each is characteristic of good composition in computer programming, and computer programmers routinely use software that directly takes account of each of these qualities.

Programmers use version control systems to work with the entire history of a document to update, restore or compare versions. Version control systems also simplify collaboration, and allow mulitiple contributors to work simultaneously on a document. Changes can be silently integrated and shared; if two authors simultaneously make conflicting changes, version control systems can recognize that, and offer authors options to reconcile conflicts manually. There are many good, freely available version control systems. One reason that humanists are less familiar with them than they should be is that version control systems work best with textual data: the binary formats that word processors produce are a major obstacle to integrating our writing in version control, but once we have adopted a text-based semantic format, that obstacle vanishes, and we have a writer’s desktop that lets us write iteratively and collaboratively.

Programmers also provide a model for reusing our writing. Units of code are often packaged in libraries that other programs use. Programmers working on large projects manage the potentially complex interrelations and dependencies of of different libraries and programs using build systems. We are not yet accustomed to thinking about automating the reuse of our writing, but there is no technical obstacle to doing so. We could use build systems to assemble chapters into a book, incorporate common navigational headers into all the pages on a web site, or automatically update an index if one section of a text changes, to name just a few obvious examples.

So our checklist of required tools for writers includes:

an editor that works comfortably with semantically structured text
a version control system
a build system

I plan to add a series of posts with the tag writing to look at how we can work with tools like these to write more effectively in a digital setting. Meanwhile, take the checklist to your college or university IT department, and ask what specific software they support for semantic editors, version control systems and build systems. I would love to learn of an academic institution that is not just pressing commercial word processing software on its students and faculty, but I don’t know of one.

The recursive arithmetic of tenure

2012-01-30T05:06:00.000-08:00

The long career path from college student to a tenured academic job is designed to be conservative. A student in the humanities who discovers a passion for an academic subject in his or her first year of college can expect that four years of college will be followed by, say, six years of graduate school that not only provide training in a discipline, but initiate the student in its culture. The (increasingly rare) PhD who then immediately walks into a tenure-track job typically faces seven years of scrutiny before a tenure decision. Newly tenured professors have proven that their work meets the professional standards of their colleagues — seventeen years after entering college.

Like many professors, I hope that a college education is a formative experience in the lives of my students. Imagine that the newly tenured professor was inspired, seventeen years ago, by an exciting teacher and scholar. That person of course would have climbed the rungs of the same professional ladder, so the youngest tenured professor who could have inspired today’s youngest tenured professor might in turn have first been inspired as a new college student … 34 years ago.

In 1977, the late Steve Jobs was just starting a company he had formed the previous year to sell the computers he and Steve Wozniak were building in his father’s garage.

We’re trying to cross an ocean by standing at the shore and waiting for continental drift to carry us to the other side.

The humanities-that-must-not-be-named

2012-01-30T03:50:00.000-08:00

I’m not thrilled with the term “digital humanities.” When people refer to the “humanities,” I think I know what they mean: those disciplines that are concerned with human activity and everything it produces, and take as their task both to preserve and transmit that culture on the one hand, and to understand and interpret it on the other. But what is the sense of qualifying that noun with the adjective “digital”?

In the twenty-first century, the phrase can’t really stand in opposition to an implied “analog humanities”: no such thing exists. (When was the last time anyone submitted a hand-written or manually typed manuscript to be edited with grease pencil before being manually typeset with hot lead?) “Digital humanities” refers instead to scholarship in the humanities that consciously takes account of the fact that we all work digitally now.

What troubles me is that our use of the marked term “digital humanities” implies that the unmarked term, “humanities,” is being used to refer to scholarship that does not reflect on the media we all work in (a usage that is sadly accurate in the academy today). I am particularly disturbed because I would like to imagine that an education in the humanities encourages the kind of critical self-awareness that would enable us to think more meaningfully about our relation to the environment we live and work in, including our technological environment and the ways it is interwoven with our institutions and values.

By using “digital humanities,” we’re allowing the term “humanities” to stand for an uncritical scholarly practice that is at odds with the goals of a humanistic education.

I can understand why there is not a spontaneous groundswell of support in academic departments around the world for a term meaning “work that unthinkingly perpetuates obsolete forms of scholarly practice,” or “scholarship that is oblivious to the media we use today,” but rather than accept without reservation the marginalizing label “digital humanities,” I’ll offer my own suggestion. We could extend Richard Feynman’s “cargo-cult science” to “cargo-cult scholarship” more generally, and refer to the “cargo-cult humanities.”

“Digital natives”

2012-01-29T12:52:00.000-08:00

I recently attended a workshop at my home institution where I heard teachers confidently assert that today’s students are so adept at technological tasks that we can rely on them to help their older teachers develop important technological skills.

Really?

For more than 15 years, I’ve introduced Classics students at Holy Cross to XML markup. To build on any prior experience they might have, I routinely begin by asking who has ever peeked behind a web page to view its HTML source. Fifteen years ago, I would usually find anywhere from a quarter to a half of the students would say yes. Today, if I ask a group of 20–25 students, I will get one or two “yes” answers.

I do not know if my students were telling me the truth fifteen years ago (or today), but that doesn’t much matter for my present point. Fifteen years ago, far more students either had seen HTML or felt some kind of pressure to pretend that they had.

What does it mean? I suspect that the “digital natives” I teach have indeed grown up so familiar with information technology that they are more oblivious to it than their elders. I worry that they are also incurious, or at least need to learn to be curious about it.

My personal experience makes up only a limited sample, of students in Classics at a small liberal-arts college, but the trend among those students is very clear. Unless someone can show me better evidence, I’ll remain very sceptical about a priori assertions concerning the skills that “digital natives” will confer on their teachers.

Note: new tags

I’m using a couple of new tags on this post: “sceptical” (for obvious reasons), and “yam” [yet another meeting] to help me find posts responding to ideas I’ve gathered from yammering at meetings. I hope to post soon on a couple of additional “sceptical” topics, and several “yam” topics (since January is a big month for meetings in the academic world).

Tending our garden

2011-10-20T09:01:00.001-07:00

To an astonishing extent, Google really has changed the way the world's information is organized.

A pair of trivial examples:

[1] Liddell-Scott-Jones is the best lexicon of ancient Greek in any language, but I don't always find the search interface to the digital version from the Perseus project to be the easiest navigation system. I recently discovered a simple way to look up an entry in Perseus: type the lemma of the Greek word in Google. Duh.

The point is that Perseus does not — and should not — have to worry about how people search for articles in LSJ as long as the Perseus edition is cleanly organized by article with a recognizably tagged lemma. Since I'm frequently browsing the web from OS X, typing UTF-8 Greek is trivial, thanks to SophoKeys. Perseus does not — and should not — have to worry about keyboard input systems. The digital LSJ was available from Perseus long before Google or SophoKeys came along, but it's more readily usable and that much more valuable to me now because the Perseus project did take responsibility for its area of domain-specific knowledge, and properly organized the contents of the lexicon by article.

[2] urn:cts:greekLit:tlg0012.tlg001 is a URN referring to the Iliad. URNs are not direct addresses like URLs: they are technologically independent (although they follow a machine-parseable syntax). Enter that URN in Google, and you'll probably get some amusingly off-target advertisements for cremation urns, but I scrolled through several pages of Google's hit list without encountering a single irrelevant match.

One implication of these examples is not trivial. If we can publish meaningful units of content in association with canonical identifiers like URNs, we can count on the ever-increasing momentum of the internet to create new ways of discovering and working with that content. Like Pangloss, we need to tend our own garden; Google or others will find it rapidly (whether or not we think that makes for the best of all possible worlds).

Steve Jobs on building things

2011-10-06T07:17:00.001-07:00

Amid today's many recollections of Steve Jobs citing his quotable aphorisms, two particularly caught my attention:

People don't know what they want until you build it.

and related to that theme:

The builders are the real thinkers.

Jobs was immediately referring to engineers and consumers, but the statements are as true in scholarship as they are in any other endeavor.

Access to scholarly work

2011-09-28T07:43:00.000-07:00

Members of Princeton University's faculty have unanimously voted for a policy guaranteeing open access to their scholarship (blogged with quotation of the key passage here).

Contrast with that the capital campaign of the American Philological Association (a professional organization purporting to represent the discipline of Classics). The APA's "Campaign for Classics" plans to offer access to digital resources, but in many cases that access will be restricted to APA members.

If the contrast is not pointed enough, think of it this way: as of September, 2011, Princeton faculty members risk violating their university's policy if they contribute scholarly work to the APA project.

If you believe that consistent principles should guide our behavior, then Princeton faculty members who are dues-paying members of the APA face a real ethical dilemma: how can they support the work of an organization that directly conflicts with the policies unanimously adopted by their university?

Druids in Oxford?

2011-09-20T07:00:00.000-07:00

A colleague recently pointed me to the "Ancient Lives" project allowing members of the general public to view and transcribe papyri from the vast Oxyrhynchus collection that remains largely unpublished more than a century after its discovery.

I imagined something like the ground-breaking Suda Online (or, SOL) -- an astonishingly successful project that has now translated for the first time ever more than 29,000 of the 30,000-odd articles in a Byzantine encyclopedia of the ancient world.

Instead, when I followed the link on copyright on the "Ancient Lives" website, I read:

Images may not be copied or offloaded, and the images and their texts may not be published.

This reminded me not of the forward-looking SOL, but of Julius Caesar's description of Gallic Druids. From Caesar, Gallic Wars, 6.14:

Neque fas esse existimant ea litteris mandare ... Id mihi ... instituisse videntur, quod neque in vulgum disciplinam efferri velint ..

"They consider it wrong to commit these [sacred texts] to writing. I believe that they have established this practice because they do not want their professional knowledge to be published to the common people."

Chrome apps and extensions

2011-01-18T05:34:00.000-08:00

You follow good practices, archive your priceless material in standard formats, and make it accessible through network services. Now where do you build applications to use those services?

In digital scholarship, your time is your single most valuable resource, so the cost of a full-blown cross-platform desktop application is usually prohibitively high. As CSS 3 and javascript give Web browsers better and better support for visually rich interactive applications, the browser becomes more attractive as a platform for applications, rather than just a viewer for documents.

HTML5 clearly understands the browser this way. (If you're not familiar with HTML5 yet, see http://www.html5rocks.com/) The blogosphere is full of comments about HTML5's audio and video capabilities, but its integration of local and networked resources raises more interesting architectural questions. HTML5 can give you access to local file storage or locally persistent databases as well as providing networked communication with remote processes. With an HTML 5 platform, the browser is more like a local application with good access to remote data.

Google has taken this idea a step further with the definition of extensions and apps for its Chrome browser. They are written in HTML, javascript and CSS: it is perfectly possible to write build a "Chrome app" that is nothing more than a web page displaying equally well in any browser. (For a web developer, the road to "Hello, world" has never been shorter.) The Chrome app is defined in a simple JSON manifest file. The manifest can define permissions for access to remote resources (no more work-arounds to deal with restrictions on unsafe cross-site scripting). It also defines how your app or extension is integrated into Chrome. A single line in the manifest can tie your app to a button permanently available on your toolbar, accessible from your choice of user action (button click, key-press combination, etc). Full Chrome apps can be distributed through Google's App Store where any Chrome user could install the app locally.

If you've ever looked at developing browser-based web applications, or have ever thought about extending Firefox with its extension mechanism, you owe it to yourself to take a quick look at

http://code.google.com/chrome/extensions/index.html. That's all it takes to get started.

Thinking about citation

2010-10-11T09:39:00.000-07:00

In digital scholarship, citation is the most fundamental form of ontology. Identifying an entity in a recognized form of citation allows scholars to agree that the object exists, while leaving room to disagree over how to represent it. Once we agree that there is an object I call "the Parthenon in Athens," and can unambiguously cite that object, we allow software to recognize that the same object might be represented in a GIS with spatial data, in a photo gallery with a collection of photographs, or in an architectural database with structured fields of textual data.

For digital scholars, it's hard to avoid talking or even thinking about scholarly reference systems without getting bogged down in technology-specific tar pits. Take the W3C's Resource Description Framework as an example. Its triplet model is brilliantly simple and powerful: whether you think of it as "subject-verb-object" or "object-directed link-object", it is general, extensible, and lends itself readily to both abstract graph models and real machine implementations. Its syntax is expressed in terms of URIs, which could equally well be URLs (real addresses in the "http" schema), or URNs (abstract names in the "urn" schema). (See the W3C's document "URI Clarification" or this discussion "Untangle URIs, URLs, and URNs" to sort out the acronyms.) But when was the last time you saw a scholarly project using RDF to describe relations among objects identified by abstract name, rather than by address? In an RDF graph, "http" is a good URI scheme for an application retrieving material on the internet, but a poor choice for a description of relations among persistent and immutable objects, a case where the "urn" scheme would be more appropriate.

For canonically citable texts, work at the Center for Hellenic Studies on has led to:

identifying abstract properties of canonical texts
developing a human readable and machine actionable notation for citation that expresses these properties (the CTS URN notation)
defining a service for identifying and retrieving texts identified by this notation (the Canonical Text Services protocol)

I've written a bit about this in a dry article on "Digital infrastructure and the Homer Multitext Project" but here I would just note that this three-tiered hierarchy has been very useful both in trying to think about citation outside of any particular technological context, and for defining machine-actionable tests to evaluate implementations of text services. Only level [3] deals with specific technologies on the internet. If you don't want to interact with a Canonical Text Service, the CTS URN notation [2] can still be used by any application referring to canonically cited texts. If you don't like the CTS URN notation, you can still evaluate any alternative notation by seeing whether it implements the abstract properties of [1]. And if you are dissatisfied with the identification of those properties, you can start from scratch and redefine what you think a canonically citable text really is.

I think this graded distinction of abstract property, reference notation, and application could be equally valuable in citing other kinds of material. In a following series of posts, I'll look successively at each level to analyze how we might cite uniquely identified objects.

Is 2010 "the year of open data" in Classics?

2010-08-28T04:01:00.000-07:00

Tim Berners-Lee, the creator of the World Wide Web, has called for "raw data now"; in a TED talk this spring, he showed examples of what can happen when people have access to openly licensed and freely reusable data sets.

The American Philological Association thinks the internet is a gated community. The lead story on the APA's website is the continuing effort to raise funds for a "portal" that will help members find resources available only to subscribers.

Compare Berners-Lee's talk (freely licensed so I can legally embed it in this blog post), with the APA's video presentation of its campaign (from the APA website either in Quicktime or Windows Media format). Which vision of sharing scientific and scholarly data do you see as the future of Classics?

What's the difference?

2010-08-27T02:24:00.000-07:00

How do we compare two texts? For line-by-line comparison of electronic files, Unix systems have had a diff command since time immemorial. (A version had already been around long enough for Hunt and McIlroy to write an article about it in 1976, before I had ever laid eyes on a computer, and before any of my students were born.) For XML documents, XMLUnit is a java library that can describe differences not only in the text nodes of two documents, but in their XML structure as well.

Both diff and XMLUnit describe differences between electronic documents in specific formats, and report on differences in terms of that format (line-by-line structure from diff; XML structure from XMLUnit). Neither solution is adequate for the Homer Multitext project. Its collection of texts is not defined in terms of specific document formats: instead, texts are managed and manipulated by the more abstract references of canonical citation

(e.g., Homeric papyri; and texts from Byzantine manuscripts). How would we compare two passages of texts identified by canonical citations?

Traditional Homeric scholarship suggests an approach. Homerists refer to "vertical" vs. "horizontal" variation across versions of the Iliad. "Vertical variation" refers to entire lines that are present in one version but not another; "horizontal variation" refers to differences across versions in "the same" line (that is, a line that is canonically cited by the same reference). This can be generalized to any canonically citable text, if we think of vertical variation as the differences not just of Iliadic lines, but of the citation units of any text. (By this way of thinking, manuscripts of the Gospel of John that include the longer ending would show "vertical" variation compared to manuscripts that stop after the shorter ending.) Horizontal variation then would be a description of the variation within a single citation unit (whether that is a line of the Iliad or verse of John).

We could then reduce vertical difference to an operation on two ordered lists of citation values. Analogously, if we can tokenize the textual content of a citable passage, we could treat horizontal difference as an operation on two ordered lists of tokens. (Tokenizing texts has its own challenges, but I'll blog on that topic some other time.)

This is a happy result. First, we can acquire the data for a comparison with existing requests in the CTS protocol, to determine citation values, or to find textual content of a passage. Second, we can apply a single algorithm to both vertical/structural variation, and horizontal/intra-node variation. Third, comparing two ordered sequences of tokens is one of the best studied problems in computer science (since applications like comparing DNA sequences have attracted a lot more attention than differences in Homeric manuscripts). I googled up this site and had a basic collation class running in a few minutes (but a good programmer would be able to do this far more easily than my stumbling efforts).

This raises the far more interesting question: what exactly do we want to know about the vertical and horizontal differences between two passages? Some of the things we can determine include:

what tokens are unique to passage A?
what tokens are unique to passage B?
what is the longest common sequence of the two lists?
what is the complete ordered union of the two lists, and the status of each token in the list (A only, B only, in both, in both but in a different position) ?

I've blogged a very minimal example comparing two manuscripts of the Iliad. There's lots more to do.

Who does Classics? Where?

2010-08-24T03:07:00.000-07:00

David Bamman's presentation was only one of several high points at last week's meeting at Tufts University on "Greek, Latin, Arabic." We heard from the Alpheios project about recent development of their language learning tools. I'm thrilled to be using alpheios this fall both as a teacher of intermediate Latin and a student of first-semester Arabic, but what continues to impress me most about the project is the thoughtfulness of its architecture. The lexica (such as Liddell-Scott-Jones for Greek, and Lewis-Short for Latin) and linguistic information (very comprehensive morphological analyses, and for some sets of texts, syntactic tree banks of the kind David Bamman's research uses) are cleanly organized as services that are accessible over the internet. If there is another project in the digital humanities that has grasped this fundamental architectural principle as clearly as the Alpheios project has, I'm not familiar with it.

Also in attendance was Google's Will Brockman, who was able to comment on the recent public release of scans of over 500 Greek and Latin texts. (Six copies from three different editions of Pomponius Mela! Can you do that in your home library?)

A dynamically constructed lexicon; network services exposing Greek and Latin lexical and linguistic information to the internet ; a corpus of freely available texts — individually, these are major contributions to the study of Classics. Collectively, they really do lay the foundations for a radically altered discipline — and they exist today. If I wasn't constantly hearing from fellow classicists that our discipline is in crisis, I would think that there has never been a better time to study Greek and Latin.

Oh, and who does this work? David Bamman is a senior researcher at the Perseus project, not a member of an academic department. The Alpheios project is independent of any academic institutional affiliation. Google — well, you've heard of Google.

I was reminded of Brockman's blog post when he first announced Google's free release of Greek and Latin texts in June. He cites three examples of studying ancient texts that excite him: reading a Latin text in the Perseus project's interactive edition; reading an article about Sophocles in English from the Suda On Line; and consulting the high-resolution photography of the Venetus A manuscript from the Homer Multitext project. His selection caught my eye, because I've been involved in all three projects, and know some of the back stories. None of the junior members of the original Perseus project were tenured at their original home institutions: all moved to other jobs, or left the field altogether. When an external review committee visited the University of Kentucky in the 1990s, after an extensive presentation about the Stoa prominently including the Suda On Line, a classicist asked the late Ross Scaife, "In what way does any of this constitute scholarship?" (A curious question about the first effort ever to translate into any language the rich and complex text of the Suda.) The Homer Multitext project has not faced such overt hostility, but it is interesting to note that it originates from the Center for Hellenic Studies (a branch of Harvard University in Washington, D.C., independent of the Department of the Classics), and that the two editors and two project architects all hold academic positions at institutions that do not grant PhDs in Classics. (Speaking for myself, I couldn't be happier about that.)

Connect the dots however you like. I draw two conclusions: first, that the study of classics is far too important to leave to classicists; and second that the study of Greek and Latin is still exciting enough to attract brilliant contributions from committed scholars who are not shackled with a title like "Professor of Classics." In 2010, I'm starting to envy my students, and wish I had a few more decades to continue this work.

Greek, Latin, Arabic

2010-08-23T17:52:00.000-07:00

Last week, I attended a meeting on "Greek, Latin, Arabic" at Tufts University. Some of the most stimulating discussion focused on how it's possible to exploit more or less limited corpora of structured digital texts to find valuable information in a larger, less structured morass (think: the internet). Lots of interesting research worth blogging about (although you won't be likely to hear about any of it if you go to an APA convention), but I want to comment briefly on David Bamman's presentation, because I think his work is as significant as any research I've seen in classics in the past 30 years.

For some time, Bamman has been pursuing interesting work in two distinct areas: (1) automatic alignment of texts in different languages, and (2) using dependency treebanks to represent the syntax of Greek and Latin. I've followed his progress for a couple of years, but last week was the first occasion when I've begun to realize how he can weave these two strands of work together. (That's only a comment on my own obtuseness.) If you want to jump over the methods, and go straight to an astonishing result, follow this link.

It's a dynamically induced lexicon. Stop, and reread that sentence. It's a dynamically induced lexicon.

I've read this (preprint of an) article about projecting markup across translations a couple of times. That's not really enough, and I'll probably reread it tomorrow, but let me reduce one result to this summary: at the level of individual words, Bamman achieves about a 70% success rate aligning roughly five million words of Greek with seven million words of English. That in itself is fairly astonishing, but Bamman also leverages his work building treebanks to model the syntax of Greek and Latin texts. Crossing the syntactic models from his treebanking with information gleaned by aligning versions texts in different languages, Bamman builds a dynamic lexicon that can take a Greek term and trace how translations in a language like English render that term in different sets of texts, including recognizing the syntactic constructions in which the term appears; or conversely, he can take an English term, give the most closely corresponding Greek terms, and from there, again lead you through the history of the Greek term, as glossed or explicated (automatically) by its English translations.

In his presentation, as in the written publications I have seen to date, Bamman's work simultaneously shows the general implications of his research for computational linguistics, and how the Latin and Greek case studies he has chosen are distinctive. That tension between generality and specificity is, I think, often the hallmark of really great scholarship — a category David Bamman's work clearly falls into, in my view. If there are any lovers of classical languages or literature who still doubt whether they are computational linguists, Bamman should persuade you that in 2010 we all are all computational linguists.

MIT faculty mandate open access to publication

2009-03-26T11:26:00.000-07:00

Blogged here by Hal Abelson, chair of the committee that composed the resolution: the MIT faculty have voted unanimously to adopt a resolution that includes these phrases:

[E]ach Faculty member grants to MIT a nonexclusive, irrevocable, paid-up, worldwide license to exercise any and all rights under copyright relating to each of his or her scholarly articles, in any medium

and

The policy is to take effect immediately

Think that's clear enough for any of the hair-splitting legalists out there?

I believed there was time when humanists set the moral direction for the academy. We're lucky that our scientists' pursuit of truth seems to be generating enough of a draft to pull us along.

The vocabulary of ancient Greek

2008-11-26T05:17:00.000-08:00

What is the vocabulary of ancient Greek? That is, what set of words, or lexical entities, actually occur in our extant texts?

The First Thousand Years of Greek project (announced here) aims to simplify posing such straightforward questions, but we need more than online texts to talk unambiguously about words. One essential piece of infrastructure is an inventory of uniquely identified lexical entities in Greek. In print publications, lexical entities have traditionally been identified by a word's lemma form. While lemmata are valuable labels, they are potentially ambiguous. Instead, basic principles of information design dictate that arbitrary identifiers guaranteed to be unique should be associated with lemma strings, so that references to a lexical entity can be unambiguously machine processed (using the identifier), and remain intelligible to human readers (using the labelling lemma string).

The Perseus project has given classicists two monumental resources that must be coordinated with an inventory of lexical entities: the digital LSJ lexicon of Greek, and the Morpheus morphological parsing system that can associate surface forms of words with a lemma. Taken together with the invaluable list Peter Heslin has created by running Perseus' morphological parser over the word list of the TLG project's E disk, they provide an obvious starting point for an inventory of Greek lexical entities would be to compare these two resources.

The digital LSJ has already been provided with unique identifiers for each entry, and each entry includes a lemma string. Perseus' morphological analyses identify entities by lemma. Where there is a one-to-one mapping between the parser's lemma and the LSJ lemma (normalized so that LSJ's markings of long and short vowels are removed), we can fairly assume that they represent the same entity, and could simply adopt the LSJ identifier to refer to the more general notion of the lexical entity — an unambiguous reference that could be associated with an entry in the lexicon, with morphological analyses, or with any other information.

While this simple (and easily automated) task takes care of the vast majority of the vocabulary in both the LSJ and in the parser's output, there are several categories of problematic cases. They include:

entities where LSJ's orthography differs from the parser's orthography. This is actually a large group with several subcategories, some of which can probably be reliably resolved automatically. For example, LSJ and Morpheus sometimes disagree on whether the lemma form of a verb should be active or middle/passive voice: a a careful script could accommodate that kind of variation, but human intervention would be necessary when LSJ and Morpheus use alternate forms of the lemma.

entities that appear in the parser's list of lemmata, but not in LSJ. This occurs frequently with compound verbs that are not given separate articles in LSJ. In these cases, since there is no LSJ identifier to reuse, we would, obviously, need to create new identifiers for those entities not in LSJ.

"ghost entities." For reasons that are not clear to me, LSJ routinely lists verbal adjectives in -τέον as distinct entities, unconnected to the verb from which they are formed. (E.g., the adjective λυτέον is a distinct entry, unrelated to the verb λύω.) Whatever the reasoning, in a digital environment, this is the wrong taxonomy: the morphological analysis should allow applications to distinguish verbal adjectives from other forms deriving from the same verbal root, while the identifier for the lexical entity should recognize verbal adjectives and conjugated forms of a verb alike as forms of the same entity. Mapping these LSJ and Morpheus lemmata to the correct verbal lemmata will be a relatively straightforward task, but again will need human supervision for some common cases (e.g., δοτέον < δίδωμι).

entities in LSJ but not in the list of lemmata generated by running the parser over the TLG E word list. Presumably, these result from the contributors to LSJ covering texts that are beyond the scope of the TLG E disk's corpus. As a basic principle, we should make absolutely explicit what digital corpus of texts an inventory of lexical entities is based on. Since our first pass is working from Heslin's analysis of the TLG E corpus, we should not enter these LSJ IDs into our inventory — at least, not yet. As the inventory is checked against further texts, new vocabulary may appear, and at that time new candidates for addition to the inventory will need to be checked in both LSJ and Morpheus.

That is a substantial, but I think manageable, list of tasks. One easy way to begin would be to limit the scope of coverage further, and rather than beginning from the entire TLG E word list, start with a word list created from a specified corpus of texts. As lemmatized word indices for the First Thousand Years of Greek are released, we will guarantee that all surface forms of a word are resolved to a uniquely identified lexical entity.

Beyond text

2008-10-30T05:09:00.000-07:00

If you are interested in the architecture of scholarly resources, run, don't walk, to Gabe Weaver's new sourceforge site, episteme. The nascent site (opened to coincide with the public release of "digital product" from the Archimedes Palimpsest project) documents his work representing and manipulating information encoded as mathematical diagrams.

There's already a lot to think about here, but one intriguing aspect is that entities in figures are referred to with identifiers that can be coordinated with canonical references to passages of textual content from the same document. (Short-term consequence for me personally — urgent need to re-think my presentation for the "Text and Graphics" panel at next week's TEI meeting in London. Ouch.)

Oh, and if you just want to enjoy some beautiful drawings, there's an Easter egg with a larger display of the image above — a collage of figures from book 1 of Archimedes' treatise On Floating Bodies. You can see it here.

(Updated Oct. 31: Episteme now includes interactive eye candy, too.)

Epidoc transcoding transformer bats 1.000

2008-08-06T13:31:00.000-07:00

Hugh Cayless's transcoding transformer library (available from the Epidoc project's sourceforge site here) is indispensable for anyone working with ancient Greek texts in java or groovy. How reliable is it?

I decided to test it against two significant lists of unique Greek strings. For each list, I converted the TLG's beta code word to UTF-8, then converted the resulting UTF-8 back to beta code, and compared that result to the original. (For an overview of the TLG's beta code conventions, see this guide.)

The first list was composed of 858715 words excluding proper names. The transcoder round tripped to its starting point in 858709 cases. Six failures doesn't sound bad (99.999% success rate). But look more closely: in five of the six failures, the TLG entry in fact breaks the TLG's encoding rules about order of accents, breathings and iota subscripts, while the transcoder correctly follows the rules with the consequence that its conversion back to beta code actually corrects a data entry error in the TLG! The sixth case is a sequence found only in a papyrus fragment. The beta code series o(= should represent an omicron with rough breathing and circumflex – an accentuation that is not possible in Greek.

The second word list I tried was composed of proper names, including the tricky sequences beta code introduces in its conventions for capitalization. Out of 53167 capitalized words, the transcoder round tripped perfectly in all but one – again, an error in the TLG data entry that the transcoder corrected!

That's a total of 911882 unique strings. (That's going way beyond carefully chosen unit tests!) Remarkably, the transcoder had a 100% success rate in correctly formed words.

Half empty or half full?

2008-07-10T05:51:00.001-07:00

I frequently assert that classicists, along with biblical scholars, share the distinction of using logical citation schemes to refer to the works they study. This practice is important, since it means that references can apply to any version of a work, in print or digital form. (Briefly, in an earlier post.)

I have made this claim so often, that I decided it would be a good idea to find out if it were true.

The TLG offers the largest corpus of ancient Greek, so one way to evaluate how classicists cite their works would be simply to count and summarize the citation schemes used in the TLG. Sadly, athough this would have been possible until 2000 when the TLG distributed data to its licensees, there is in 2008 no way around the preconceived query interface of the TLG web site. (The fact that such a simple question as "what citation schemes are used?" is now out of reach illustrates the catastropic consequences for classical studies of the TLG's decision to reverse its decades-old policy of distributing data, in favor of selling access to predetermined user interfaces.)

As in an earlier post estimating the size of the surviving Greek corpus by period, we can still use the 2000 version of the TLG Canon distributed on the TLG E disk to get an impression of classicists' citation practice, however.

As in that post, we'll want to limit ourselves to works transmitted by manuscript copying. I'll take the simplest approach possible: count the number of "works" that use each citation scheme. I won't attempt to normalize in any way the definition of a work: the five-line Homeric Hymn to the Dioscuri is one work, as is the entire Iliad. With that caveat in mind, let's look at the results.

The TLG E canon includes 3810 works transmitted by manuscript and having defined citation schemes. (Note that the Canon includes works not in the E disk; 584 of these works did not yet have a defined citation scheme at the time of the E disk's publication, so I exclude them from our results.) These 3810 works are represented by an astonishing 194 distinct citation schemes!

As we might expect, however, the distribution of these schemes is very uneven: 104 citation schemes are used for a single work; only 16 citation schemes are used for more than 13 works. Let's look more closely at these top 16 citation schemes, which cover 3426 (90%) of the works surveyed.

Citation scheme	Number
volume/page/line	1014
section/line	710
page/line	517
line	348
chapter/section/line	334
stephanus page/section/line	114
book/chapter/section/line	75
jebb page/line	54
book/section/line	49
bekker page/line	44
kuehn volume/page/line	39
harduin page/section/line	32
epistle/section/line	25
chapter/line	25
epistle/line	23
scholion/line	23
Total physical schemes	1814 (53%)
Total logical schemes	1612 (47%)
Grand total	3426

The overall results are not encouraging. The entries in black are logical schemes: they total only 47% of the 3426 works. The entries in red refer instead to physical artifacts like book pages, 53% of the group. It's small consolation that the numbers are a worst-case scenario: some works may be cited by both logical and physical reference; where the TLG uses a logical reference, we can be sure that a logical scheme exists, but where the TLG uses a physical reference system, we can't always exclude the possibility that an alternative logical scheme is available. For example, the 44 works cited by Bekker page are, of course, the Aristotelian corpus: many of these have alternative citation schemes by chapter or section.

If we break the numbers down further by the chronological period of the original text, however, the picture changes. With the notable exception of Plato, where Stephanus' great edition became the standard for citation, citation by logical scheme is much more prevalent in works of the classical period. The following table breaks out from the previous listing works dating before about 300 BC.

**Citation schemes in works of classical date**
Total physical schemes	120 (23%)
section/line	229
line	98
bekker page/line	43
stephanus page/section/line	38
chapter/section/line	20
volume/page/line	18
page/line	16
book/chapter/section/line	11
fable/line	9
book/line	5
ode/line	4
book/section/line	4
tetralogy/section/line	3
demonstratio/line	3
epistle/section/line	3
book/demonstratio/line	2
thevenot page/line	2
epistle/line	2
idyll/line	1
page+column/line	1
sententia/line	1
lexical entry/line	1
proverb/line	1
folio/line	1
fable/version/line	1
exordium/section/line	1
usener page/line	1
Total logical schemes	399 (77%)
Grand total	519

The 519 works are cited in 27 different citation schemes. We could think of that as an "average density" of about 19-20 works per citation scheme, essentially the same as for the overall corpus (194 schemes for 3810 works is also a density of about 19-20 works per citation scheme). But in this listing, only 23% (120) of the classical works use physical reference systems. The corpora of Plato and Aristotle constitute the bulk of this material (81 works); apart from the two great philosophical corpora, only 39 works of the classical period are cited in the TLG by physical reference system – about 8%.

It's probably the height of political incorrectness to suggest that the most traditional canon of work has been the object of better quality scholarly study (although it's plausible enough that more scholarship should produce better results), but by the single, one-dimensional yardstick of how a work is cited, editors of classical texts have done a far better job capturing the logical structure of their texts than have editors of ancient Greek overall.

So for classicists interested in creating a digital corpus of Greek, the "news" is mixed. Roughly half the works in the TLG E Canon already depend on logical reference systems, so we already have a good standard in place for many of our texts. The classical period is in markedly better shape.

Citation schemes: empty content elements considered harmful

2008-04-11T14:53:00.000-07:00

Classicists have, by and large, relied on standard, logical citation schemes to cite works of ancient literature. In the scheme of the Functional Requirements for Bibliographic Records (or FRBR), we could say that classicists have cited notional works using references that could then be applied to any manifestation or expression of that work.

In the print world, this practice has made it possible for scholars to apply a reference to different printed editions or translations of a work. As the internet becomes our library, this practice can turn references into machine-actionable entry points to the library (whether the reference is automatically discovered, or manually cited by a scholars). It is therefore a vital prerequisite that digital editions encode standard, logical citation data such as the book/chapter/section divisions of Thucydides, or the book/line divisions of the Iliad.

The TEI Guidelines (as so often) offer more than one way to approach the problem. It is valid TEI to encode citation values as attributes on containing elements that define the logical structure of a document. Book/chapter/section in Thucydides might be represented by a successive hierarchy of TEI div elements, for example, or book/line in the Iliad by div elements containing l elements; the citation values could be placed in the @n attribute of each container.

Alternatively, since the earliest work of the TEI in the 1980s, the Guidelines have included empty elements (such as the milestone) that could be used to mark transitional points in a document. It is easy to find examples of scholarly texts using such empty elements to mark the beginning of a new unit like a chapter or section.

Arguably, there was little difference between these two approaches in SGML. In XML, however, scholars should avoid using empty elements to encode citation data.

A host of supporting and related technologies have developed around XML in its first decade. One of the most important is XPath, a notation for referring to parts of an XML document by the document's structure. Higher-level technologies such as XSLT or implementations of the DOM model in many programming languages in turn support XPath expressions. The result is that programmers working in many environments can succinctly retrieve a unit like "book 2, chapter 5" of Thucydides with a simple XPath expression like

/TEI.2/text/body/div[@type='book' and @n='2']/div[@type='chapter' and @n='5']

Content between empty elements, on the other hand, cannot be addressed directly with XPath expressions.

Placing citation data on empty elements cuts programmers off from a galaxy of technologies they can use when citation data is kept on containing elements. Empty citation elements should never be necessary if the citation scheme is in fact a logical hierarchy: if it is not, consider whether there is a problem either with your choice of citation scheme or with your design of the rest of the document's structure.

Separation of concerns applies to document content, too

2008-04-11T14:26:00.000-07:00

Twenty years ago, before the internet was open to the public, the print publishing industry was a leader in SGML document markup, and scholarly markup projects tended to think of "documents" as the content bound between a pair of covers. This heritage is clearly reflected in the TEI Guidelines' thorough inventory of elements to identify "front" and "back" material of documents, or a variety of groupings or collections of texts.

The major syntactic differences between XML and SGML — insistence on a single hierarchy of elements, each with explicitly marked end — were introduced in part to adapt markup to the needs of a very different environment: a network of computers exchanging information dynamically. The already well-understood distinction between semantic markup and presentational markup certainly contributed to the articulation of "separation of concerns" in the design of network applications. Individuals with different skills could apply appropriate technologies to the different parts of a network application, so in creating an application to run in a web browser, programmers might write the controlling code in javascript, and design specialists define its appearance with CSS. In a network of semantically structured content, XML plays the vital roles of defining the data structure (explicitly via a schema or DTD, or implicitly in the case of well-formed XML), and of providing a format for data exchange. The question of what this XML should look like — the kind of question the TEI has considered since the 1980s — had to be rethought. Humanists might rephrase Sun Microsystem's famous slogan, "The network is the computer," as "The network is the library."

When applications can exchange structured content, it is straightforward to create compound documents. Asymmetrically, it can be more difficult to disaggregate a complex document into component parts, since an application then needs a more detailed knowledge of the internal structure of a necessarily more complex document. An application could easily juxtapose a document in original language with a document in translation, or weave together a commentary with a text associated through a common citation system, for example, but disentangling interleaved translation or commentary from a complex document is more problematic.

I've been thinking about this in designing a set of TEI documents to represent the multiple texts of the famous Venetus A manuscript of the Iliad. There are four distinct sets of scholia, in addition to the manuscript's text of the Iliad. I chose to treat each set as an independent document, and as I am now reaching the stage of putting together applications drawing on those documents, I am glad that I did: cleanly separated, discrete documents are making that job much easier than it otherwise would be.

I expect that I will never use the elaborate TEI mechanisms to document the relation of a transcribed document to graphic images. In keeping with the guiding principle of separate, discrete documents, I'm associating images of the manuscript with ranges of text through external indices: here, too, the standoff markup of a separate, simple (non-TEI) document is easy to marshall together with the TEI document of the transcribed text.

In many ways, TEI P5, with its support for XML namespaces, is nudging scholars towards this kind of document organization. But we need to push harder: it's time to move away from monolithic TEI replicas of print or even manuscript sources. In editing scholarly texts for use on the internet, let each logical component stand alone.

Coordinating separate documents in a networked library requires a common understanding of how to cite them. I'll follow up with a note on how editors of TEI texts should think about that part of their markup.

The first thousand years of Greek

2008-03-05T07:22:00.000-08:00

How much Greek survives from the classical period? From the Hellenistic period?

Those questions were impossible to quantify when I was an undergraduate. It still might be difficult to get a very precise answer if we wanted to consider inscriptions and papyri, but if we limit ourselves to ancient Greek transmitted to us by manuscript copying, we can get a pretty satisfactory answer for the first thousand years or so of ancient Greek very quickly using the Canon from the TLG E disk.

The data in the Canon can be systematically manipulated using the Diogenes perl library. For each work in the TLG, the Canon contains three fields of information that are of special interest for this question: one indicates the method of transmission; another contains the word count of the TLG's on line text; and a third field contains a date description. The method of transmission is important, because the TLG includes "works" that are known only through testimonia or citation — "fragments," as classicists misleadingly call them — where we instead want to estimate how much Greek actually exists. (We don't care about geographic "fragments" of Hipparchus that are really passages of Strabo. To get an idea of how much of the TLG is made up of this doubling of content, the TLG E disk contains roughly 75-76 million words; over 4 million words — roughly 5% of the whole TLG E disk — are quoted "fragments" or testimonia!)

While it would be possible to write perl code to query the TLG Canon directly via the Diogenes API, most people would probably find it easier to transform the contents of the Canon into some format where they can use standard technologies. (I have created both a hierarchical XML version of the Canon, and a normalized relational database version; possible topics for another blog entry perhaps.)

The word counts are integer values; the methods of transmission are indicated by a controlled vocabulary: manuscript transmission is either 'Cod' or 'cod'. The only challenge is parsing the Canon's quasi-regular strings describing dates, but there are only a little over 100 unique strings, so scripting a little text munging in your favorite language that supports regular expressions is pretty straightforward.

The Canon's dates are to a precision of a century, so I interpret all dates as ranges. A date of "first century AD" could be interpreted as a range of 1-100 AD, and a date of "third or second century BC" could be interepreted as a range of 299 - 100 BC, for example.

At this point, it's time to let the computer do the counting. Here are some results to consider: through 300 AD, the TLG contains over 20 million words, but their chronological distribution is very uneven:

For works dated after or equal to...	... but before	Number of words	Running total
Earliest Greek writing	500 BC	384528	384528
500 BC	400 BC	2251766	2636294
400 BC	300 BC	1762944	4399238
300 BC	200 BC	921255	5320493
200 BC	100 BC	178655	5499148
100 BC	1 AD	1745320	7244468
1 AD	200 AD	7583759	14828227
200 AD	300 AD	5373095	20201322

Caveats

Roughly 10% of the contents of the TLG E corpus (7680878 words) have dates given as "INCERTUM" or "VARIA": these are completely omitted from the counts. We can't really know how Greek is distributed beyond the period of the TLG E Canon's coverage, because the TLG project no longer makes the Canon available, except through its "one-size-fits-all" interface (or to answer the questions raised here, "one size fits none"). This is the more troubling since the TLG's online corpus is now a third again as large as it was in 2000 when the E disk was prepared (by the estimate of the TLG website, 99 million words vs. 76 million words for the E disk).

Prediction confirmed: ubiquitous Diogenes

2008-02-22T05:14:00.000-08:00

In January, when I blogged a short note about running Diogenes on Linux PPC, I called Diogenes "ultra portable," and ended by asking, "Diogenes on your XO laptop, iPhone, or other device, anyone?"

Bruce Robertson has now answered. He emails, "I thought you'd like to know that I've been running diogenes quite happily on a Nokia N800 palmtop computer running OS2008 for the past two days."

Another great example of what happens when scholarship and software are open: other people can (in this instance, quite literally!) take it places the original author probably never imagined.

Scholarly markup in XML's second decade

2008-02-19T03:44:00.001-08:00

XML is now ten years old. (For those interested in an insider's view of how that all happened, Tim Bray has republished XML People.) For scholarly projects involving semantically structured texts, it is practically a given that they will rely on XML.

But in actual practice, texts produced by current projects often don't look very different from scholarship based on SGML in the 1980s. In the next postings on this blog, I want to discuss three suggestions based on my experience with XML over the last decade, and how it contrasts with my experience of SGML in the preceding decade. In each case, I'll focus on how to follow these suggestions using the Text Encoding Initiative's guidelines.

Separation of concerns applies to document content, too. (Now here.)

Citation schemes: empty content elements considered harmful (Now here.

What's the diff? Rethinking the critical apparatus.

Stay tuned.

Looking for an honest man — on Linux PPC

2008-01-09T07:45:00.001-08:00

Peter Heslin's Diogenes 3.1 is extremely cleanly designed, and ultra portable. The server functionality is written in perl, and the new user interface is a XUL application. One result is that Heslin can provide simple binary installations for Mac OS X, various Windows operating systems, and Linux on x86 architecture.

This design also makes it easy to install and run Diogenes on any operating system with perl and a XUL application environment. Using Ubuntu Linux 7.04 on a PPC system, for example, after you download and install the Linux version of Diogenes, you can run Diogenes at least three different ways:

1) use xulrunner to run the graphic interface

If xulrunner is not already installed on your system, use Synaptic or apt-get to install it. (xulrunner is in the Development section of the Ubuntu universe repository.) You can now start Diogenes from a terminal with the command

   xulrunner /usr/local/diogenes/application.ini

Better still, edit the properties for the Diogenes menu item that was created by the Diogenes installer. In the Launcher Properties, enter the command to start xulrunner as illustrated here. Now you can run diogenes from the menu selection.

2) use Firefox 3 to run the graphic interface

Version 3 of Firefox includes a full XUL environment that can run external XUL programs like Diogenes. Beta version 2 of FF3 was released in December; when a stable release version appears, look for it to show up as an upgrade to Firefox in your Ubuntu repository. When Firefox 3 is installed on your system, you may alternatively start Diogenes with the command

   firefox -app /usr/local/diogenes/application.ini

As with option 1, you can edit the Diogenes menu item to run this command.
Technically inclined users who are eager to play with the beta version can download source code for the beta release, and follow the very clear instructions here to install it. All the prerequisites are standard libraries that are available in Ubuntu repositories.

3) Browse and search texts from the command line

The command line user program (named dio) works just as it does on any other Linux. Run dio with no arguments to see its various options.

The importance of this flexibility is not that it opens up Diogenes to a vast number of Greek scholars using Linux PPC, Solaris, or some other particular operating system. Its importance is rather that it keeps Diogenes open to any platform meeting its simple requirements — including future platforms.

Diogenes on your XO laptop, iPhone, or other device, anyone?

Open access to federally funded research

2007-12-27T08:34:00.000-08:00

Linked from Slashdot today: tucked into the appropriations act just signed by President Bush, a requirement that the NIH must provide online access to research it has funded. This is a tremendous precedent, the first time that the US federal government has made open online access a condition of receiving federal funding for research.

The NIH is the focus, not the NEH, in part because people understand that medical research matters (as the respective budgets of the NIH and NEH also show). But the NIH was also in the Congressional spotlight because of the sustained advocacy of leading scientists, such as the open letter to Congress signed by 25 Nobel laureates in 2004 and by 26 Nobel winners in 2007.

Meanwhile, the American Philological Association, the professional organization that purports to represent classical studies, has inaugurated a multimillion dollar fundraising campaign to establish a "Digital Portal" centered on subscription-based access to a bibliography of print publications.

fuimus Troes, fuit Ilium

Zoom!

2007-12-27T07:17:00.000-08:00

In the Ur-web of the early 1990s, images came in fixed sizes. You might get a thumbnail-sized image, a smaller version or a larger version, but generally what appeared in your browser was a full, one-to-one view of a distinct image as it was delivered to you from a Web server.

Today, it's increasingly common for server- and client-side applications to manipulate what is, at least notionally, a single image that a user can navigate through. Google defined the current state of the art in browser-based image navigation when it introduced Google Maps in 2005. Its clever use of AJAX to load adjacent tiles at appropriate scales creates the illusion of continuous navigation of the whole earth.

The same technology can be applied to any image. At University College, London, the Centre for Advanced Spatial Analysis has developed "The Google Maps Image Cutter," an application to generate from any digital image the image tiles required by a Google maps-style web application.

A couple of projects I'm working on apply this technique to browse images that cannot be displayed in full detail in a single view because of their high resolution or awkward shape. The Center for Hellenic Studies' Homer Multitext Project has Google-mapped high-resolution photographs of Iliadic manuscripts. I've recently Google-mapped drawings and photographs of several dozen inscriptions in the Lycian language.

This is an easily implemented and effective way to let users explore an image. It comes at the cost of one tiny little white lie: we have to pretend to Google that the coordinate space of our rectangular image works like a Mercator projection of a spheroid (the earth).

This is innocent enough, if we recognize what we're doing, but it should provoke more serious reflection about how we use images and cite them in scholarly work. We need to define recognizable ways of referring to parts of an image independently of the state of a user's panning and zooming. I'll post more on that topic before long. For now, enjoy the pictures.

Vingt ans après

2007-12-11T04:36:00.000-08:00

Tonight, several of the Perseus project's original musketeers are gathering to observe the twentieth anniversary of the grant proposal to the Annenberg Foundation that jump-started the project. I'm sure that gray hair, sagging waist lines and altered career paths will prompt private reflections, but here's the fact that grabs me now: the Perseus project is older than three quarters of the undergraduates I teach.

My current students were still toddlers when the first public version of Perseus was released on CD. I doubt any of them have heard of, much less remember, Apple's HyperCard; it will be hard for them to imagine how exciting it was when a hypertext system first became available on personal computers.

They were learning to read or just beginning elementary school when Perseus made its astonishingly rapid transition to a Web delivery system. They probably are unaware that the internet was not always open to commercial use, and have little experience that would help them appreciate the importance of design decisions early in the history of Perseus. Can they grasp how the choice of SGML for markup of texts made it possible to generate both HyperCard stacks and Web pages from a single source?

Now they are in college, and the Perseus project has open-sourced both its code and key data including all its ancient texts (as I observed on Thanksgiving). Will they understand how this opens up to them unprecedented opportunities to build on the work of their predecessors, or have we conditioned them to see themselves only as passive consumers?

Are we raising up a new generation to join in the hard work ahead of us? All for one, and one for all!

Thanksgiving

2007-11-24T17:42:00.000-08:00

If you study ancient Greek, you can be thankful in 2007. This fall, two of our discipline's most important scholarly instruments have gone through extraordinary metatmorphoses. First, Peter Heslin released version 3 of Diogenes (http://www.dur.ac.uk/p.j.heslin/Software/Diogenes/); then this month, the Perseus project (http://www.perseus.tufts.edu/hopper)
announced that source code and text data are being made available under open licenses.

Diogenes now directly integrates automated morphological analyses of ancient Greek from the Perseus project's morphological parser. The Perseus project's new open licenses guarantee that Peter Heslin will not be the last scholar to draw on the rich resources created at Perseus over the past two decades.

Perhaps these developments would be unremarkable in disciplines where contributions through collaborative work and critical assessment of evidence are valued more highly than career advancement. In the humanities, they stand out against a bleak landscape of subscription services and other forms of restrictions on access to scholarly work.

Taken together, Diogenes and Perseus illustrate the kind of cross-pollination that is possible when reuse of digital scholarly works is not outlawed. If enough classicists notice, we may have more good Thanksgivings ahead of us in the future.

Remembering Ted Brunner

2007-11-01T05:38:00.000-07:00

This summer I read the Washington Post's lengthy obituary of Ted Brunner. Few classical scholars are made the subject of so many column inches in a national paper, so I was surprised this fall to discover that none of my Classics students knew who Ted Brunner was. The same quite serious majors who recognized the authors of eminently forgettable footnotes on Greek or Latin texts had apparently never heard of the director of one of the later twentieth century's most influential digital projects in the humanities. We classicists really have a lot of teaching to undo.
I leave it to others who knew Ted better than I to eulogize or analyze him. I offer only two observations from first-hand experience.
First, he remained always relentlessly focussed on data. The TLG was not about producing software: if you wanted software, Ted's attitude was that you should write your own. (Dinosaurs like myself will recall how far he could take this position. In the early years of the TLG, the project's at best arcane, in many ways bizarre data formats were almost aggressively undocumented: you got a nine-track tape, and if you wanted to understand the data, you were welcome to reverse-engineer the format as best you could.) In his own way, Ted Brunner was an early advocate of separation of concerns, and his view has been validated by the range of software developed over the past two decades for using the TLG's data. Most recently, Peter Heslin's release of version 3.x of Diogenes is a stunning piece of work (and deserves far more recognition than it has received). It integrates the TLG data with output from the Perseus project's morphological parser — a piece of software that in turn would probably never have been developed if the TLG had not existed. What a pity that since Ted's retirement the TLG has turned its back on this principle, and permits access to material digitized since 2000 only through its own, one-size-fits-all web interface.
Second, however sharply he could react to people he saw as threatening the TLG's work, he was extremely generous with his time to anyone interested in the TLG, no matter how unimportant. When I was a very lowly graduate student at Berkeley, I had a chance to visit the TLG project at Irvine, and Ted set aside an entire morning to give me a personal tour and answer my questions. (I am sure that I am not the only visitor to the TLG to come away with a vivid memory of Ted starting the standard pre-recorded TLG slide show and proudly pointing out that the narrator's incredible bass voice was none other than the voice of Tony the Tiger.)
So two small points — he focused on his data, and was generous to people who could not obviously or immediately help him.
I hope someone could remember as much about me after reading my obituary.
The Feast of All Saints, 2007.

Identifying discrete objects using the dnid scheme

2007-10-31T05:49:00.000-07:00

I've been working with Sebastian Heath at the American Numismatic Society on a scheme for referring to discrete objects with unique identifiers.
The fundamental idea is very simple. Since internet domain names already provide a means of uniquely identifying a namespace (think of XML namespaces), we can apply domain names as qualifiers to ensure the uniqueness of existing, stable IDs for many kinds of materials that humanists cite.
I'll have more to post on this topic in the future, but for now, you can see where we're headed at http://www.dnid-community.org.

Vitruvian design

2007-10-31T05:23:00.000-07:00

All people, not only architects, are able to appreciate what is good work. The difference between architects and uneducated people is that the uneducated person cannot understand what the work will be unless he has seen it completed; whereas the architect, as soon as he has built it in his mind, but before he has actually begun, has a complete vision of what kind of work it will be in respect to the elegance, the efficiency, and the correctness of its design.

That is my rendering of Vitruvius 6.8.10:

namque omnes homines, non solum architecti, quod est bonum, possunt probare, sed inter idiotas et eos hoc est discrimen, quod idiota, nisi factum viderit, non potest scire, quid sit futurum, architectus autem, simul animo constituerit, antequam inceperit, et venustate et usu et decore quale sit futurum, habet definitum.

Today we are desperately short of architects for scholarship in the era of the internet. In this blog I will occasionally comment on some of my own work, and on other digital scholarship in the humanities that, in my view, is contributing to the construction of a better edifice.