In the past, museums could gauge the scholarly value of collection catalogues through reviews, citations, and awards. For online publications, these metrics are still a work in progress. Most scholarly periodicals still do not routinely review online publications, and the humanities have not yet developed easy ways to track online citations.
With printed scholarly collection catalogues, publishers knew how many catalogues were sold, but there were few methods to determine how they were being used or by whom. Electronic publication offers the opportunity to better understand the audiences for scholarly art historical publications online. Analysis of Web statistics, such as number of visitors, number of pages viewed, length of time spent on the site, entry points, and the geographic distribution of the users, helps museum staff to understand the impact of their work.
User studies of the SFMOMA and Walker OSCI catalogues showed that:
55%
Of catalogue readers were from outside the U.S.
15%
Visited the catalogues more than 9 times
75%
Did not enter through the homepage
Tate tracked over 28,000 visits to the Camden Town Group OSCI catalogue in its first year after launch. The team also received specific feedback from academic users, who integrated the catalogue into their own research and teaching. Visit data for SFMOMA’s OSCI catalogue collected over an eleven-month period revealed that the top two pages of artwork on its entire site are both contained in the Rauschenberg Research Project. The same data also confirmed that less than five percent of their catalogue’s users visited the publication’s home page.
The Art Institute of Chicago decided to track usage of its OSCI catalogues by user type. By segmenting out their users into three categories, using the domain of their Web browser—general, academic, and in-house—they were able to see that the average session visit nearly tripled when narrowing their audience from general down to identified academic users visiting from an .edu Web domain. Analytics also allow the Art Institute to track usage of its citation tool for all digital catalogues (current data shows the tool has been used more than 700 times), but they do not have a record of where these citations appeared.
Additionally, three of the OSCI museums (LACMA, SFMOMA, and the Walker) have completed user evaluations, and the Art Institute plans to evaluate its catalogues in fall 2017. LACMA commissioned a usability/user experience study from the firm Creative Pursuits for its Southeast Asian art catalogue. Usability studies are particularly important for digital projects given how easy it is for visitors to leave a website if they encounter any difficulties. All it takes is a slow data load or confusing navigation, and a user will move on to look for a different resource to meet their needs. The study helped LACMA identify and prioritize updates to the catalogue’s navigation, information architecture, design, and performance.
SFMOMA and the Walker Art Center commissioned evaluations from the firm Frankly, Green + Webb. Both surveys underscored that the OSCI museums were truly diving into unchartered waters with digital publishing and that their online catalogues represented a new form of scholarship that brings both rewards and challenges. First, the good news: the OSCI catalogues of both institutions are reaching large, diverse audiences. For example, more than 55% of the Walker’s Web traffic for its inaugural OSCI publication is from outside of the United States. The core intended audience of scholars also rated the content of the catalogues very highly and indicated that they were “trusted” resources that could be cited. In fact, surveys suggest that the prestige of the authors mattered more to these users than peer review in terms of intellectual credibility.
Both evaluations also pointed to the ongoing challenges of digital publishing. Unlike the traditional catalogue experience that starts with opening the cover, 75% of the users of SFMOMA and Walker’s catalogues do not enter these publications through the homepage. This makes clear navigational elements and digital wayfinding especially critical. The surveys also confirmed what the OSCI museums reported anecdotally: discoverability—finding the catalogues through search engines—is a challenge. Museums needs to consider an ongoing program of communication to build traffic and awareness of these resources.