Looking Deeper
A Discussion about Imaging and Conservation

GEORGE BALLARD is president of GB Geotechnics, with offices in the United Kingdom, the United States, and Australia.

JOHN DELANEY is senior imaging scientist at the National Gallery of Art in Washington, DC.

DAVID SAUNDERS was formerly keeper of conservation and scientific research with the British Museum and is currently the inaugural Getty Rothschild Fellow.

They spoke with KAREN TRENTELMAN, GCI senior scientist, LORI WONG, GCI project specialist, and JEFFREY LEVIN, editor of Conservation Perspectives, The GCI Newsletter.

JEFFREY LEVIN: The word imaging can mean many different things. What does it mean to each of you?

DAVID SAUNDERS: I think of it as digital imaging. It's a visual mapping of information. We're dividing up these images into pixels—or voxels if they're three-dimensional—and assigning information to each of those subunits. If it's a color image we're using for documentation or for a presentation, then it's color information we assign to these. If we're using it in an analytical sense, we might assign spectral information to each point. Or we can use each point as a tag for a whole lot of other information about an object.

GEORGE BALLARD: For me, imaging constitutes the actions we humans take to transmit information to each other graphically. For example, if I were to describe a hole underneath a large building, I might use a gravity meter to determine the density of the ground and measure it carefully to find its shape and any variation in it. Then I'd present either a 2-D or a 3-D image, which is the information transfer. Imaging involves everything from analytic presentation of abstract information to transferring photographic images. Photography still exists. Even if it's digital, you're still drawing a representation via light.

JOHN DELANEY: Imaging is a thing we do—a process. But in a way, it's really the product. The visual mapping of information. The types of data sets we're looking at are multidimensional—there could be a time element, there could be a 3-D element, there could be an energy or spectral element. But it always relates back to spatial information. It includes the multidimensional information and spatial representation of a scene or an object.

BALLARD: You're transferring information among people. One difficulty I often have in engineering is expressing graphically some information or idea that I want to transfer to someone. They are all necessarily imaged and presented graphically. We often use photography extensively because we can look at things so much more efficiently. I can stand thirty meters away from a building with a long lens and a fifty-megapixel camera, and back at the office I can put all the images into a complete mosaic, rectified and processed. I can actually pretend at my desk that I'm two inches away from the structure—which is amazing.

LORI WONG: You actually may be able to see more. It's providing us with information that wasn't possible to get just a few years ago.

SAUNDERS: Yes, on one hand it's a question of resolution, but also the ability of current software to rectify the perspective. In the past, we would have struggled to stitch together all the individual images, even if we used a camera with a really good lens.

DELANEY: Having done a fair amount of high-resolution aerial photography work in my prior career, I'd note that a large high-resolution static image of a scene is both a copy of that scene and a temporal snapshot. We've gone beyond just creating a high-quality representation of a scene to capturing it as it evolves, which provides critical information. One thing that interests people in conservation is the deterioration rate of something, if measured with metrics that allow us to follow it so that we can intervene. Change detection is a commonly used process in remote sensing to find changes in the environment, in both urban and natural settings. A lot of those tools are coming into our field, and we can now try to visualize color loss in an art object over a short enough period of time to intervene—hopefully by collecting high-resolution spatial and color information, but in a temporal sense by aligning it in time.

BALLARD: That's comparable to looking at buildings. What we've found is that a stock of old photographs can be digitalized, stitched together, and processed in a way that allows us to use those as comparisons to study rates of deterioration. The old information is not lost.

SAUNDERS: This aspect of change detection was how I got involved in imaging. Back in the 1980s and 1990s, we measured color differences in paintings over time to detect change. Before that, we had to select a few points on the painting, and we might miss changes. The idea of making an image of a whole painting, and then redoing it a bit later, drove our research. It's interesting that John mentioned remote sensing, because at that time the people who had the money to invest in such systems were in remote sensing. On one of my first California visits, I went to the Jet Propulsion Laboratory to see how they used remote sensing to study changes in crops over time. We then applied the same technology to paintings.

TRENTELMAN: Since it's rare for instrumentation to be developed specifically for the study of cultural heritage, perhaps we could talk more about how technology has been adapted from other disciplines for conservation imaging, and where we might look in the future for new technologies?

SAUNDERS: We're getting a lot of technologies from medical imaging. A big drive in medical imaging that helped us is X-radiography, including both two-dimensional X-radiography where we're collapsing 3-D information into a two-dimensional image, and increasingly three-dimensional imaging using computer-aided tomography. And new multispectral CAT methods have been used to study mummified bodies and sealed objects you don't want to unwrap. Optical coherence tomography, also from medical imaging, looks at subsurface layers and has been adapted for the study of materials in ceramics, documents, and paintings.

WONG: The medical industry has also driven innovation in terms of portability and affordability, which has trickled down to our field, yielding things like handheld microscopes that we can put it our pockets and take into the field.

SAUNDERS: The big worry a few years ago was that we weren't going to be able to make X-rays in the future because manufacturers were scaling down their film production. This was because the medical industry was moving increasingly to digital imaging, but it did mean that a lot of money was put into developing lower-cost alternatives to film, which made digital technology affordable in museums.

DELANEY: What I see from doing research on nondestructive analysis with X-rays is that the requirements for nondestructive analysis of pipes and the like drove the requirements for the digital capture more significantly than medical needs had done for our field. The medical industry is quite happy with very low space resolution data. But that's not acceptable in nondestructive testing where you want to achieve film-quality resolution and have a dynamic range the medical industry doesn't require. So I see our greatest benefit coming from the nondestructive field looking at pipes for flaws and things like that.

BALLARD: Cross-pollination among disciplines will always be beneficial to a discipline that tends to be underfunded. But each discipline has a different set of resolutions it requires. I came to structural investigations through geophysics, which is all about understanding the center of the earth, where resolution in millimeters is patently not necessary. On the other hand, the techniques we developed for that, which I then brought into structural inspection, use derivations of seismics, derivations of radar, and so on, to look at the minutiae. I currently use a radar system originally developed for finding large objects in the ground with no specific size, for tracking microcracking in the concrete encasements of nuclear reactors. That activity is the result of the extensive work I've done on the conservation of stone, where I've looked for microcracking generated by corroding ironwork within the structures. Things go around and around as you find ways to use technology to good effect.

LEVIN: Lori mentioned the increasing portability of instrumentation. Can each of you talk about how that's changed the way we do things?

BALLARD: The ability to take a sophisticated scientific laboratory up onto the side of a building, hanging off some scaffold, is quite remarkable. The fact that you don't have to lug a couple of lead-acid batteries up with you, and that it can all be done with a set of AA cells, is fantastic. It's allowed me to continue my chosen profession late into my career!

DELANEY: The portability in the analytical chemistry field has been a big driver for a series of stand-alone devices used by nontechnical people in the field to make qualitative decisions. They're very tempting devices and are exciting to see. But there's also risk with some of these devices. The parameters under which they operate might be questionable with respect to the standards we have for examining objects, and they can create some confusion in interpretation. Getting more equipment to more people is beneficial, but the equipment also requires more care.

SAUNDERS: We're taking equipment once used only in the laboratory and making it more rugged and easier to use, and also reducing the power requirements so it can be taken out into the field. The desire for portability is driving lab-based research, in that we look at techniques with an eye to how they might be used in fieldwork. We generally begin by testing instrumentation not necessarily on the objects we see in the field but on analogous material, just to see how much information you get, so that when you are in front of the object you have confidence you're going to get sufficient data to make interpretations when you get back. You rarely have the luxury of undertaking data interpretation out in the field.

Photo: Tricia Zigmund. National Gallery of Art.
Imaging is a thing we do—a process. But in a way, it's really the product. The visual mapping of information.

LEVIN: Are you suggesting that these technologies, now that we have access to them, in some ways drive the kinds of research being done?

SAUNDERS: I don't think we should say, "Here's the set of tools we have—we'll use these and nothing else." The tools we have now we can use better, while still looking for the next technologies. We all have thoughts on what might be the next big lab-based piece of equipment, and what might be the next existing lab-based equipment that could be adapted for field work.

BALLARD: Is the great "intelligence" that's been put into instruments part of that move forward? I'm constantly switching off the automatic processing of my camera, which tells me how to take a better photograph.

SAUNDERS: I don't think we should throw away the opportunity to use the intelligence built into equipment, but we ought to be aware what it's doing. There are occasions where you want to turn off those automatic features and have the raw information coming out of the instrument.

DELANEY: There is a philosophy that any data you archive for future analysis should be in the raw form—unprocessed and uncorrected, that is. In the remote sensing and scientific fields, they want the raw data. But in your day-to-day work, you may not need it. One thing I worry about is that there are two levels of instrumentation. There are instruments that give qualitative answers rapidly, and then there is instrumentation that is traceable to some level of standards. That difference in quality makes a big difference for complex systems. Because of cost, we're tending toward using those less expensive instruments that may be insufficient. Unlike the scientific community, we don't have groups comparing instruments and rating them for quality.

BALLARD: We also don't have ratings of the operators either! Regarding raw data, I think we'd agree that storing raw data is an absolute necessity—so that you preserve the information in its most basic form—but at the same time, making it easier for the operator to interface with that data.

SAUNDERS: In storing raw data, having a point of reference is very important. Measurements made on reference materials are critical because when our future colleagues look at that information, they'll need to calibrate the data using the reference points. In fact, they may have better ways of using this information than we have now. That means we have to be careful how we archive and reference data. As for operators, we don't have standards. One of the great fallacies of digital imaging was that suddenly anyone could take an image. What we found within our institution was that the people who took the best digital images were the photographers, even if they weren't necessarily the people who understood the more complex aspects of digital technology.

DELANEY: My observation is that many people using digital cameras make the same mistakes. They don't have proper range of the camera to the object to optimize the spatial sampling, they don't quite know how to do frame averaging to reduce noise, and they don't know how to trade off the camera's f-stop with the amount of illumination needed.

TRENTELMAN: The first step toward managing large amounts of data is developing, and implementing, standards. Could you each comment on the state of the field regarding standardization in imaging?

SAUNDERS: There are a few manuals for specific types of imaging. I was involved in a European project that produced a handbook for imaging in the UV, visible, and infrared regions, which was applicable both to museum objects and for use in the field.

DELANEY: There was a recent European project, COSCH [Colour & Space in Cultural Heritage], which looked at 3-D imaging, instrumentation, and procedures and was supposed to give guidance on best practice.

BALLARD: The problem with best practice guides is that they tend to be written regarding a specific object and are somewhat inflexible. For example, there's a surveyors' guide to metric surveys, which is the standard form for most conservation. Its definition of representational accuracy is now being applied to ancient buildings. But as you move from general surveying into ancient building surveying, you need to shift your baseline of what representational accuracy actually means, to take full advantage of the digital opportunities given by the building information and modeling.

LEVIN: How has imaging affected your ability to communicate your work to colleagues?

DELANEY: In the gallery environment, new analytical imaging techniques have made more curators want to revisit art objects that were studied extremely well in the late 1980s and early 1990s with X-rays, infrared reflectography, and cross sections. They want to answer questions about the way in which the art objects were constructed and modified, and to test a lot of hypotheses that were generated from microsampling. And these images are directly accessible.

SAUNDERS: We work in a very visual profession, and the use of images plays to those strengths. Our colleagues in museums understand images, and we can use images to present information to them. Digital imaging allows you to produce so many more images very easily. Now that's a double-edged sword, because sometimes you can drown in this information. Nevertheless, it used to be that you might photograph a couple of details from an object when it was in the studio. Now you can image every detail you want, since you're never quite sure whether the detail you're interested in is the detail that will interest your colleagues. Having images to convey all those data to colleagues is very powerful.

Photo: Courtesy of George Ballard.
With new instruments, we'll always be pushing the boundaries of what we can find in terms of data to look at, analyze, interpret, and hypothesize about.

BALLARD: You take that a stage further with a 3-D object. You can paste all those multiple images together and turn them into a 3-D image on the screen. You can look at every single part and have the opportunity to choose your own point of interest.

DELANEY: There are a lot of connections that can be made about how material is distributed or was applied. The mapping ability, especially in the chemical domain, allows you to test that hypothesis. That's new, and it enables scholars to return to unanswered questions they've had. Along the way, you typically stumble across things because the earlier microsampling analysis didn't bring everything to light. It's serving a very powerful purpose of answering outstanding questions about an art object.

SAUNDERS: Yes, it's not just about having images—it's about having quality images. I recall going to curators in the past with an image and saying, "Look, this shows X and Y," and, quite understandably, they couldn't see it. Today, the images are so much better quality, and now when you say, "Look, there's a drawing underneath here," they get it.

DELANEY: I had a case where someone wanted to know if a hand on someone was painted on the ground layer or painted on the other person. The previous data was someone looking through a microscope and saying, "Through this crack I can see some ground, so it's painted on the ground." Well, multi- hyperspectral imagery showed that it was on the other person, and that was clear to the art historian. We can derive new information from the new results.

SAUNDERS: Paintings are incredibly heterogeneous systems. Across the surface of even a single area of a painting, there are changes in the paint thickness and underlying materials. Combining techniques has proven very useful—for example, making a multispectral scan and then using something like optical coherence tomography to look at the way that corresponds to the layer structure. The OCT image tells you about the structure without having to take a sample, and piecing the two together produces a better interpretation. Perhaps a smart instrument of the future might use those techniques in tandem.

BALLARD: If I am looking at some Roman tesserae, the material underneath, and, potentially, the structure below that, I'll use various techniques to get multiple images of the information on the layering of the structure, to which I'll then apply my knowledge of how to build a Roman floor—which would be very different from how one built a nineteenth-century Victorian tiled floor, even though they're both multilayered structures. Of course, intelligent input from knowledge is potentially vulnerable in the interpretation.

DELANEY: There's no doubt about that. Knowing when an art object was produced and knowing where it's from allow you to make a reasonable assessment about the paint layer structures. We can do that for very simple systems. But there is such subtlety in this human craft we call art work that those nuances would be hard to guess at. Sometimes that's where the creativity is—in the unique skill of the person creating the art.

BALLARD: The brain is a very good filter, but one should never forget that results are subject to the experience and knowledge of the filtering agent.

DELANEY: Well, yes and no. I'll go back to remote sensing, where you have different levels of exploitation of the data. You have people looking at stuff relatively quickly who may be seeking a quick estimate of changes in color. And then you have people trying to squeeze everything out of the data sets. When people do remote sensing of the environment, they typically get only about six or seven principal components from the hyperspectral reflectance image cubes. I've seen paintings where anywhere from twenty to forty principal components come out. There is a huge amount of data, and a lot of the questions asked don't fully mine it. You do need a lot of experience.

BALLARD: What you're getting to is how we go through that process.

DELANEY: I think it goes to what questions people are trying to answer.

BALLARD: And the process you're talking about is developing questions, putting up a hypothesis, testing that hypothesis with data, and sometimes realizing that you have three or four hypotheses that are pretty stupid. Before anybody got a telescope, the hypothesis was that the earth was flat. Then we got telescopes and could see things floating around in space. With new instruments, we'll always be pushing the boundaries of what we can find in terms of data to look at, analyze, interpret, and hypothesize about.

SAUNDERS: We shouldn't close our minds to new information that may come from these techniques, but if we're examining an object we must consider those things we already know about it that come from other solid scientific work. One could, theoretically, scan a painting, come out with five principal components, and assign them on the basis of some spectral library, irrespective of the knowledge that those materials would never occur in that object. Sometimes in the scientific literature you see such reports from people with an instrumental background who are unfamiliar with our field and not in day-to-day contact with paintings and painting materials.

TRENTELMAN: What are some of the other dangers with respect to reliance on imaging?

DELANEY: All these techniques are based on the idea that these materials can be spatially classified as being similar by their signature. That's generally the field of multispectral techniques. Microspectral techniques were brought in with the hope of getting at spectral signatures that are the quality of library data in order to do assignments. But sometimes you get to a point where you lack sufficient information to make an assignment. People can be too quick to say, "That sort of looks like that, so therefore it could be this." Still, a lot of these techniques work because they're better at classifying than we are with our eyes.

Photo: Rebecca Zamora, The Getty.
What's fascinating for me is that these methods often help us discover things about objects that align with written sources related to their making.

SAUNDERS: I always start by looking, and it's on that visual record that we base our subsequent imaging. I'm not saying we stop once we've looked at an object, but we put ourselves in a dangerous position sometimes when we don't start with careful observation.

LEVIN: How do the challenges of imaging built heritage differ from imaging objects in collections? Obviously, there are techniques used by both, but there are also some distinctions.

BALLARD: The biggest problem for built heritage is that its conservation is active, requiring a major rethink at least every twenty-five years, and some intervention every five years. Most of the information I collect is directed toward what to do next for a structure to stop it from deteriorating. We're fitting that to an imaging system that enables us to transfer information among different professions, because maintaining a structure will be partly its architectural surface and impact, partly its engineering stability, and partly its material robustness and durability, with each requiring different imaging sources. Conservation of buildings is very much controlled by maintenance budgets, and that imposes timing, finance, and understandability of the image that one produces—which is why I stress being able to transfer information from one person to the next as being the essence of imaging.

SAUNDERS: Many of the imaging issues faced in museums are similar to those in the built heritage environment, but on a much smaller scale. We sometimes struggle in a museum when dealing with a large object, but how much more you struggle when you've got a cathedral as the subject of your study. And then there is accessibility. Sometimes we curse because we can't take something off the wall. But for built heritage, it may actually be the wall you're interested in—or the floor. So it's not that we don't face similar challenges, but they're at a totally different level.

BALLARD: At one point I was charged with looking at a set of fantastic terracotta animals along the outside upper parapet of the Natural History Museum in London. The only access was via the drainage gutter behind the parapet wall, with me balancing one foot on the scaffolding and one foot on the sloping tilework on the roof. Now this was before digital photography, and I was there with my trusty Zenit, a grossly solid Russian SLR of the 1970s and 80s, and the only camera I found that I could drop forty feet from scaffolding, pick it up, and go back to taking photographs! That's the kind of environment we worked with. Now I can use a cherry picker and take a photograph twenty meters away that's far better than anything my Zenit ever managed. And I have a range of techniques that allow me to combine images together, accurately proportioned, derived from those photographic images, digitized, 3-D-ized, and rectified. Life has become a lot easier through that technology.

TRENTELMAN: We've talked about imaging technologies that have been developed over the past couple of decades. What sort of advances in imaging might we expect—or want—in the future?

SAUNDERS: Well, it's worth looking at the past. A lot of digital imaging started as replacements for photographic techniques we've used in the past, such as X-ray, visible, UV, and infrared photographs. Of course, many new techniques—multispectral techniques, terahertz imaging, X-ray fluorescence imaging, FTIR imaging, and acoustic technologies for buildings—don't have a history in photography. I suspect these will mature in coming years, and instrumentation might become cheaper and more available. And there may be a new generation of instruments using other wavelengths that we're not investigating now. We'll hopefully see a progression of the instruments we use in laboratories making their way into applications within museums and out into the field.

BALLARD: That parallels my feeling, coupled with the application of greater intelligence and being able to take computing power out into the field. In the 1960s and early 1970s when I first started doing fast Fourier transform analysis of the frequency content of data, it took me quite some time to make the paper tape punch cards, take them to the computing center, wait a week for results, and then analyze them. Now I'm doing the same process out in the field in real time. The ability to apply not only intelligent programming on-site, but also artificial intelligence, will play a part in the future. Artificial intelligence gives us the opportunity to turn dumb instruments into intelligent instruments, and that aids us in getting to answers faster. Still, it's a dangerous area, because you are handing a lot over to the machine, and you have to be certain that the artificial intelligence you create is the right sort.

DELANEY: Something we haven't talked about is the ability to consume large amounts of data quickly. There is a very high capture rate of those systems that add a temporal element to the dataset. When you see a video of an object as opposed to stills, things change in terms of how much you can glean about the object. The same is true if you get multispectral data as a multidimensional object is turning. There will be more video coming with a large number of spectral images—not just color—and the time required to collect all these modalities will lessen.

LEVIN: Imaging has vastly improved our material understanding of an object—but ultimately all these things are about creativity. How has imaging enlarged your understanding and appreciation of the creativity behind the things you've studied?

SAUNDERS: These techniques are immensely useful in understanding the artistic process and the choices made. What's fascinating for me is that these methods often help us discover things about objects that align with written sources related to their making. For instance, when you look at the surface of a fifteenth-century painting, it might not be obvious how it relates to written descriptions of painting production in that period. But when imaging lets you see the materials beneath the surface, it keys in nicely. And images can be very helpful in how we communicate this. There were a couple of exhibitions at the British Museum where we presented images showing the preparatory stages of drawings, and by looking beneath the surface you could find careful planning of the composition, at odds with the common view that these were spontaneous representations. That understanding of process and the artist's intent can be very powerful—and communicable to the public through those images.

DELANEY: One thing that's amazed me in looking at detailed registered multispectral infrared images of some artworks is the ability of an artist, at particular times, to start with something very well planned and then abandon it on the fly, making minute changes with great confidence—a process that is quite surprising. It's always delightful to see that.

BALLARD: That resonates with me, too. I've been particularly excited when I'm working on a Christopher Wren structure to recognize the way he comes up with yet another surprising solution to a problem in physics. It's the imaging that gets you to that, but it's a very personal appreciation of what he's done—for instance, how he solved the problem of the great piers in St. Paul's Cathedral, which was built on clay. He estimated that the cathedral would sink by twenty-two inches over the following hundred years. He was slightly off. It was about twenty-three inches! But he built the structure to accommodate that, with a sufficient margin that his error was inconsequential. And that's stunning.