The Center for Art/Science Studies (CASS) was established at the University of California, San Diego (UCSD) in 1974 in order to foster formal interactions between students and faculty in the arts with students and faculty in the sciences. The majority of these cross-disciplinary activities have involved members of the Schools of Visual Arts, Theater, Oceanography, Physics, Medicine, Chemistry, Space Sciences, and Applied Mechanics. In its early years CASS developed into a program of "science in the service art." Technologies such as isotopic analyses, holography, laser illumination and surface modification, digital-computer image processing, ultrasonic imaging, and magnetic resonance imaging (MRI) were applied to art in the areas of display, history, interpretation, performance, restoration, conservation and creation. For the most part the stimulus for developing such connections came from searching for ways of applying current scientific innovation to the arts. However, it was a surprising revelation to eventually realize that the artists were contributing to the sciences, as well. This reversal that we experienced has roots going back to the earliest civilizations and includes notable recent artistic contributions to general relativity, quantum electrodynamics, and cosmology. At the present time the activities of CASS are being carried on by the Center for Research in Computing for the Arts (CRCA).

Over the past years the progresses of electronic imaging have encouraged researchers to develop applications for the fine arts sector. In particular the aspects that have been mostly investigated have regarded, the high quality acquisition of paintings (both from the point of view of spatial resolution and of color calibration), the actual restoration of the works (for giving to restorers an aid to forecast the results of the tasks they choose), the virtual restoration (to try to build a digital copy of the painting as it was at the origin), and the diagnosis (to automatically highlights, evaluate and monitor the possible damages that a work has suffered). Partially related to image processing are also the technologies for 3D acquisition and modeling of statues. Finally particular care has been given recently also to the distribution of the digital copies of cultural heritage objects over the Internet, thus posing novel problems regarding the effective browsing of digital multimedia archives, and the protection of the Intellectual Property connected to art-works reproductions. The goal of this paper is to review the research results that have been obtained in the past in this field, and to present some problems that are still open and can represent a challenging research field for the future.

Optoelectronics and Information Science may usefully employed in conservation of artworks, providing a mutual benefit for the technology innovation in this field. In the Tuscany Region the Optocantieri project is pursuing this task, involving in a network a number of research centers, conservation institutions, restoration laboratories, high-technology producers and professional end-users. This article describes objectives, structure and preliminary results.

The measurement of the shape of an artwork usually requires a high-resolution instrumentation, in order to catch small details such as chisel marks, sculptural relieves, surface cracks, etc. 3D scanning techniques, together with new modeling software tools, allow a high fidelity reproduction of an artwork: these can be applied either to support and document its repair or for the realization of 3D archives and virtual museums. Starting from a high-resolution digital model of an object, a further step could be its reproduction by means of fast-prototyping techniques like stereo-lithography or electro-erosion. This work is aimed at showing the performance of a high-resolution laser scanner devoted to Cultural Heritage applications. The device is portable and very versatile, in order to allow in situ applications, accurate and reliable, so to capture intricate details. This laser profilometer has been used in a few surveys, the most significant of which are the monitoring the various phases of the restoration process of an ellenistic bronze (the Minerva of Arezzo, Florence), the cataloguing of some archaeological findings (from the Grotta della Poesia, Lecce) and the documenting of wooden panels surface conditions (the “Madonna del Cardellino” by Raffaello and “La Tebaide” by Beato Angelico).

Modern 3D scanning technologies allow to reconstruct 3D digital representations of Cultural Heritage artifacts in a semi-automatic way, characterized by very high accuracy and wealth of details. The availability of an accurate digital representation opens several possibilities of utilization to experts (restorers, archivists, museum curators), or to ordinary people (students, museum visitors). 3D scanned data are commonly used for the production of animations, interactive visualizations, or virtual reality applications. A much more exciting opportunity is to use these data in the restoration of Cultural Heritage artworks. The integration between 3D graphic and restoration represents an open research field where many new supporting tools are required; the David restoration project has given several starting points and guidelines to the definition and development of innovative solutions. Digital 3D models can be used in two different but not subsidiary modes: as an instrument for the execution of specific investigations and as a supporting media for the archival and integration of all the restoration-related information, gathered with the different studies and analysis performed on the artwork. In this paper we present some recent work done in the framework of the Michelangelo's David restoration project. A 3D model of the David was reconstructed by the Digital Michelangelo Project, using laser-based 3D scanning technology. We have developed some tools to make those data accessible and useful in the restoration. Preliminary results are reported here together with some directions for further research.

General concept of 3D data processing path, which enables to introduce information about shape and texture of real 3D objects into complex virtual worlds, is presented. Minimal requirements for input data, in the most common case coming in the form of cloud of (x,y,z) co-ordinate points from 3D shape measurement systems, are specified with special emphasis on implementation of multidirectional data and texture information. The algorithms for data pre-processing like filtering, smoothing and simplification are introduced. The techniques for merging of directional data into single virtual object are also employed. The algorithm for triangulation of merged cloud of points to form virtual object accepted by multimedia environments is presented. The various techniques of texture creation and mapping are introduced. All steps are illustrated by measurement and processing of a representative 3D object for art applications.

A high resolution Amplitude Modulated Laser Radar (AM-LR) sensor has recently been developed, aimed at accurately reconstructing 3D digital models of real targets, either single objects or complex scenes. The sensor sounding beam can be swept linearly across the object or circularly around it, by placing the object on a controlled rotating platform, enabling to obtain respectively linear and cylindrical range maps. Both amplitude and phase shift of the modulating wave of back-scattered light are collected and processed, providing respectively a shade-free, high resolution, photographic-like picture and accurate range data in the form of a range image. The resolution of range measurements depends mainly on the laser modulation frequency, provided that the power of the backscattered light reaching the detector is at least a few nW (current best performances are ~100 µm). The complete object surface can be reconstructed from the sampled points by using specifically developed software tools. The system has been successfully applied to scan different types of real surfaces (stone, wood, alloys, bones), with relevant applications in different fields, ranging from industrial machining to medical diagnostics, to vision in hostile environments. Examples of artwork reconstructed models (pottery, marble statues) are presented and the relevance of this technology for reverse engineering applied to cultural heritage conservation and restoration are discussed. Final 3D models can be passed to numeric control machines for rapid-prototyping, exported in standard formats for CAD/CAM purposes and made available on the Internet by adopting a virtual museum paradigm, thus possibly enabling specialists to perform remote inspections on high resolution digital reproductions of hardly accessible masterpieces.

The recent needs of visualization of variable in time 3D objects require development of new approach towards rapid 3D shape measurements. One of them applies colour structure light projection which utilizes combined multicolour phase-shifted and colour Gray codes. The usage of colour multiplexing enables to reduce the number of images required for shape measurement to one (slow gradients of shape) or three (high gradients) images and provides quasi real-time analysis. However the usage of projection and detection of colour patterns may produce significant errors due to noncorrect analysis and interpretation of images. These phenomena are shown at exemplary results of analyzing of real colour coded fringe patterns projected onto monocolour 3D objects. The main sources of errors connected with the use of a digital colour light projector and colour CCD camera (with single and 3 CCD matrixes) are discussed. The analysis of RGB and HSI color models are shown in reference to the system calibration procedure. Additionally the basic concept of system calibration and modification of images processing for the case of color 3D object analysis is put forward.

Imaging techniques are widely used for the diagnostics of paintings. The results, collected by applying various imaging techniques are generally compared by the art historian who studies the object. These 2D data can be effectively integrated to form a multi-dimensional dataset, and added to a 3D digital model of the painting, thus creating a complete package of information about the opera. In this work we present some examples of applications of data integration with measurements performed on some important panel paintings. The imaging analysis was carried out by applying techniques such as infrared reflectography, colour and false colour imaging and ultraviolet fluorescence. The IR reflectogram, the false colour and the colour image are simultaneously acquired with a scanning device characterized by a high resolution (16 dots/mm²), a high tonal dynamic and point-to-point correspondence between these three images. A multi-spectral device, based on a high-resolution CCD camera, is used for UV fluorescence. The 3D relief was obtained by means of scanning micro-profilometry with a quota resolution of about 1 micron. The integration of the 3D model with the results given by several imaging techniques greatly increases the information and it eases the analysis of the painting under investigation.

The study of the interaction of high power lasers with a solid-state matter is as old as the laser itself. The processes in which dominate ablation depend primarily on the laser power density and wavelength. Cleaning of surfaces of historical objects and artworks with pulsed laser radiation is an universal technique. However, it requires high precision in selection of parameters of pulsed laser radiation particularly for cleaning of unique artworks. The paper presents influence of laser radiation parameters on velocity of encrustation removal from artworks and heritage architectural objects. The influence of laser fluence, pulsewidth and pulse repetition rate on parameters of graphite layer ablated from aluminium substrate (temperature, pressure density and velocity of removal) is discussed on the basis of developed numerical model. In conclusion, selected experimental results of influence of laser wavelength and fluence on velocity of encrustation removal for different heritage objects are given as well as remarks on laser cleaning process are presented.

Infrared reflectography is a prominent optical technique for non-destructive diagnostics of paintings, which allows the visualisation of details hidden by the paint layers, because of their transparency characteristics to IR radiation. High-resolution reflectography was introduced around the end of the 80s by the Istituto Nazionale di Ottica Applicata, where a prototype of an innovative scanner device was developed. This technique was recently improved with the introduction of a new optical head, able to acquire simultaneously the reflectogram and the colour image, perfectly superimposing. The technical characteristics of the IR-colour scanner guarantee: a high spatial resolution (16 points/mm²), a high tonal dynamics (thousands of grey levels), a uniform lighting of the scanned area and the punctual superimposition of the colour and IR images. Moreover we can print distortion-free reflectograms, false-colour and colour images of paintings on a 1:1 scale. The quality of the acquired reflectogram is presently higher than that obtainable with any traditional detection system, like CCD or Vidicon cameras. The point-by-point comparison between the reflectogram and the colour image of the painting, along with digital processing of the recorded images, open new possibilities for the analysis of the reflectogram. Some examples of application to the study of ancient paintings are shown.

The goal of this paper is to present the research that has been carried out over the last 10 years in the Image Processing and Communications Lab of the University of Florence for developing applications for the cultural heritage field. In particular research has focused on the following issues: high resolution acquisition of paintings by means of mosaicing techniques, colour calibration of the acquisition devices, tools for forecasting the results of restoration processes (in particular with reference to the cleaning process), systems for producing virtually restored digital copies of paintings (in particular for filling in cracks and lacunas). The problems related to the distribution of the digital copies have also been considered, in particular with reference to the watermarking of the images for copyright protection. The methodologies developed by the Lab with reference to the above mentioned issues will be described, and the main results discussed.

We provide a unified framework of nonlinear vector techniques outputting the lowest ranked vector. The proposed framework constitutes a generalized filter class for multichannel signal processing. A new class of nonlinear selection filters are based on the robust order-statistic theory and the minimization of the weighted distance function to other input samples. The proposed method can be designed to perform a variety of filtering operations including previously developed filtering techniques such as vector median, basic vector directional filter, directional distance filter, weighted vector median filters and weighted directional filters. A wide range of filtering operations is guaranteed by the filter structure with two independent weight vectors for angular and distance domains of the vector space. In order to adapt the filter parameters to varying signal and noise statistics, we provide also the generalized optimization algorithms taking the advantage of the weighted median filters and the relationship between standard median filter and vector median filter. Thus, we can deal with both statistical and deterministic aspects of the filter design process. It will be shown that the proposed method holds the required properties such as the capability of modelling the underlying system in the application at hand, the robustness with respect to errors in the model of underlying system, the availability of the training procedure and finally, the simplicity of filter representation, analysis, design and implementation. Simulation studies also indicate that the new filters are computationally attractive and have excellent performance in environments corrupted by bit errors and impulsive noise.

The rapid growth of image archives increases the need for efficient and fast tools that can retrieve and search through large amount of visual data. In this paper we propose an efficient method of extracting the image color content, which serves as an image digital signature, allowing to efficiently index and retrieve the content of large, heterogeneous multimedia databases. We apply the proposed method for the retrieval of images from the WEBMUSEUM Internet database, containing the collection of fine art images and show that the new method of image color representation is robust to image distorsions caused by resizing and compression and can be incorporated into existing retrieval systems which exploit the information on color content in digital images.

SICAR is an internet-based system for the management of the information gathered during restoration analysis and intervention (alphanumeric and raster data), which is mapped and geo-referenced to 2D or 3D models. Joining the skills of a set of multidisciplinary partners, an integrated and innovative system will be developed, to permit to link to an accurate 2D or 3D model of the monument under restoration all the data gathered during the restoration analysis and intervention [raster images, document in TXT format, hypertext (HTML) or semi-structured text (XML)]. As far concerns semi-structured texts, a software module will automatically process documents, extracting their structure. SICAR will be developed to be completely accessible via web, for data entry and displaying, and will be suitable to wireless and intranet connection (GPRS, Bluetooth, etc.), in order to document the status of the objects to be restored, in restoration sites or laboratories. In this way data will be accessible even with low-speed connection.

The preservation and recovery of monuments are hotly debated topics in the field of cultural heritage conservation. In the early 1990's, our group in Siena started a study of the stone materials used in the architecture of Siena. The data were then processed by a GIS (Geographic Information System), which allows one to perform a series of interactive data analyses. An important example of the application of this methodology is the marble floor of the cathedral of Siena, consisting of 58 main scenes framed by decorations, for a total of 2500 m². The analysis involved: a petrographic study of the main lithotypes used in the scenes and of the various types of deterioration detected; realization of a full digital photo-image and of digital models for the reliefs; identification of anomalies beneath the floor by means of geo-radar and geo-electric instruments; monitoring of thermohygrometric conditions; mapping of the 22 stone varieties employed, their state of preservation (33 types of deterioration) and the previous restoration interventions.

Works of art exposed to excessive lighting suffer from color changes due to photo-oxidation or photo-reduction of the pigment/dye. Chromatic damage to artworks is not only the result of exposure to overly intense light, but can also be caused by the combined action of light with other environmental factors such as thermo-hygrometric variables and pollutants, which can amplify and accelerate photo-alteration. Therefore, the prevention of chromatic damage requires measurement of the equivalent-light dose, i.e., the dose capable by itself of producing a color variation, without the contribution of other environmental factors. The system presented in this work performs a continual measurement of the equivalent-light dose so as to indicate the potential for chromatic alteration of the exhibition environment. Experimental results are presented, that were obtained during continual measurements over 300 days, part of which at the Uffizi Gallery of Florence.

The basic methodologies used in animation are presented and their most significant problems connected with combining real and virtual worlds are recognised. Measured objects are classified into groups based on their mechanical properties and behaviour. The optical methods of shape and movement parameters determination adequate for fast virtual object generation are presented. Combination of various fringe projection techniques with photogrammetry is proposed to calculate the temporal shape and position of the object points. The spatio-temporal analysis of a projected fringe pattern delivers information about object shape and its out-of-plane deformation. The shape of measured object is calculated based on algorithm spatial carrier phase shifting technique. The analysis of marker dislocations during the measurement provides information about object shifts within the measurement volume. Combined information about object shape and its dislocation during the measurement enable to generate a virtual model that can be used to generate e.g.: realistic animation. The short comparison between proposed method and currently available systems is given and exemplary results of measurements are presented. The further directions of development of proposed techniques are presented.

The realization of an accurate 3D model of a building, a piece of architecture or a terrain has been a prerogative of the photogrammetric systems for a long time. However, recent developments in opto-electronic technology and 3D analysis software made the production of 3D models by laser scanning a practical proposition. The main advantages of laser scanners are accuracy and speed, allowing, thus, the collecting of data on a dense sampling of the object. For many architectural and industrial applications it is important to integrate the data acquired with different instruments, but a problem met with many commercial systems is the lack of compatibility with classic survey methodologies. Moreover, superimposition of results from different techniques is possible only if the output is metrically correct. This work is aimed at showing the results of some architectural and archaeological surveys realized by means of a 3D scanning device, based on the Time-Of-Flight (TOF) technology. The instrument, devoted to architectural applications in the Cultural Heritage field, was set up in order to provide the characteristics of reliability and compatibility to other systems. Such a device is composed by a high precision scanning system equipped with a commercial low-cost distance-meter.

A novel 3d reconstruction method for medical application has been applied for the examination and documentation of a 2000-year-old bog body. An ultra-fast pulsed holographic camera has been modified to allow imaging of the bog body from different views. Full-scale daylight copies of the master holograms give a detailed impressive three-dimensional view of the mummy and can be exhibited instead of the object. In combination with a rapid prototyping model (built by the Rapid Prototyping group of the Stiftung caesar, Bonn, Germany) derived from computer tomography (CT) data our results are an ideal basis for a future facial reconstruction.

Physical access to historic and artistic manufactures can be limited by a lot of factors. In particular, the access to the collection of the ancient coins is difficult, especially for students. Indeed, for coins digital archive of high quality three-dimensional model and remote fruition is of great interest. In this work we propose 3D acquisition and digitizing techniques for the virtualized reality of ancient coins (virtual gallery). The system used for creating 3D shape of coins is based on conoscopic holography. This technique is a non-contact three-dimensional measuring technique that makes possible to produce holograms, even with incoherent light, with fringe periods that can be measured precisely to determine the exact distance to the point measured. It is suitable to obtain 3D profile with high resolution also on surface with unevenness reflectivity (this situation is usual on the surface of the ancient coins). By conoscopic holography, high-resolution 3D model can be obtained. However, accurate representation and high-quality display are fundamental requirements to avoid misinterpretation of the data. Therefore, virtual galleries can be obtained through a procedure involving 3D acquisition, 3D model and visualization. In conclusion, we propose an optoelectronic application, integrated with multimedia techniques, in order to improve the access to collection of ancient coins belonging to museums or privates.

In the presented preliminary experiments, a simple glass cup enables to isolate the cleaned surface and to introduce a laser beam and a measuring probe. A sensitive opto-acoustic measurement system was able to detect the time averaged changes of concentrations of benzene, carbon monoxide, carbon dioxide, methane, sulphur dioxide and water vapours at the level of ppm. Final concentrations of carbon monoxide and sulphur dioxide (well known as main pollutants of the atmosphere) can significantly increase during laser cleaning in closed areas. Moreover, concentrations of measured atmosphere contaminants can be considerably different in the case of ablation of encrustation on different substrates (wood, stone, plaster, parchment etc.). The measured values of gas concentrations emitted during laser cleaning showed large changes from point to point, depending on local properties of soiling, its quantity and type.

Laser radiation is often used in cleaning and conservation of artworks. Interaction of laser radiation with matter is so sophisticated process that analytical solutions rarely bring the valuable formulas. Even numerical methods seldom give quantitative insight into the physics of processes. Chemical and physical properties of surface impurity layers may change from point to point within the same sample. Absorption coefficient of the layers depends on such factors as weather or air humidity. In spite of this, theoretical description of laser cleaning is necessary, as it allows to explain some characteristic features of processes under investigation. In this work we present a model for laser pulse interaction with graphite layer placed on aluminium substrate. The model is limited to one dimensional hydrodynamic equations. The following phenomena are included into the model: absorption and reflection of laser radiation, heat conductivity, radiation transport in grey body approximation, shock waves, ionisation and the elastic properties of the media. Numerical calculations and experimental results give a qualitative agreement.

The work begins a short presentation of laser ablation method, together with exemplary results of artwork cleaning. Next parts show the application of dynamic infrared thermography for study thermal phenomenon induced by pulse laser radiation of various fluency and time duration. These investigations focused only on the results of irradiation, as both too short time and complexity of the ablation phenomena had made unable all measurements of pure ablation properties. Thermal data about pre- and post-ablation state were delivered by infrared thermographic system in the form of sequences of thermograms. Information of interest occurred on small parts of these images only, and study of thermal movies or typical graphs showed to be inconvenient. The paper presents applied transformation of every sequence to single synthesized images: temporal-spatial thermograms of the isolate fields only. Numerous examples of these images, named as field dynamic thermograms, presented for various laser ablation experiments prove the great informing potential of modern thermography and in particular -- signal processing for research and cleaning procedures optimizing works. Mentioned method of extracting useful information for dynamic of thermal distributions visualization can be of interest for other applications.

The knowledge of the shape of an artwork is an important element for its study and conservation. When dealing with a statue, roughness measurement is a very useful contribution to document its surface conditions, to assess either changes due to restoration intervention or surface decays due to wearing agents, and to monitor its time-evolution in terms of shape variations. In this work we present the preliminary results of the statistical analysis carried out on acquired data relative to six areas of the Michelangelo’s David marble statue, representative of differently degraded surfaces. Determination of the roughness and its relative characteristic wavelength is shown.

The smoothing function of widely used vector filters such as vector median (VMF), basic vector directional filter (BVDF) and directional distance filter (DDF) is designed to perform the fixed amount of smoothing. It may become the undesired property, because in some image areas these filters introduce too much smoothing and blur thin details and image edges. In general, the common problem is how to preserve some desired signal features while the noise elements are removed. An optimal situation would arise if the filter could be designed so that the desired features were invariant to the filtering operation and only noise would be affected. In case of the impulsive noise corruption, the problem is stated often as searching for the switching function that allows to reduce the filter effect only to noisy samples. In this paper, a new nonlinear filtering scheme for the removal of impulsive noise in multichannel digital images is presented. A new class of multichannel sigma filters is based on the combination of the standard sigma-filter concept provided by Lee and the robust order-statistics theory. With respect to a variety of the measures (e.g. vector distance expressed through Minkowski metric, angular distance or their combination) for quantification of the distance between multichannel samples, we provide a rich class of adaptive vector sigma filters taking advantages of the threshold structure with the approximation of the standard deviation and also the fully adaptive filter structure. Thus, by adaptive switching between the smoothing function and the identity operation, the behavior of the proposed method is attractive for filtering of image environments degraded by impulsive noise, bit errors and outliers. The new filtering scheme is computationally efficient and able to achieve excellent balance between the image detail preservation and the noise suppression. The achieved results show that the new filtering class has excellent preservation capabilities and provides significant improvement in comparison with well-known vector filters such as VMF, BVDF and DDF in terms of all commonly used quality measures.

In this paper we address the problem of impulsive noise reduction in multichannel images. A new class of filters for noise attenuation is introduced and its relationship with commonly used filtering techniques is investigated. The computational complexity of the new filter is significantly lower than that of the Vector Median Filter, (VMF). Extensive simulation experiments indicate that the new filter outperforms the VMF, as well as other techniques currently used to eliminate impulsive noise in color images.