Combining photography and computed tomography to make three dimensional models of the Blaschka Glass Marine Invertebrates.

An increasing number of museums are using 3D imaging both for archival recording and to allow the global public to experience objects in collections that they would otherwise never be able to get close to. Photogrammetry, optical scanning and X-ray computed tomography scanning are all being used. These efforts benefit from steadily improving technology and abundant software options, and they build on extensive earlier work done primarily in archaeological, zoological and biomedical applications. Notable efforts are those at the Smithsonian Institution and the British Museum. However, rendering 3D images that are faithful to the original object remains difficult for many specimens. Fine detail or lack of detail, thin structures, hidden or glossy surfaces, and transparent or semi-transparent volumes are a few of the challenges. Intricate antique glasswork, while especially challenging for 3D imaging, also benefits tremendously from it because the specimens are typically very fragile; for conservation reasons, they should be handled as little as possible, if at all.

In a paper published in the journal Digital Applications in Archaeology and Cultural Heritage in September 2020, Peter Fried of the Department of Applied Physics at the New York University Tandon School of Engineering, Jonathan Woodward of the Museum of Comparative Zoology at Harvard University, David Brown of the Herbert F. Johnson Museum of Art at Cornell University, Drew Harvell of the Department of Ecology and Evolutionary Biology at Cornell University, and James Hanken, also of the Museum of Comparative Zoology at Harvard University, present the results of a project which for the last three years has been making 3D images of the intricate and beautiful glass models of Marine Invertebrates created between 1863 and 1890 by the father and son team of Leopold and Rudolf Blaschka.

The Blaschkas made thousands of these models, principally as teaching aids. They were shipped from the Blaschkas’ workshop in Dresden, Germany, to universities, schools and museums around the world. The Blaschkas made their models based on their own extensive observations of living Animals in aquaria and during several ocean voyages, and also based on drawings by contemporary taxonomists. The glass models of soft Marine Invertebrates were especially valuable because the shape and color of live specimens did not preserve well after death in fixatives used at that time. These models serve as a record of ocean life 100 or more years ago, and therefore are valuable to studies of evolution and the impacts of climate change.

Today, the principal collections of Blaschka Marine Invertebrate models are at Cornell University, the Harvard Museum of Comparative Zoology, the Corning Glass Museum, and University College, Dublin. Many smaller collections exist at institutions around the world. Fried et al. have imaged models at Cornell University and the Harvard Museum of Comparative Zoology.

Fried et al.'s imaging work began with glass models that are relatively simple in form and completely painted, so as to develop the basic photogrammetry workflow required, e.g. the minimum numbers of photographs and angles needed to yield high-resolution images. They then optimised control of the lighting, especially polarisation, using somewhat more complicated models that include sections of bare glass and/or glossy paint. Finally, there are some models with substantial transparent areas and/or intricate detail for which photogrammetry alone is inadequate. For these models, Fried et al. generated meshes with both photogrammetry and X-ray computed tomography scanning, which were then combined to create the final reconstruction.

 

European Squid, Loligo berthelotii. Model B-560 of the Cornell University Collection of Blaschka Marine Invertebrates (length about 6.4 cm). Fried et al. (2020).

The photogrammetry involved between 250 and 700 photographs of each glass model, which were taken against a black background on a turntable using 2–4 different camera angles. When possible, the model was arranged on the turntable in several different orientations to achieve 2–3 more or less orthogonal axes of rotation. A large number of photographs is needed both to insure good inter-photo registration, or 'alignment', and to insure adequate coverage of the entire complex geometry of the models. To further insure proper alignment, distinct coloured targets were mounted on the turntable, away from the glass model, near the edge of the field of view.

 

Fontaine’s Octopus, Robsonella fontanianus. Model SC-380 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates (spanning 13 cm from arm to arm). Fried et al. (2020).

Traditionally, specular reflections are minimized by applying a powder or spray coating to the subject. Such treatment, however, is not possible for the Blaschka models, especially in view of the fragile organic paints used to manufacture many of them. Controlling the polarisation of incident light by applying filters to the lens and/or the lights themselves can also be used to minimize (or maximize) specular reflections.

 

Stout Bobtail Squid, Rossia macrosoma. Model B-588 of the Cornell University Collection of Blaschka Marine Invertebrates (length about 15 cm). Fried et al. (2020).

The final position of the camera and lights was a compromise among the polarisation control and additional issues: The need for high resolution; the need to maximize relative depth of field; the need to minimise exposure time; and available resources, including lenses and controllable polarisers.

 

Sea Anemone, Phymactis pustulata. Model SC-51 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates (width about 10 cm). Fried et al. (2020).

For the 3D models illustrated here, Fried et al. used a 55 W, 1400- diameter ring light and a Nikon 7100 DSLR camera with an 80- or 90-mm macro lens. Lights and lens were located between one and three feet from the turntable. Apertures of f11–f20 were generally used. The camera, mounted on a firm tripod, was usually operated by remote control and typically set at a high ISO value (1600–2500). Excessive sensor noise can sometimes degrade inter-image alignment, but this did not appear to be the case at these ISO values. For some models, Fried et al were able to operate the turntable continuously at 1 rpm while making 1/50- or 1/25-second exposures every 0.5–1.0 seconds. However, crossed polarisers on the lens and ring light were used when necessary to control specular reflections, and this procedure required longer exposures (e.g. ¼–1/10 seconds) and, hence, manual stepping of the turntable.

 

Greater Argonaut, Argonauta argo. Model SC-363 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates (length about 7 cm). Fried et al. (2020).

Among the many different software packages currently available, Fried et al. used Agisoft Photoscan (recently updated and renamed Metashape). This software is affordable, completes the processing locally and has a good user interface that allows the user to control the separate processing steps in a manner suitable for this work. The software is used on a Dell Precision Mobile Workstation M4800 with 32 GB of RAM, an Intel Core i7-4910MQ Processor running at 2.9 GHz, and an NVIDIA Quadro K2100M Graphics card with a clock rate of 666 MHz.

 

Blue Button Jellyfish, Porpita porpita. Model SC-138 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates (diameter about 7 cm). Fried et al. (2020).

Key factors that affect processing time are the number of photos, the number of pixels in the unmasked portion of each photo, and the required accuracy level. The most challenging step in the photogrammetry, but not necessarily the most time-consuming, was always the inter-photo alignment. For several models, Fried et al. included alignment targets off the model at the edge of the turntable. These were useful as long as the model’s orientation on the turntable remained unchanged. To align photos of a given model in multiple orientations, in one case Fried et al. created separate partial digital models 'chunks' in the Agisoft nomenclature, for each orientation and then merged the chunks. In other cases, Fried et al. used the off-model alignment targets in one orientation and masked them in others to achieve a successful alignment. In any event, the software occasionally failed to align or created erroneous alignments due to accumulated small errors. Solving these cases involved various combinations of (a) adding more photos, (b) removing selected photos, (c) reordering photos, and (d) sequential re-alignments of subsets of photos. The highest-accuracy mode, which also is the most time-consuming, was usually required to successfully align the large number of photographs.

 

Football Sea Squirt, Diazona violacea. Model SC-418 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates (width about 9 cm). The high definition of the digital model results from a mesh made from a computerised tomography scan. This was aligned and merged in Agisoft Metashape with a mesh made from 699 photographs. The combined mesh was then textured from the photos. Fried et al. (2020).

There were some glass models for which even the techniques described above would not yield satisfactory results. Typically, these combine complex design with hidden or partially hidden surfaces, transparent or semi-transparent sections, and/or significant specular reflecting surfaces. For these glass models, Fried et al.  generated an X-ray computed-tomography scan that would be combined with a photogrammetry scan of the same specimen. 3D reconstruction from computed-tomography data has been extensively developed for metrology and biomedical applications.

 

Partial reconstructions of the Football Sea Squirt, Diazona violacea. (a) The reconstruction created by using only X-ray data, and (b) the reconstruction created by using only photographic data. Fried et al. (2020).

For computed-tomography scanning, Fried et al. mounted the glass model on an archival foam block, stabilizing it with Parafilm strips wrapped around archival cotton pads. We scanned the model using a Bruker Skyscan 1173 Micro-Computed Tomography Scanner, with a source voltage of 105 kV and a source current of 60 μA. Fried et al. interposed a 1.0 mm aluminum filter to reduce scatter artifacts in the final model.

 

Blaschka Sea Cucumber. Model SC174 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates. This is a roughly 7-inch glass model of a Sea Cucumber, Sporadipus tremula. The glass model is one of thousands done by the father-and-son team Rudolf and Leopold Blaschka in the late 1800s. Peter Fried/Sketchfab.

For reconstruction of the computed-tomography scan (creating slice images) and 3D model building, Fried et al. used NRecon and CTAn, respectively (each is part of the Bruker '3D.SUITE' software package). The slice images went through an initial thresholding step to separate the model from the background, a de-speckling step aimed at further reducing noise, and a model-creation step that resulted in an STL surface model file. Finally, in Meshlab, Fried et al. manually removed islands of artifact noise before downsampling (for ease of use) using Quadric Edge Collapse Decimation.

 

Blaschka Nudibranch, Red-Finger Aeolis, swimming. Model B-370, the Red-Finger Aeolis, Flabellina veruccosa, a Nudibranch, of the Cornell University Collection of Blaschka Marine Invertebrates. The Red-Finger Aeolis has been spotted in the Salish Sea (off Washington State). This is a roughly 4-inch glass model of a similar-sized Marine Invertebrate. The glass model is one of thousands done by the father-and-son team Rudolf and Leopold Blaschka in the late 1800s. Originally made as educational models, they are now in collections in over 50 institutions around the world. Peter Fried/Sketchfab.

The greatest challenges encountered during the micro-computed-tomography scan and reconstruction were attributable to the very thin and relatively radiotransparent glass of the Blaschka models. Several scanning attempts were required to achieve a set of slice images in which the model could be cleanly thresholded from the background. After even the best of these attempts, several levels of noise reduction were necessary.

 

Blaschka Vilella – By-the-wind Sailor. Model B-221, By-the-wind Sailor, of the Cornell University Collection of Blaschka Marine Invertebrates. The classification is: Phylum: Cnidaria, Class: Hydrozoa, Order: Anthoathecata, Family: Porpitidae, Genus: Vilella, species: vilella. By-the-wind Sailor is a free-floating colony of |Hydroids that, with a stiff upright 'sail', is blown about on the surface of the open ocean. They eat plankton and often are found stranded on the Northwest coasts of North America. This roughly 4-inch glass model is one of thousands done by the father-and-son team Rudolf and Leopold Blaschka in the late 1800s. Originally made as educational models, they are now in collections in over 50 institutions around the world. Major collections are at Cornell, the Corning Glass Museum, Harvard and University College, Dublin. The model was made from 310 photographs. Photogrammetry with Agisoft Photoscan and touch-up applied in Blender. Peter Fried/Sketchfab.

For some complex models, the mesh can be defined entirely by the X-ray computed-tomography data. Any surface coloring, however, such as that in the texture file, must be provided by photographic data. For other models, certain parts of the structure are made of paint and other filler materials, which are radiotransparent. These parts must be defined with photogrammetry meshes that are then merged with the X-ray-generated sections.

 

Blaschka Sea Anemone. Model SC64 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates. This is an approximately 3.5-inch glass model of a Sea Anemone, Tealia crassicornis var. purpurea. The glass model is one of thousands done by the father-and-son team Rudolf and Leopold Blaschka in the late 1800s. Originally made as educational models, they are now in collections in over 50 institutions including Cornell, the Corning Glass Museum, Harvard and University College, Dublin. The model was made from 312 photographs with processing by Agisoft Photoscan and touch-up in Blender. The white arms were a challenge, both due to their frosted, semi-transparent glass and also to the thick cross sections which hid many of the surfaces. Peter Fried/Sketchfab.

Some models required no additional work after the mesh and texture files were completed. Others, however, required post-processing, which was done with Blender. The 'Sculpt' mode in Blender provides tools for smoothing portions of the mesh, removing distortions and improving the definition of narrow crevices. Blender can also be used to divide the mesh into sections of different materials (defined by the.mtl files). This allows for different levels of transparency and translucence, albedo, roughness and surface gloss.

 

Blaschka Nudibranch. Model SC-308 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates. This is a glass model (length about 3.5 inches) of a Nudibranch (originally listed as Pterogasteron marginata, currently known as Elysia marginata). The Nudibrancia, an order in the subclass of sea slugs, include more than 3000 species, of widely varying appearance and colour. This model is one of thousands made by Rudolf and Leopold Blaschka as educational models in the late 1800s. The light-colored material on the bottom of the model is residual epoxy, used to affix a mounting wire for exhibit in decades past. The thin dorsal fins were a challenge to mesh, requiring 528 photographs shot in 7 orientations. The processing was done with Agisoft Photoscan and the surface was post-processed with Blender. Peter Fried/Sketchfab.

Once post-processing was complete the 3D files were uploaded to the SketchFab viewer where, after adding a background and appropriate lighting, Fried et al. adjusted surface qualities of the model. 

Fried et al.'s objective is to demonstrate archival capture of challenging 3D subjects, specifically glossy, translucent and highly detailed and delicate glass models. Much of their workflow optimises well-known techniques of light and polarisation control and photogrammetric processing. Significantly, we have also merged photogrammetry and computed-tomography data to create meshes of intricate structures made of multiple materials including glass.

 

Blaschka Cuttlefish. Model SC-369 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates. This glass model is one of thousands made by Rudolf and Leopold Blaschka in the late 1800s as educational models. This species of Cuttlefish (labeled by the Blaschkas as Sepia bisserialis, but currently known as Sepia elegans), never larger than about 3.5 inches, is found in the Mediterranean and in the Eastern Atlantic from Scotland in the north to the west coast of Africa in the south. The 3D model was made from 365 photographs using Agisoft Photoscan. In addition to glass the Blaschkas used other materials. For this Cuttlefish the dorsal surface was made of painted paper. This thin surface posed a challenge for the photogrammetry alignment and required careful masking of the photos prior to processing. Blender was used to assign four sets of material properties to the dorsal area, the remainder of the body, the tentacle and the eyes. Peter Fried/Sketchfab.

Recently, software and cameras have become available with automated focus stacking. This feature holds promise for some combination of higher resolution, larger effective depth of field and shorter individual exposures, but only if the processing load  for 250–700 separate photos is manageable.

 

Blaschka Sea Cucumber. Model SC168 of the Harvard Museum of Comparative Zoology Collection of Blaschka Marine Invertebrates. This is a roughly 5-inch glass model of a Sea Cucumber, Holothuria maculata. The glass model is one of thousands done by the father-and-son team Rudolf and Leopold Blaschka in the late 1800s. Originally made as educational models, they are now in collections in over 50 institutions around the world. Major collections are at Cornell, the Corning Glass Museum, Harvard and University College, Dublin. The 3D model was made from about 200 photographs taken at the Museum of Comparative Zoology. The photogrammetry processing was done with Agisoft Photoscan and minor touch-up was applied in Blender. Peter Fried/Sketchfab.

Future work should (a) identify ways to improve efficiency and resolution through focus stacking; (b) develop further use of computed-tomography scanning, especially for imaging highly transparent glassworks (e.g. Blaschka Jellyfish); and (c) explore the utility of voxel-based processing for digitising such objects. The use of these additional techniques, as well as those demonstrated so far, should expand the universe of objects suitable for 3D digitisation.

See also...

Follow Sciency Thoughts on Facebook.

Follow Sciency Thoughts on Twitter.