Applications of Geo-Referenced Underwater Photo Mosaics in Marine

Author Posting. © Oceanography Society, 2007. This article is posted here by permission of Oceanography Society for personal use, not for redistribution. The definitive version was published in Oceanography 20, 4 (2007): 140-149.

Successful applications of photo mosaicking include archaeological imaging by Ballard et al. (2000) and Singh et al. (2004a) as well as marine biological photo mosaics for species inventory (Singh et al., 2004b) and spectral image analysis to map the abundance of bacterial mats (Jerosh et al., 2007).
Today, high-resolution data can be used to generate a complete overview of a seascape and details of organisms, substrate, and artifacts.Likewise, high-resolution images can be used to

Methods ANd MAteRiAl s Underwater Vehicles
The work at the marine biology site was carried out by an electric observation remotely operated vehicle (ROV; Sperre AS Sub-Fighter 7500; Figure 1 The WROV was moved in x and y directions by rack rails and gear motors, as described by Søreide and Jasinski (2005).

Camera and light Configuration
In both surveys, constant lighting from two 400-W HID (High Intensity Discharge; OSRAM HMI) lights were used.One wide-beam light source was placed on either side of the camera to illuminate the image-frame area as evenly as possible.On the observation ROV, the lights were mounted on booms to increase the lateral distance between light source and camera, with the aim of reducing backscatter light to the camera (see Figure 1).The lateral distance between the camera and light source was approximately 0.8 m.On the WROV, a lateral separation of light sources and camera of 0.7 m was obtained without light booms.
During the marine biological survey, the camera was mounted on the underwater vehicle with a horizontal image axis (i.e., aiming orthogonally at the rock wall).For the marine archaeological survey, the camera pointed downward, again imaging its target, the seabed, orthogonally.The camera used in both projects was a 12-bit dynamic range camera (Uniqvision) that captures one image every four seconds.

Marine Biological Survey
During data acquisition for the marine biological study, the ROV followed horizontal paths along a vertical wall, with each path one meter shallower than the previous one, so that the rock wall was  Once the interest points are established, a Zernike moment is calculated for each interest point to provide it with a unique identity or tag.The Zernike moment (A nm ) can be calculated according to the following equation: More details on Zernike moments can be found in Khotanzad and Hong (1990).

Re sUlts Marine Biological example
The area shown in Figure 3  The horizontal extent of the area shown in Figure 4 is approximately 18 m and the vertical extent is about 8 m.The relative to the next image can be calculated (Pizarro, 2003).
For the data set acquired within the excavation support frame at the marine archaeology site, the processing was extended to include three-dimensional processing by means of navigation data.The three-dimensional positions of interest points were calculated to obtain a detailed bathymetrical model of the site.The method is described in Ludvigsen et al. (2006).

Image Blending
To make the mosaic appear seamless, it is necessary to avoid steps in image intensity on the borders between the images.
After the topology has been estimated, intensity steps are reduced by merging the images in a process called blend-ing (based on Burt and Adelson, 1983).
In the first step of the blending process,

Marine Archaeological example
The photo mosaic in Figure 5 shows  (Kirk, 1994).

Geometrical Distortions and Corrections
In the marine biological example, the extents of the subjects along the image axis were large.This causes geometrical distortions when the three-dimensional topography is projected to two dimensions in the resulting photo mosaics.
In Figures 3 and 4, the distortions can be seen as varying sizes of a feature, for example, the giant file clams.The low vertical relief in the marine archaeological example is beneficial, but some geometrical errors are nevertheless seen along the keel line of the shipwreck in (e.g., Heikkila and Silven, 1997;Zhang, 1999).To further enhance the geometrical consistency of photo mosaics, mosaic processing and image matching should be done in three dimensions so that a fully three-dimensional scene can be extracted (Pizarro, 2003).This will require instrumentation on the underwater vehicle to measure the position and orientation of the camera for each image.Three-dimensional processing can be used to document the bathymetry of the site in high resolution.But, to reduce the geometrical distortions of a two-dimensional photo mosaic, the three-dimensional scene could be used to reproject the images into a twodimensional scene.

Quality Control
To increase the integrity of the information in photo mosaics, a procedure for quality control should be established.
The simplest way to do this would be to physically place a linear scale bar in the imaged area and compare the dimension and shape of the scale bar with the known values.
Measuring the quality of the topology estimate represents a more sophisticated quality control (Pizarro, 2003) (Fitzgibbon and Zisserman, 1998).

Alternative survey Methods
Other high-resolution documentation methods are available for biological and archaeological research surveys.
Laser line scanners, available since the early 1970s (Funk et al., 1972), are effective and produce clear images, but the resulting imagery is monochrome and less detailed than images from optical cameras.Interesting for biologists, the laser line scanner can be used to record fluorescence to measure, for example, the presence of chlorophyll at a site (Jaffe et al., 2001).The most important disadvantages of laser line scanners are, however, complexity, availability, and price.Video is the most likely alternative to photo mosaics.To cover an area larger than one video frame, the camera is moved and a virtual map is constructed in the mind of the viewer.The motion of the video camera can enhance understanding of a recorded object, as the object can be seen from different angles.
But video is more complicated to communicate and present than a photo mosaic and the virtual map can vary between viewers.

CoNClUsioN
Detailed site investigations may become more important in the future.Surveys like the biological example presented here can be useful for documenting damage caused by bottom trawling and other fishing methods (Fosså et al., 2002).Stony corals may become useful as climate markers because a climateinduced reduction of pH levels in the ocean is likely to degrade their aragonite skeletons (Sinclair et al., 2006).Photo- photo mosaics were produced by physically stitching images together to create new images from the arrangements of individual picture frames.One of the first examples of underwater photo mosaics in the literature shows the sunken submarine Thresher (Ballard, is among the most famous examples.As technology progressed, specialized hardware emerged, combining two images into a new image using slides and variable projection planes.Development of the computer and digital image processing gradually led to digital production of image mosaics in the 1990s.Then, the scientific discipline of computer vision emerged, focusing on automatic information extraction from imagery.Many of the most common image mosaicking schemes were developed in a subdiscipline of computer vision called SLAM (Simultaneous Localization generate three-dimensional images of objects of interest.Geo-localized and high-resolution images are important for revisits to a given site or object of interest and are very important for obtaining time series (e.g., for process-oriented and climate-related research in marine biology).We present two case examples to elucidate applications of photo mosaics in marine biology and marine archaeology.The marine biological example is part of a habitat mapping survey in the fjord of Trondheim at Stokkbergneset (63°28´N, 9°54´E), approximately 20 km west of the city of Trondheim, Norway.The site comprises a vertical rock wall rising from 530 m to 200 m depth.The objective of the survey was to map the characteristics of benthic filter feeder communities dominated by stony corals, horny corals, and bivalves attached to the vertical wall at 250-450 m depth.The archaeology photo mosaic example was made during an investigation the of the pipe line routes for the Ormen Lange gas field off Aukra (62°4´N, 6°54´E) in the Norwegian west coast.Norwegian cultural heritage authorities requested a marine archaeological survey of the pipeline routes for the gas field, which led to the discovery of a historical shipwreck close to the pipeline route at 170 m depth.The field developer was ordered to conduct a detailed investigation of the site (Søreide and Jasinski, 2005).Two of the photo mosaics resulting from this detailed investigation are presented in this study.The first covers the whole wreck site, while the second shows a condensed area at the stern of the wreck where sediment was removed to uncover historical artifacts.
figure 1. Configuration of the remotely operated vehicle Minerva used for image acquisition in a marine biology survey on the vertical rock wall of the fjord of trondheim, Norway.The doppler velocity log, high-intensity discharge lights, light booms, high dynamic range camera, and video cameras are all pointing forward.
survey track design Optimal image overlap, sidelap, seabed resolution, and acquisition efficiency are essential to mosaic quality.They are achieved by carefully choosing proper altitude, line spacing, and velocity when planning the survey.The term "over-MARtiN lUdViGseN (martinl@ntnu.no) is Ph.D. candidate, Department of Marine Technology, Norwegian University of Science and Technology, Trondheim, Norway.BjøRN soRtlANd is Associate Professor, Department of Marine Technology, Norwegian University of Science and Technology, Trondheim, Norway.GeiR johNseN is Professor, Department of Biology, Norwegian University of Science and Technology, Trondheim, Norway.hANUMANt siNGh is Associate Scientist, Applied Ocean Physics and Engineering Department, Woods Hole Oceanographic Institution, Woods Hole, MA, USA.

figure 2 .
figure 2. The excavation support frame placed over the aft part of a shipwreck during a marine archaeology survey in the ormen lange gas field off Aukra on the Norwegian west coast.The frame, leveled by jacks in each corner, is used to stabilize the workclass RoV during image acquisition.Figure provided by Fredrik Søreide figure 3. Photo mosaic with different taxa of benthic filter feeders growing on a vertical wall at 430 m depth in the fjord of trondheim, Norway.living stony corals appear white.dead stony corals appear grey due to loss of antibacterial mucus production and consequent silt sedimentation.The area covered by the photo mosaic in the left image is approximately 2 m x 5 m.

Figure 3
Figure 3 is made from nine images, while the photo mosaic in Figure 4 is constructed from 73 images.
1) where f[x,y] represents an image patch surrounding the interest point, V * nm represents the Zernike polynomial for the image patch, n is the order of the polynomial, and m is the number of repetitions for deriving the Zernike polynomial.The Zernike moment is a dependant of the image patch; hence, image patches showing the same feature will have similar Zernike moments.
By requiring the spatially coincident interest points to match, the relative position, rotation, and scale of an image final step of the blending procedure.By choosing a narrow transition zone for the high-frequency components, and a wider transition zone for the lowerfrequency components, double appearance of smaller objects is avoided, while the larger structures in the initial image set are merged smoothly; the resulting mosaic thus appears seamless.
photo mosaic shows a benthic filterfeeder community comprised of benthic cold-water corals, bivalves, sponges, and numerous other taxa.Abundant horny corals like the Paragorgia arborea and Paramuricia placomus are attached to the exposed rock wall.The dominant species are the stony coral Lophelia pertusa and the European giant file clam Acesta excavata.The bivalve appears in large numbers, found in high density on the rock wall beneath the hanging stony coral branches.Both living and dead stony corals are visible.Mucus containing antibiotic compounds, which also removes sediment and particles (Mortensen, 2000), makes living stony corals look "clean and white."Dead stony coral, in which mucus production has ceased, is grey and brown due to sediment accumulation, microfauna attachment, and invasion by boring sponges.
figure 6. detailed photo mosaic made inside the excavation support frame seen in figure 2. Part (a) illustrates how the data set can be processed in three dimensions to measure the topography of the site.insets (b), (c), and (d) demonstrate how the mosaic can be zoomed to reveal details of the artifacts observed.The photo mosaic is compiled from 270 images.
photo mosaic can be magnified to reveal details of the site.The photo mosaic shows exposed wooden hull parts, bottles, bottle necks sticking out of the sand, dishes, mugs, and other artifacts from the ship.Bottle shapes include round and square; there are bottles made of glass, and bottles made of ceramic material.In the middle of the area, there are wooden planks remaining from the bottom and keel of the vessel.

Figure 5 .
Figure 5.The keel line is actually straight, though it appears slightly curved in the photo mosaic.Camera calibration reduces distortions originating from imperfections in the camera and is the first step toward increasing geometrical consistency in photo mosaics.Calibration can be performed either automatically or through a designated calibration procedure

Few
Figure 5, a side-scan sonar image of the wreck site is inserted in the upper lefthand corner and can be directly compared to the photo mosaic to illustrate differences in coverage and details.The main characteristics of the site are recognized in both the Figure 5 sonogram and in the photo mosaic.More site details can be perceived from the photo mosaic than from the sonogram.
Photo mosaicking is a useful tool for illustrating small-and medium-sized underwater scenes of known extent.With the present methodology, one can expect a single photo mosaic to cover up to a few thousand square meters of seabed.For larger areas, the disadvantages of geometrical distortions and low dataacquisition rate prevent geospatially correct images.Usually, it would be sensible to compromise resolution and obtain a coarser search with sonar.To our knowledge, neither the filter -feeder communities documented in Figures 3 and 4 nor the wreck site shown in Figures 5 and 6 could have been documented to the same level of detail by other methods.In Figures 3 and 4, we can characterize the filter-feeder habitat by the most dominant taxa, the type of seabed, and the co-existence among species.The photo mosaics of the archaeological site reveal the condition, position, and orientation of a large number of historical artifacts that together outline the size, approximate age, and type of shipwreck.The characteristics of the wreck site can be perceived within a few seconds from the photo mosaics.It would take hours to obtain the same level of understanding and knowledge of a site by means of single images or video.