THE APPLICATION OF VIDEO IMAGE PROCESSING TO THE STUDY OF NEARSHORE PROCESSES

h o u r s . . . PERHAPS THE MOST POPULAR oceanographic setting for the public is the beach and nearshore environment. Poetic accounts since early history describe the comings and goings of tides, waves and sand. More recently economic, recreational and defense pressures have lead to greatly increased efforts to achieve a more quantitative understanding. Critical to progress in understanding nearshore dynamics has been the abi l i ty to make physical measurements under natural conditions. This has t rad i t iona l ly been accompl i shed with arrays of f ixed-point measurements of fluid mot ions and surveys of beach response. Although costly and logistically difficult, the resulting insight has been invaluable . Yet. in some ways, these data sets have provided only a glimpse of the variability of a natural beach system. This paper will explore an alternative approach to sampling in the nearshore, taking advantage of the visible nature of the fluid motions and, indirect ly , of the beach response . Over the past decade, the avai labi l i ty of low-cos t video hardware and of image processing systems capable of extract ing valuable data from video images has provided a valuable tool for s tudying the nearshore, In the fol lowing section, we descr ibe the general physics of the nearshore and the resulting sampling constraints. We then discuss the basic elements of video image processing. Finally, we illustrate several examples of fluid and bathymetry measurements that can be made at low cost and over long periods of t ime using these video techniques.

PERHAPS THE MOST POPULAR oceanographic setting for the public is the beach and nearshore environment.Poetic accounts since early history describe the comings and goings of tides, waves and sand.More recently economic, recreational and defense pressures have lead to greatly increased efforts to achieve a more quantitative understanding.
Critical to progress in understanding nearshore dynamics has been the ability to make physical measurements under natural conditions.This has traditionally been accomplished with arrays of fixed-point measurements of fluid motions and surveys of beach response.Although costly and logistically difficult, the resulting insight has been invaluable.Yet. in some ways, these data sets have provided only a glimpse of the variability of a natural beach system.This paper will explore an alternative approach to sampling in the nearshore, taking advantage of the visible nature of the fluid motions and, indirectly, of the beach response.Over the past decade, the availability of low-cost video hardware and of image processing systems capable of extracting valuable data from video images has provided a valuable tool for studying the nearshore, In the following section, we describe the general physics of the nearshore and the resulting sampling constraints.We then discuss the basic elements of video image processing.Finally, we illustrate several examples of fluid and bathymetry measurements that can be made at low cost and over long periods of time using these video techniques.

The Nearshore Problem
The study of nearshore processes can be considered to be composed of two interdependent R.A• Holman, College of Oceanography, Oregon State University, Ocean Admin Bldg.104, Corvallis, OR 97331-5503; T.C. Lippmann, Dept. of Oceanography, Naval Postgraduate School, Monterey, CA 93943-5100; A.H. Sallenger, J.W. Haines, Center for Coastal Geology, U.S. Geological Survey,600 4th St. South,St. Petersburg,FL 33701,USA.components: the dynamics of a wave field over the shoaling bathymetry of a beach and the reciprocal response, through sediment transport, of the beach bathymetry to that wave field.The objective of nearshore processes research is to develop a predictive understanding of each component, and also of the nonlinear systems aspects of the feedback linking the two components.
In contrast to the deep ocean where waves are linear and are well modeled with sinusoidal shapes, waves in the nearshore can be strongly nonlinear.This is visually apparent through the familiar steepening and eventual breaking of shoaling waves.It can be expressed also in a wave spectrum by the transfer of energy from a central incident wave peak (typically at around 0.1 Hz for ocean waves) to higher frequencies (Elgar and Guza, 1985).Perhaps less visibly apparent, nonlinear processes also force flows at lower frequencies, including mean longshore currents and undertow, as well as, for natural wave fields, low-frequency flows called infragravity waves (for which 60 seconds, the time scale of a typical wave group, might be a representative period; Guza and Thornton, 1985).The overall result is a substantial evolution of the spectrum.The original deep water energy is spread to frequencies from 0 Hz to many times the incident wave frequency and longshore length scales from meters to kilometers.This all occurs over a very short distance, often within about 100 m of the shoreline• Moreover, the existance and extent of this evolution is a direct result of the presence and nature of the bottom bathymetry.
The nearshore problem becomes more interesting when it is remembered that the sandy bed, the bottom boundary condition that defines the nearshore and whose presence causes the nonlinearities described above, is itself erodeable.Thus, the wave and current field will drive sediment transport and cause significant changes to the bathymetry itself on time scales as short as hours to one day.The detailed mechanics of this process appear complex, and the result is profound.Due to the possibility of feedback between the wave field and the bathymetry, the nearshore takes on the characteristics of a nonlinear dynamical system with an associated range of nonlinear and even chaotic behavior.Proper sampling of such a system may require very long time series.

Sampling Constraints
Time scales of interest range from 1 s for fluid motions to years for the bathymetric response.Fortunately, the two components can, to a good approximation, be separated by frequency.With the exception of possible stratification effects within the bottom few centimeters of the boundary layer, the erodeable nature of the bottom boundary can be neglected for time scales shorter than roughly one day (bed response to storms can be faster).Thus, fluid motions can be studied in the presence of a local, fixed bathymetry that must be measured each day.All previous major field experiments have operated under this assumption.
Even with this assumption, several severe sampiing constraints remain.Foremost is spatial inhomogeneity.In the cross-shore, strong inhomogeneity often occurs over an incident wave wavelength.In the longshore, bathymetric variability can occur over a range of scales from beach cusps (of order 10 m) to large scale bar systems (100s to 1,000s of meters).While fluid motions occur over a continuum of longshore wavelengths, typical circulation cells and infragravity motions occur over scales of order 100 m.

Traditional Solutions
Traditional sampling involves the use of fixed pressure or velocity sensors in an array that spans cross-shore and longshore scales of interest.The measured variables are usually diagnostic and well understood, and the sensor response has been well tested.Sample records are often of order 4 hours, a compromise between the need for long records for statistical stability and short records to minimize depth variability associated with the tide.These records, generally spanning 100-1,000 cycles of the dominant periods of wave motions, allow an excellent statistical description of the wave field (for example, Thornton andGuza, 1982: Oltman-Shay andGuza, 1987).
However, several limitations exist.First, the spatial description of the wave field is always a compromise based on the availability of instruments.The sampling array is usually focused on either cross-shore or longshore sampling depending on the emphasis of the experiment, and longshore bathymetric homogeneity is almost always assumed by default.Second, all wave motions standing against the shoreline (the majority of measured energy close to shore) have a frequencydependent cross-shore structure that must be ac-counted for in measurements made with in situ instrumentation.Third, the logistics of maintaining a fixed instrument in the harsh environment of the nearshore can be daunting.
These constraints led to the concept of using video signals to monitor nearshore processes.The potential of such an idea is reinforced by the fact that many nearshore processes have a visual manifestation, and that, since most nearshore flows are shallow water flows with no significant vertical gradients (excepting undertow), the visual surface layer is representative of the entire water column.In addition, the logistical advantages of removing the sensors from the hostile water column are substantial.This paper documents the application of relatively standard video image processing technology to one oceanographic field of study, the understanding of nearshore processes.Presumably the same technology can be directed toward a range of other oceanographic studies.

Basic Elements of Video Image Processing
The basis of video image processing is the quantification of intensity variability of an image into a two-dimensional array of picture elements, or pixels.A variety of image types can be used including gray-shade pictures from black and white cameras, full color or intensity from a color band or combination of bands, or even nonvisual-band data such as infrared.The optimum basis signal depends on the process of interest.From the intensity data.some geophysical measure is inferred, for example the runup of a wave on the beach face, or the location of a breaking wave over a submerged sand bar (Holman and Guza, 1984;Lippmann and Holman, 1989).The use of video allows the time dependence to be characterized for these measurements.
Thus, the successful use of video image processing for any study requires an understanding of three component problems.In increasing order of difficulty, they are as follows: 1) temporal aspects of video sampling, 2) spatial aspects and the transformation between image and real-world coordinates, and 3) the relationship between image data and geophysical signals of interest.
Video records are stored and displayed as a series of discrete images or frames, analogous to a traditional movie.Black and white video consists of 30 frames per second.Color video standard is 29.97 frames per second, a discrepancy of 7.2 seconds over a standard 2-hour videotape that must be accounted for.Video can also be recorded in a time-lapse mode using a number of commercially available recorders (for which the primary market is surveillance).Thus, a standard 2-hour tape can last up to 20 days if a frame is recorded every 16 seconds.To allow synchronization with other data sources, time code from a standard source can be written as either SMPTE code on the audio track of the tape or as VITC code in the vertical interval (the horizontal "bar") between frames (SMPTE and VITC are IEEE standards for time code).Clock oscillators on standard commercial cameras and VCRs are usually excellent, accurate to within a small fraction of a second over the 2-hour duration of a tape.
Unlike a movie, each video frame consists of a series of 480 horizontal scan lines (split into two interlaced fields of 240 lines each).Information within each scan is digitized into a series of pixels, usually 512 or 640 per scan line although the information content may be bandwidth-limited by the camera or recording medium (standard VHS and S-VHS provide 240 and 400 lines of resolution, respectively).Thus, the basic spatial resolution of image data is simply determined by the angle subtended by a pixel times the range from the camera, while the dynamic range available depends strictly on the number of pixels.Perhaps counter-intuitively, increased "sub-pixel" resolution can be obtained for objects much larger than a single pixel by calculating their center of mass.However this technique is only applicable to special, large targets, for example control points in an image (discussed below).
Key to the use of image data is the ability to convert between image and ground coordinates for features of interest.For any particular object located by its three-dimensional (3-D) ground coordinates, the associated 2-D image location can be found uniquely using one of several transformation algorithms (e.g., Lippmann and Holman, 1989).The opposite process, the determination of the 3-D ground location of a 2-D image feature, is underdetermined and further information is needed.A traditional photographic solution has been to use stereo techniques, the simultaneous estimation of ground location of an object by two or more independent cameras.Alternatively, several situations occur in which image features are naturally constrained.For example, waves can be assumed to occur on a horizontal plane whose elevation can be estimated by a local tide gauge.Similarly, shadows lie in a plane whose location can be found knowing the object location and basic ephemeris geometry.A requirement for any of these calculations is knowledge of the camera location (and potentially descriptors of image distortion) and field of view including azimuth, tilt, focal length and roll.While these parameters can be measured directly, the precisions achievable would be a severe limit to accuracy.It is preferable to use a least-squared inverse algorithm, based on many control points at known ground and measurable image locations, to solve for geometric image parameters.

Image Data Types
Figure 1, an example frame from a video recording of a nearshore wave field, illustrates three basic approaches toward image analysis.Most simply, intensity data can be observed at an individual pixel, or from an array of pixels at locations that might correspond to other submerged instrumentation.In this mode, the pixels are analogous to individual instruments and time series data of video intensity can be treated in exactly the same fashion as from other fixed sensors.
Somewhat more data-intensive is the treatment of video data in terms of vectors of contiguous pixels corresponding, for example, to a crossshore transect across a beach face to calculate position of the water's edge (known as runup).In this case, each sample consists of an intensity value from each of the pixels and time variability can be presented as a time-space plot or a "timestack" (Fig. 2) (Holland and Holman, 1993).
Finally, data can be considered in terms of an image area when 2-D spatial patterns are of interest.Most often, the area will be the oblique mapping of a simple horizontal surface, for example, when examining the sand bar morphology over a nearshore area or perhaps the spatial extent of a plume.Due to the storage size of these individual subimages, temporal sampling of area data is usually sparse.

Application to Nearshore Processes Research
Video image processing provides some distinct advantages over traditional methods.First, an oblique image (such as Fig. 1) combines a nearfield region of good spatial resolution, close to the camera, with extensive, but lower resolution, longshore coverage, in this case out to the horizon.This results from the tangent relationship between image and ground coordinates.Second, once the basic image geometry has been determined, the design of spatial sampling arrays can be made at any subsequent time and can be modified whenever necessary.If increased spatial resolution is needed, additional pixels or runup transects can be postsampled.The addition of these extra data streams comes at no further cost except computer time and disk space.Finally, the financial and logistical costs of video field deployments are relatively low, as is the potential for storm damage.
The primary disadvantage of video data is the often unknown relationship between visual signals and geophysical variables of use in testing theory.Ideally, video tools should allow measurement of both wave and beach data.In the following section, we first describe techniques to measure wave motions and then those that allow monitoring of useful bathymetric quantities.

Measurements of Fluid Processes
The progression of incident waves across the nearshore is clearly visible to the eye, so it is no surprise that time series of intensity from individual pixels also show wave-like variability.Lippmann and Holman (1991) compared pixel intensity time series with those of a co-located pressure sensor.Visual similarity of the records was usually strong, and spectra of video intensity always showed coherent peaks at the correct frequency, allowing estimation of the peak wave period.The observed zero phase difference between video intensity and local pressure suggests the use of video as a proxy for incident wave phase.For example, Figure 3 shows an observed cross-shore phase structure of progressive incident waves approaching a barred beach, based on a cross-shore array of pixel intensity time series.Discrepancies between observations and linear theory appear related to finite amplitude effects (wave height can no longer be considered infinitesimal in the analysis).Similarly, longshore arrays of pixel time series have been used to make high-resolution estimates of incident wave angle, in exactly the same way that longshore pressure arrays have been used (Lippmann and Holman, 1991).Incident wave angle is a critical input to the estimation of mean longshore currents and longshore sediment transport.
The magnitude of the intensity signal associated with any individual wave crest depends on whether the wave is breaking or not.Lippmann and Holman {1992) have used this to characterize the statistics of wave breaking and wave-breaking modulation across the nearshore.Because wave dissipation can be theoretically equated to the radiation stress gradients that force mean or low-frequency flows (Longuet-Higgins and Stewart, 1964), it is hoped that video may offer a valuable dynamical tool when the relationship between video intensity and wave dissipation is better understood.
Wave height cannot be directly estimated offshore from a single camera, a consequence of viewing the 3-D world in a 2-D image.However, as the waves reach the beach in the form of swash, the runup must lie on the known 2-D surface of the Cross-shore Distance (m} Fig. 3: Shoreward propagation q/" incident band wave crests over a barred bathvmetrv (bottom) as evidenced by the phase propagation (top) of the Jirst empirical orthogonal fimction of video intensity time series.The plethora qf sampling locations (0,11) results from the ease of sampling video at as many locations as desired.Observed phase speed (0 ) is slightly fitster than linear shallow water theory (---), possibly a consequence qf finite anq~litude.The observed amplitude structure (middle) plw,ides some measure of the intensity of breaking seaward attd over the bat" crest (fronz Lippmamz and Holman. 1992).
beach and can be directly transformed into a time series of sea surface elevation.The timestack (Fig. 2) illustrates the video intensity time series from a vector of pixels that correspond to a particular crossshore beach profile (e.g., the solid line in Fig. 1).From this timestack, the runup can readily be digitized either manually or using an automated edge detection algorithm.In contrast to data from locations seaward of the shoreline, the shoreline is always an antinode and the frequency-dependent nodal structure does not obscure the observations.On the other hand, wave runup can be influenced by several local effects such as percolation and nonlinearities that may themselves obscure the relationship with the offshore wave field (Holland and Holman, 1991).A particularly valuable application of video mnup comes from the digitization of runup spectra at multiple longshore locations and the computation of frequency-wavenumber spectra to detect edge wave modes and shear waves (Holman et al., 1990).This option is particularly attractive since the only cost associated with the additional longshore coverage is increased analysis time.
Other data streams appear available from video images.For example, Holland and Holman (1991) tracked the leading edges of overwash events recorded by a remote camera to make a statistical estimate of the importance and dynamics of overwash on the erosion of low-lying barrier islands.In addition, close examination of the timestack (Fig. 2) shows foam streaks on the water's surface that appear to move back and forth with the waves.Quantification of water velocity through analysis of these foam streaks appears to be a profitable research area that we are only beginning to investigate.In general, the eye is capable of forming quantitative assessments of many processes when viewing the nearshore.The task at hand is to translate these human skills into computer algorithms.

Measurements of Bathymetry and Morphology
Measurements of the bathymetry of the beach fill three purposes.First, the bathymetry represents the response of the system to wave forcing.and must be known for dynamics studies of that wave forcing.Third, beach profile data are sometimes needed as an input to other measurements.For example, foreshore profile data are needed in the digitization of wave runup data (described above).
Since the estimation of 3-D information from 2-D images is underdetermined, techniques evolve around mechanisms to provide further information about imaged objects.Traditional stereo techniques simply provide two independent sets of image coordinates of an object so that its realworld coordinates become over-determined and can be derived in a least-squares sense.However, stereo relies on the identification of unique features in each view, and identifiable unique features are uncommon on the beach face.
A more promising approach is to constrain one dimension in the problem.One method has been the use of either sheets of light (produced by shining light through a slit or using a fan-beam laser) or shadows to slice the observed field along a plane.For example, Holman et al. (1991) used shadow equivalents, falling across a beach face, to quantify the beach profile along the shadow line to within an rms vertical error of 5 cm.
A more common constraint is to assume that image patterns in the water have a vertical coordinate that is at mean sea level, and so can be measured with a local tide gauge.Horizontal errors, on the order of the vertical displacement from mean sea level times the tangent of the camera look angle, yield only insignificant changes to the patterns described below.Lippmann and Holman (1989)   ages (e.g., Fig. 4A).For a beach face of moderate steepness (approximately 1:10), they found a good correspondence between the surveyed shoreline and the location of a white band associated with shore break.Variation of this "shoreline contour" with tidal elevation is now being explored as a remote technique to estimate the swash zone beach profile (N.G.Plant, personal communication).Offshore, Lippmann and Holman (1989) showed that bands of white, separated from the shoreline by dark regions, represented preferred breaking over offshore sand bars.Thus, the white band served as a good proxy from which the location and morphology of submerged sand bar systems could be quantified.
Figure 4B shows a subarea of the oblique image (Fig. 4A) that has been rectified into horizontal coordinates based on the assumption of mean sea level and the standard transformation equations between image and ground coordinates.From Figure 4B, cross-shore and longshore length scales of bar and shoreline morphology can be extracted.

The Future~Development of Large Databases
Major nearshore field experiments of the last several decades have typically lasted several weeks, largely based on available funding.During this time, an average of 2-3 storm cycles of wave forcing and beach response have been recorded.These data form much of the basis for our understanding of nearshore dynamics (and have been invaluable).Yet the apparent complexity of natural sand bar systems (Fig. 4 and a plethora of other examples) suggests that stable statistics may require many beach cycle realizations.
The logistical ease and low cost of the time exposure technique, combined with the apparent richness of the resulting images, has lead to the development of long-term, unmanned installations, called Argus stations, at several coastal sites.These PCbased image processors are programmed to collect time exposure images on approximately an hourly schedule.Along with supporting environmental data, the images are automatically down-loaded to a central computer each night and added to a constantly growing large database on coastal variability.Figure 5 shows selected monthly images from the Oregon Coast station, illustrating the richness of information and apparent complexity of behavior from this energetic beach system.
The development of Argus capabilities is presently proceeding much as did that of satellites.Basic imagery is now being collected by three stations, with several more expected to come on line over the next year.Image geometry is well known, but our understanding of the relationship of image features to geophysical processes continues to evolve.The inclusion of further types of images (e.g., timestacks) and measurements into the Argus program is an ongoing process, as is the extraction of more information (e.g., foreshore profiles) from existing and incoming data.As with satellites, a major task for the future will be the development of software to allow the easy examination and manipulation of the increasing image database.It is our hope that this enormous increase in available data on nearshore behavior will provide a valuable aid to the formulation and testing of nearshore system models.

Fig. 1 :Fig. 2 :
Fig.1: Sample video frame showing the wave field at the U.S. Army Corps Field Research Facility, Duck, NC.Small white features on the dune crest are control markers that can be used to solve for the camera geometry.White plus signs represent a short cross-shore array of example pixel locations from which pixel intensity time series could be collected.The black line, a vector of contiguous pixels corresponding to a particular cross-shore profile, is used to quantify wave runup at that location.The dashed box represents an example of an area of interest, in this case a 200 m length along the crest of a submerged sand bar.

Fig. 4 :
Fig. 4: (A) Ten-minute time exposure of the wave breaking patterns at Agate Beach, on the central Oregon Coast.White bands indicate preferential breaking over submerged sand bars or at the shoreline.Intermingled darker areas correspond to deeper channels and troughs.The white box spans 2 km in the longshore direction.The jetties at Newport, 5 km to the south, are seen as the dark band across the beach in the far field.Images like this are collected hourly by the Yaquina Head Argus Station.(B) Rectification of the boxed region in A, showing the horizontal map view of the morphology imaged.

Fig. 5 :
Fig. 5: Monthly time exposures from the Yaquina Head Argus Station showing the system variability over the first year of installation.The range of scales that must be measured to understand the system becomes painfully apparent.These examples were culled from hourly images taken throughout the year.