FEATURE VIRTUAL REALITY IN OCEANOGRAPHY

A S HUMANS WE SEE and relate to the world around us in three dimensions, yet as scientists we usually attempt to understand our data by portraying it in visual form as a plot or a picture• These two-dimensional portrayals of inherently three-dimensional processes are often difficult to understand in a total context due to the size of the data fields or the complexi ty of the system being portrayed. "Virtual Reality" (VR) is a methodology by which human beings interface with mul t id imensional environments created from computer-based data. VR allows us to view and interact with threedimensional data in a three-dimensional environment and provides an actual sense of presence that can inherently change the way we analyze the data. In short, an artificial three-dimensional world is created based on a data set of interest which one is then able to enter, navigate through, and directly interact with. The data may be from recorded observations, computer simulations, or from artistically crafted models. The device providing an interface with the data may range from s imple two-dimensional display systems like a workstation screen to more sophisticated user-immersive threedimensional systems. In all cases, the data must be visualized in some fashion and the user should be able to manipulate or otherwise interact with the d isplayed imagery. The addit ion of auditory and tactile feedback can be used to enhance the feeling of presence, and give the user more information about the (virtual) world. The use of this new technology will have profound impacts on the way we view our data, how we develop our models, and how we interpret the results• Before you classify this statement as a heady, euphoric portrayal of the promise of new technology, consider the advances made in scientific computing over the last decade• Computing power has doubled roughly every 11⁄2 years, enabling calculations once requiring massive computational assets to be performed on a desktop workstat ion in an office. Animat ion and volume


AS
HUMANS WE SEE and relate to the world around us in three dimensions, yet as scientists we usually attempt to understand our data by portraying it in visual form as a plot or a picture• These two-dimensional portrayals of inherently three-dimensional processes are often difficult to understand in a total context due to the size of the data fields or the complexity of the system being portrayed."Virtual Reality" (VR) is a methodology by which human beings interface with multidimensional environments created from computer-based data.VR allows us to view and interact with threedimensional data in a three-dimensional environment and provides an actual sense of presence that can inherently change the way we analyze the data.In short, an artificial three-dimensional world is created based on a data set of interest which one is then able to enter, navigate through, and directly interact with.The data may be from recorded observations, computer simulations, or from artistically crafted models.The device providing an interface with the data may range from simple two-dimensional display systems like a workstation screen to more sophisticated user-immersive threedimensional systems.In all cases, the data must be visualized in some fashion and the user should be able to manipulate or otherwise interact with the displayed imagery.The addition of auditory and tactile feedback can be used to enhance the feeling of presence, and give the user more information about the (virtual) world.
The use of this new technology will have profound impacts on the way we view our data, how we develop our models, and how we interpret the results• Before you classify this statement as a heady, euphoric portrayal of the promise of new technology, consider the advances made in scientific computing over the last decade• Computing power has doubled roughly every 1½ years, enabling calculations once requiring massive computational assets to be performed on a desktop workstation in an office.visualization of three-dimensional data are now commonplace where two-dimensional scatter plots were once the norm.Improvements in display systems tend to follow these computational advances, as evidenced by the proliferation of high-resolution flat-screen color monitors.Indeed, desktop-based stereographic systems which provide three-dimensional display of images are available now for about the same cost as a high-resolution monitor.As more users connect to the Internet and network transfer speeds increase to the Gigabit per second rate and faster, the distinction between telephone, computer, and television systems will be blurred and voice, data and video will be available via common lines.Virtual worlds will be networked via the global Internet and linked with a hypertext-based language so that users will be able to interact with these environments.Current efforts to create a standardized method of interaction with these networked virtual worlds have culminated in the nascent Virtual Reality Modeling Language (VRML).
How can we use this technology to best advantage in oceanographic research?Perhaps the most attractive feature of using a virtual world as a data visualization aid is that it provides ways of viewing the data from perspectives not possible in the real world.For example, we could create a virtual environment consisting of the multivariate results of a model of the small scale three-dimensional structure of the first few meters of the ocean and position ourselves just beneath the surface to examine the effect of a parameter change on the processes responsible for momentum transfer from wind to waves.Or, we could see the effects of circulation on the modeled distribution of a larval fish swarm as it moved through the mouth of the Chesapeake Bay from the perspective of one of the larvae.Finally, we could view time-dependent hydrographic observations as a three dimensional animated object and fly through the data to watch the properties change with time from any vantage point• The purpose of this article is twofold• First, we'll explain some general concepts about VR and its use as an analysis and visualization tool.Next, we'll put these concepts into the oceanographic context by discussing some of our recent work in utilizing VR to analyze and visualize observed and modeled salinity data from the Chesapeake Bay.

Virtual Reality in a Nutshell
By most accounts, the birth of virtual reality coincided with the beginnings of the digital age.A primitive system consisting of a head-mounted display and hand-held control device developed in the 1960s by Ivan Sutherland was able to provide a user with the visual illusion of being in a world made up of computer-generated three-dimensional objects (Vickers, 1972;Fisher, 1990).Further development in the next decade was driven by increased military research funding as well as by the growth of the consumer electronic market.Most recently, advances in computational power and speed as well as in miniaturization of electronic devices have made possible high resolution, small size displays, and associated components.The advent of efficient, inexpensive digital circuitry and the personal computer enabled fast, digital image generation and, soon after, virtual reality as a general scientific tool came into being.The entertainment industry now has a strong hand in the development of these immersive, interactive systems and will most likely continue to shape their future direction.
Although there are several types of VR systems presently in use, they all have in common the concept that the user may interface directly with computer-generated visualizations.Some VR systems portray the virtual world on a two-dimensional screen in a stereographic projection, requiring the user to view the display with stereographic glasses to get a three-dimensional effect; these types of displays are often called "desktop VR" or "fishtank VR".In some instances, data from remotely controlled sensors in the real world are projected into a virtual world to enhance one's ability to analyze the data or enable operations in a hostile or otherwise inaccessible environment.This technique, called "augmented" or "mixed reality", is often used in medical applications combining miniature television cameras with smallscale invasive remote control surgical tools.The military uses this form of VR to simulate combat operations in any location or weather conditions, projecting computer generated hostile forces and real-time remotely sensed images into the wargaming database.The Department of Defense is actively working to connect military simulators of diverse nature and purpose into a large network called the Defense Simulation Internet.
Higher level systems immerse the user in the virtual world through the use of head-mounted displays (HMDs), which show the computer-generated objects on display screens mounted inside a helmet.The helmet is often free to move with the wearer, whose motion provides translation through the virtual world.In other cases these display screens are mounted on a tripod armature, as with Fakespace Labs' BOOM TM.The user peers into the BOOM and moves it like a periscope, though with greater mobility, to change the view of the scene.An example of such a higher level VR system is the Virtual Wind Tunnel (Bryson and Levitt, 1991).Located at NASA Ames Research Center, the Virtual Wind Tunnel is a visualization system that displays computational fluid dynamics (CFD) data in a virtual environment and allows interactive manipulation of the displayed data with a glove input device• Scientists using an HMD or BOOM are able to walk through and around the three-dimensionally displayed CFD data as well as use the glove to directly manipulate visualization parameters such as particle release points.Other examples of this type of VR application include the University of North Carolina's Nanomanipulator, designed to view atomic surfaces (Taylor et al., 1993), and the Naval Postgraduate School's ship walk-through system (Pratt et al., 1994)• Another approach to immersive display that gives an improved sense of being in a large space is the CAVE (CAVE Automatic Virtual Environment).The CAVE, powered by a Silicon Graphics Onyx parallel-processor computer, is a virtual reality visualization system composed of high-resolution projection screens arranged in a 10-foot cube with computer-generated images projected on three walls and the floor.Developed by the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago, the CAVE allows scientists to interact with virtual worlds created from observed data or simulations.A viewer wears a 6 degree-of-freedom head-mounted motion tracker and stereo-shutter glasses so that the correct projections and perspectives of three-dimensional objects are presented as the viewer changes position inside the CAVE.A wand (essentially a computer mouse) held by the viewer allows interaction with and navigation through the virtual environment.Properly using the walls and the floor to display the data truly gives the viewer a sense of being in and moving through a very large space.The inclusion of auditory input as secondary cueing provides even more realism to the scene.The CAVE system was first put on public display at the SHOWCASE exhibit during the annual SIGGRAPH computer graphics conference in 1992 (Cruz-Neira et al., 1992).The most recent CAVE exhibit included the presentation of over 40 reviewed VR applications during SIGGRAPH '94.
Consider the following not-so-fanciful scenario of a virtual Gulf Stream as an example of using a virtual world to increase our understanding of oceanographic processes.The results of a numerical simulation of the Gulf Stream are displayed in a virtual environment as a visual, graphical object.A researcher using a data glove "flies" toward the color contoured thermal front and hears a change in the pitch and volume of sound tied to temperature or velocity.The researcher feels a force and a rise in temperature in the glove as contact is made with the Gulf Stream wall.Touching a button on the glove starts a simulated wind blowing on the sea surface, the results of which become immediately apparent as the researcher is able to hear, see and feel the response of the Gulf Stream to this external forcing.We'll discuss now the component pieces necessary to create such a virtual world.

How to Make a Virtual World
As with most computer graphics simulations, creation of a virtual world requires data as input and a computational simulation process.Visualizing the results in a virtual environment further requires a graphical rendering routine.What separates VR from other visualization techniques is the ability to interact with and manipulate the data, accomplished via the interaction module.Finally, an output device is required to see the results.
The data can be observations or model results.Model results have the advantage of fine temporal and spatial resolution but may suffer fi'om numerically induced deficiencies in the portrayal of the physics of the modeled process.Observations often suffer from sparseness in coverage, both spatial and temporal, but have the advantage of being real data.A combination of modeled and observed datasets can be used simultaneously to enhance the feel of the virtual world or to create many graphical objects.
The computational simulation process is the main program of the virtual reality application and controls the graphical creation of the virtual world by the rendering routine.It is in this main simulation process that the interaction module resides and is linked to the visualization process to coordinate the interactions between graphical objects in the virtual world and between those objects and the user.It also portrays the virtual world on a time-iterated basis so that motion or action occurs in a seamless fashion.
The rendering or realization routine presents the output from the simulation process to the viewer in the form of graphics, sound or tactile cueing (force feedback).The data may be drawn as a series of points, lines or polygons, the limiting factor being the graphics processing power available.Although most oceanographers tend to avoid graphics programming like the plague (perhaps rightfully so!), an understanding of basic computer graphics techniques is instructive.A large volume of work exists on the art of computer graphics; we urge interested readers to familiarize themselves with basic principles and techniques and recommend Foley and Van Dam's work, Fundamentals of Interactive Computer Graphics (1982) or the excellent series, Graphics Gems: A Collection of Practical Techniques for the Computer Graphics Programmer (1994), published by Academic Press.A sufficient amount of computational power is required to satisfactorily portray large datasets or complex scenes as the images need to be rendered at a sufficiently high-frame rate (usually 15 to 20 frames/s) so that animation is presented without flicker.Sound may be used either as an effect which enhances the realism of the virtual world, similar to a sound track in a movie, or as an indicator which reflects the behavior of the system being modeled or observed.Visualizing the properties of complex dynamical systems by portraying these properties as sounds in a virtual world may enhance our understanding of how these systems operate.Finally, tactile cueing via force feedback or thermal indications in a data glove could allow one to °'touch" or manipulate a virtual object.
Interaction with the virtual world is accomplished via some user interface tool, such as a mouse, joystick, wand, or voice activated process.An important element of this interaction is a means of tracking the position and orientation of a real world object in the virtual world.Software is often used to implement a virtual menu button or slider which can control some aspect of the virtual environment.The output device is the computer screen, HMD, BOOM, or CAVE.

Using Virtual Reality in the Chesapeake Bay
We now present an example of a working VR application designed to explore observed and modeled salinity data from the Chesapeake Bay.Our project was developed for the CAVE as a collaborative effort among researchers from the Center for Coastal Physical Oceanography (CCPO) at Old Dominion University and the National Center for Supercomputing Applications (NCSA).Our motivation stemmed from a desire for a more complete understanding of how physical processes affect the circulation in the bay as well as how they influence the shelf-to-bay transport of shelfspawning, estuarine dependent biological species such as the blue crab (Callinectes sapidus).
The Chesapeake Bay, the largest estuary in the United States, serves as nursery grounds or spawning areas for many commercially important species and is home to an extensive fisheries industry.The larval stages of shelf-spawning, estuarine species are usually widely distributed over all three spatial dimensions as well as temporally.Biological behavior such as motile ability, buoyancy characteristics, and affinity for light or water mass characteristics all contribute to the location of the larvae on fairly small spatial scales.Equally important, yet operating on larger scales, is the advection of the larvae due to the local circulation.The complex circulation features in the estuary itself and in the bay mouth/shelf region are due to physical processes such as wind forcing, tidal mixing and runoff variability, all operating over wide time and space scales.
The estuarine circulation in the Chesapeake Bay is usually described as consisting of relatively fresh water near the surface flowing seaward atop more saline near-bottom water flowing in the opposite direction.The salinity field is controlled pri- River and observed the salinity isosurfaces (Fig. 4) while we discussed the concepts of estuarine circulation and rotationally controlled flow.The trip ended with a quick ascent followed by a hover and an explanation of the bay's seasonal salinity cycle, demonstrated by the animated salinity fields.
The implications of using virtual reality for the visualization of scientific data, both real and simulated, are clear.Visualizing our Chesapeake Bay data in a virtual environment provided a view normally unobtainable in the physical world.Aspects of the data not clearly recognizable with static twoor three-dimensional images became immediately apparent simply because of our ability to navigate intuitively through the virtual environment and examine features of the data from different positions.The complex relief of the Bay bathymetry and associated topographical effects on the salinity distributions in the Bay were obvious, confirming the pres-ence of low salinity water over shoals and higher salinity water in the channels.A seasonal cycle of the salinity field at all depths was obvious as well.
It is well known that the shallowness and boundary dominated nature of the bay should make the circulation especially sensitive to wind forcing effects, yet exactly how this is effected is not presently clear.Wind-forced simulations of the circulation in the bay should clearly show an increase or decrease in the estuary/shelf volume exchange based upon wind direction; examining these results in a virtual environment should make clear the exact progression of this process.To this end, we have ported the results from a three-dimensional hydrodynamic circulation model of the Chesapeake Bay and adjoining shelf to our virtual Bay environment and continue to examine circulation dynamics arising from wind stress, tidal forcing, runoff/precipitation and solar in'adiance.We marily by freshwater runoff, the majority of which comes from the Susquehanna River with additional input from the Potomac, James, and other sources.The most buoyant water is usually found along the western side of the bay mainly as a result of rotational influences.The mean flow in the estuary mouth and adjacent shelf is also rotationally affected and modified by bathymetry.The flow pattern consists of buoyant water outflowing along the southern reaches of the bay entrance and over the shoals whereas inflow of dense, saline shelf water is found in the bathymetric depressions or channels (Boicourt, 1981; Valle-Levinson and Lwiza, unpublished data) (Fig. 1).The outflowing buoyant water usually exhibits an initial anticyclonic turn (to the right in the northern hemisphere) after exiting the mouth, becoming a rightbounded coastal plume and coastal current system (Boicourt, 1973).Seaward of the buoyant outflow area is an area of intense mixing as the fresh outflow merges with the more saline shelf water.
Our CAVE application consisted of a three-dimensional graphical representation of a Chesapeake Bay bathymetry data set textured and colored according to depth, upon which were overlaid transparent isosurfaces derived from monthly composites of Chesapeake Bay surface salinity observations.Bathymetry resolution was 1 rain of latitude by 1 min of longitude and had some vertical exaggeration to highlight the relief.The monthly composites were generated from a data set that includes records between 1949 and 1980.An example may be seen in Figure 2. The dependence of salinity with depth was not always available from the data set and was prescribed analytically as in Hansen and Rattray (1965).The salinity isosurfaces were then animated to show seasonal changes of the regional salinity.
This virtual Chesapeake Bay environment was publicly presented in the VROOM (Virtual Reality Room) at the 1994 SIGGRAPH conference in Orlando, FL.Over the week-long conference, there were 43 such applications presented to groups of five viewers at a time.The viewers wore stereographic glasses to provide the three-dimensional effects of the CAVE projections while a position tracker assured that the data were displayed with proper perspective.Among the other applications were projects dealing with air traffic control, turbulence in flow over a flat plate and the development of tornadoes along gust fronts (see NCSA's AC-CESS magazine, Vol. ( 8)-2 for a complete list of projects).Our 5-min presentation for each group began with a south-to-north descent from 100,000 feet and 40 km south of the bay mouth to a point just above the air-sea interface.We flew north, pointing out landmarks, geography and the threedimensional virtual "signs" identifying the James and Potomac Rivers (Fig. 3).As we neared sea level, the sounds changed from sea gulls to that of crashing surf.Continuing the descent underwater, we crossed the air-water interface with an audible splash and hovered just under the translucent surface as the viewers were shown the main shipping channels and the abrupt topographic variations.We then flew to a point near the mouth of the Potomac  Fig. 3: View of the virtual Chesapeake Bay.The obeserver is located just off the mouth of the Potomac River looking northward.
have used the publicly available Princeton Ocean Model (Blumberg and Mellor, 1981) as the computational cornerstone of this current effort because of its embedded 2.5 level turbulence closure scheme and its ability to reproduce accurately tidal forcing.In addition to simple process studies, we are exploring more complex scenarios using this model and immersive visualization techniques.

/ ~ i ~;i~)i!i & ¸ ~i~ i~
Fig. 4: A view from the main channel of the virtual Chesapeake Bay looking north.Above the viewer is an isosurface with salinity of 17.
To our knowledge, our CAVE application is one of only a few applications of virtual reality in the field of oceanography.Another oceanographic VR application displayed in the CAVE during S1G-GRAPH '94 was the "JASON Interactive Mapper", which re-enacted a visit to hydrothermal vents in the Guaymas Basin in the Gulf of California (Feldman et al., 1994).Scientists at the Naval Postgraduate School have also used this technology to explore Monterey Bay with an underwater autonomous vehicle (UAV) (Brutzman et al., 1994).The ongoing development of a suite of user-friendly VR tools will make this technology available soon to the oceanographic community at large.It will be up to us to elnploy this exciting visualization and analysis method to enhance our knowledge of the ocean. If

Fig. 2 :
Fig.2: Composite surface salinity for the month in the Cheasapeake Bay.This is a month with appreciable fresh water influence in the lower portions of the Bay.Coriolis effects deflect high (low) salinity toward the eastern (western) shore of the Bay.
Fig.1: Mean flow (cm/s) during the period ofOctober 6-7, 1993.Positive--red (negative--blue)  values indicate flow into (out of) the estuary.Contour interval is 5 cm/s.Looking into the estuary, Norfolk, VA is on the left and the eastern shore is on the right.Notice that outflows are found over shoals, and inflows appear affected by Coriolis acceleration over the channels.
Animation and volume G.H.Wheless and A. Valle-Levinson, The Center for Coastal Physical Oceanography, Old Dominion University, Norfolk, VA 23529, USA.W. Sherman, National Center for Supercomputing Applications.405 N. Matthews Ave., Urbana, IL 61801 USA.
you have access to an Internet browser such as Mosaic, navigate to the URL http: //www.ccpo.odu.edu/vr.htmland look at our document VIRTUAL REALITY IN OCEANOGRAPHY for more information.Additional information about the CAVE may be found at http : //www.ncsa.uiuc.edu/EVL/docs/ html/CAVE, html