Virtual reality (VR) is an emerging technology that provides users with realistic, computer simulated interactive three-dimensional (3D) environment in which people become immersed [1] [2] . VR provides high interactivity, which is achieved when users are provided with immediate feedback through their perceptions that a mediated environment is modified, based on their input [3] . This relates to the prominent features of VR. VR interfaces provide high-quality 3D images of products, interactivity with the products, and increased telepresence [3] [4] .

Telepresence is a sense of “being there” in an environment by means of a communication medium [4] [5] . Based upon sensory stimuli conveyed by a VR interface, human beings can create a perceptual illusion of being present and highly engaged in a mediated environment, while they are in reality physically present in another place [6] . VR also have significant effects on learning process [7] [8] .

There have been ongoing attempts by researchers in bringing virtual models to life with better visualization and interaction techniques [9] -[11] using various VR devices such as Flock of Birds, Cyber Gloves, Cyber Grasps, Head-Mounted Displays, and Force Feedback Devices such as PHANTOM Devices. With new ideas, incorporated with advanced equipment, and years of evolution, many have successfully implemented various fascinating applications. VR has evolved through many years of research in hardware and software technologies and applications. In this research, we use VR technology for the visualization of the granular materials and dis- locations in crystals.

For the immersive grain visualization it is very important to understand the dynamics of the individual grain as well as their collective behavior in the presence of a grain cluster and applied shear. From the mechanics perspective, this involves understanding of the governing equations for position, orientation, velocity, accelera- tion, and the contact forces between the grains [12] . The forces in a granular media in a confined vessel due to auni-axial compression are distributed onto the interior surfaces of the confined boundary [13] . Spatial packing order and interparticle friction in a large hexagonal close packed crystal, affect directly the distributions of nor- mal forces in a three-dimensional granular media [14] .

Extensive research has been done in an effort to understand various dynamics of grain interactions [15] -[21] . Granular flows consisting of about a million particles are visualized in 3D, using parallel discrete element me- thod (DEM) simulation [22] . DEM is a family of numerical methods for computing the motion of a large num- ber of particles like molecules or grains of sand [23] . Various types of rotations at different scales, for example the rotations of individual particles, the rolling and rigid-rotation of particle pairs, the rotational interactions of a particle within its cluster of neighbors, and the rotation of material regions are studied using numerical DEM simulations on two- and three-dimensional granular assemblies [24] .

Understanding the dynamics of dislocations and its propagation is important in order to create visualization models for the same. Topology of dislocation propagation [25] and real time interactive visualization of three- dimensional molecular dynamic models for dislocation nucleation and dislocation dynamics by a potential energy mapping algorithm [26] has been studied. Numerical techniques have been developed by various authors [27] -[32] for the evaluation and visualization of dislocation accumulation in various crystals. The authors have analyzed the plastic slip deformation of crystals and metal microstructures by finite element methods and eva- luated the developments of dislocation density distribution in conjunction with gradients of plastic shear strain on crystal slip systems. These continuing researches lay out our current understanding of the dynamics behind the dislocations in crystals.

The dynamics of the dislocations under applied load is evolved according to physical laws, yielding a set of differential equations, which are solved numerically. The output from these simulations, among other thing, is the spatial distribution of the dislocation curves for each time step. Because of their interactions, dislocation curves tend to develop into low energy configurations (see examples in [33] ). A dislocation dynamics model, named micro3d, has been developed in which the dislocations are represented as discrete segments in a three-dimensional (3D) crystalline space [28] . Detecting these 3D structures is not trivial and, therefore, visuali- zation becomes important in order to identify reactions among the various dislocation curves and to identifying structures.

We see significant contributions from many researchers in modeling and visualizing the particle behavior in granular media and the dislocations. There has been tremendous work done using the DEM for the simulation of granular flows. Many researchers have done the visualization of the dislocations using continuum mechanics analysis. However, there has been no research in visualizing the granular assembly and dislocations in materials using VR technology. Our hypothesis is that by using OpenGL rendering technique and combining VR technol- ogy with the mathematical simulations of the granular flow as well as crystal dislocations, researchers will have greater insight and a better understanding of the complex mathematical models using the sense of visual stimu- lus provided by the telepresence.

Our goal is to create 3D stereographic and immersive visualizations of three different types of models, (a) gra- nular assembly, (b) Delaunay network of the granular assembly, and (c) dynamics in crystal dislocations.

In the granular assembly as well as the crystal dislocations, the positions and orientations of the grains (Fig- ure 1) and dislocations are calculated according to the fundamental laws of physics using the force and momen- tum as

where and are the force and angular momentum, is the mass, is the acceleration, is the moment of inertia, and is the angular velocity.

In the granular assembly, the force and angular momentum on particle is

where is the total number of particles interacting with the reference particle, is the viscous regularization of Coulomb’s friction law, and, are the translational and the rotational coefficients of viscous damping, is the total force vector on particle exerted by nearest neighbor particles, is the velocity vector, is the position vector, and is the angular velocity of particle. Also

where, , and are the normal Hertzian force, damping force at the contact, and the tangential fric- tional force at the contact respectively. The details of this mathematical formulation and the subsequent deriva- tion of the differential equations can be found in [12] .

The force vector in Equation (3) is the resultant of all forces acting on the particle. In general, each force is a function of position, velocity, and angular velocity of the particle. The velocities and moments of the par-

ticles are calculated by the mathematical model at each time step by the application of above laws in Equations (3), (4), and (5). We find a similarity in the mechanics of the problems for both grain structures and dislocations simulation.

For the granular assembly and Delaunay network the goal is to visualize the behavior of the individual particles under application of axial shear strain. The assembly of granules is assumed to be contained in an elastic mem- brane immersed in a fluid as shown in Figure 2. The axial strain rate and the hydrostatic pressure exerted by the fluid control the deformation of the granular assembly. In our research, we have taken a cuboidal elastic membrane for the visualization.

Each granule is treated as a discrete entity and its interaction with other particles forms the mechanics of the problem that we have discussed in previous section. In the grain structures, we visualize the position, orientation and motion of individual grains. In our visualization model we render the grain particles as solid spheres and then create the Delaunay network of the grain assembly. The Delaunay network is a collection of tetrahedrons that are formed by connecting the centers of neighboring grains. Visualization of the Delaunay network provide more clear information about the grains which are in actual contact that we refer to as TRUE contacts and which are not in contact that we refer to as FALSE contacts.

For dislocations, the goal is to be able to visualize the formation and movements of dislocations to understand transient dynamics of the dislocation mechanisms by taking advantage of the depth perception.

In the first stage of visualization, we have used the OpenGL rendering technique to build a 3D model to observe the transient dynamics of the granular assembly and the crystal dislocations using the data from numerical solu- tions. We then implement the stereographic visualization using two projectors and a 3D glass. Then we integrate the models with VR technology by implementing, the VR devices, head-mounted display and flock of birds tracking device. Head mounted display has been used to immerse inside the model, and the flock of birds as the tracking device to ensure a telepresence in the virtual 3D model.

Using computational methods we first obtain the transient locations and orientation information for the grain particles and crystal dislocations. We save these information in various data files. We render the models by reading the data files in text format and updating the data files in each frame for the purpose of simulation. The data files are created by solving for the differential equations of the displacement and rotations in MATLAB. These data files consist of the location information for Dislocations and the location, orientation, velocity, ace- leration, angular velocity, and angular acceleration information of individual particles in the granular assembly for each time step. Here time step is a small increment in time that we consider to get the particle location and orientation information as the assembly is going through the strain and pressure to obtain transient information. The data files in case of the Delaunay network consist of the connected grain numbers or indices of a tetrahe- dral.

Following sub-sections describe the algorithms used to render in 3D as well as in immersive VR environment. A general algorithm for rendering process in OpenGL is shown in Figure 3. We implement the keyboard and mouse functionality to navigate around the simulated model in the virtual environment. “OpenGL rendering of the nth data file” step of the flowchart in Figure 3 represent an image of the locations and orientations of all the particles ordislocations in the assembly at a particular time step. These locations and orientations information are obtained from the data file created from the numerical simulation of the transient dynamics.

Depending on the extent of immersion, VR applications can be classified into two categories: immersive VR and non-immersive VR [35] . In the former, users wearing head-mounted displays are totally surrounded by enclosed virtual environments. Desktop or laptop computers, on the other hand, most commonly convey non-immersive VR. Thus, for users, VR experiences are limited to what they see on their display monitors and what they hear from their speakers. Generally most researchers use non-immersive visualization due to the high expense and cumbersome equipment required for immersive VR. However, this research focuses on both non-immersive and immersive VR interfaces to provide a better understanding of the granular flow and crystal dislocations. We have first implemented non-immersive VR to create a 3D stereographic environment with depth perception, and then use the VR devices to create the immersive environment [11] .

VR offers a high level of control over computer-mediated environments, both in terms of user abilities to ad- just the information according to their individual interests and concerns, and, in general, their ability to be active, rather than passive, in their engagement with the information [36] . Through high media richness and interactivi- ty, VR can generate compelling feelings of telepresence [3] [6] .

The experiments have been done in the VRCIM Lab [11] [37] using OpenGL rendering, projectors, polarized 3D glasses, and VR devices head-mounted display and flock-of-birds (Figure 4). First we created the algorithms to simulate the models using computer programming. Then we implemented the 3D stereographic visualization by using two projectors to project the same image in two different screens with a predefined offset value in order to visualize the computer models in 3D, with depth perception, using the polarized glasses. This stereographic visualization experiment in the lab, provided with a better visualization with depth perception. Next we con- ducted the experiment by integrating our OpenGL computer code to the VR devices head-mounted display to create an immersive environment. Then we used the VR device, flock-of-birds, in order to obtain the desired orientations of the 3D models for the immersed user to be able to investigate the transient dynamics of the de- sired particle or dislocation [11] .

This visualization technique enabled us to identify and fix the problems in the mathematical model itself for example in the granular assembly with unconstrained boundary or inappropriate boundary conditions. Initially we noticed that there were not enough boundary constraints in the model based on the particles flying outside the boundary during the simulation as shown in Figure 5. Our 3D visualization technique helped us to identify the problem and fix it. Figure 6 shows the simulations of the model after applying the appropriate boun-

dary conditions. Using the VR devices we were able to create the feeling of a telepresence which in-turn helped investigate the complicated dynamics of the grains on the outer boundary as well as the inner particles (for ex- ample the particles in the center of the 3D grain assembly), which otherwise is impossible to visualize using simple 3D visualization technique on a computer.

In order to understand the dynamic contact behavior of the grains in the assembly we built a simulation for the visualization of the Delaunay network that is created using the nearest neighboring grain particles. The TRUE and FALSE contacts are shown in Figure 7.

We have represented the dislocations lying on different slip planes in different colors to highlight the propa- gation of the dislocations. With this VR experiment we were able to visualize the destruction of dislocations and formation of new dislocations using the 3D immersive visualization technique. Visualization of the dislocations simulation is shown in Figure 8. Immersion using the head mounted display inside the virtual model helped us to quickly identify dislocations and slip planes. The flock of birds tracking device helped us to walk near the re- gions of interest and investigate the behavior of the dislocation’s propagation.

The user requirements have been fulfilled as we have successfully implemented the following:

Ÿ Visualized the models in 3D using the OpenGL rendering technique and implemented the zoom, pan, and rotate feature in our visualization models.

Ÿ Added the keyboard function to have the flexibility of repeating the entire simulation cycle, and to return to the initial position to re-run the simulation.

Ÿ Used different colors to render various size grains as well as shown the TRUE contacts in thick black bold lines and the FALSE contacts as dashed blue lines in the Delaunay network.

Ÿ In the Delaunay network simulation we see the breakage of existing contacts and formation of new contacts as the simulation progress. We also clearly identify the moving and stationary dislocations as well as de- struction of the current dislocations and formation of new ones using the VR devices in the laboratory.

Ÿ Implemented color-coding to identify the dislocations lying in different slip planes. In particular we have four different slip planes, so we have used four different colors to show the dislocations corresponding to each plane.

Ÿ In all of our models we have implemented the stereo mode using the OpenGL rendering technique to visual- ize them in 3D with a depth perception. We have used two different projectors to create one image for each eye with an offset to incorporate depth perception to the whole model. We have used the VR Devices head mounted display and the flock-of-birds tracking device to be immersed and walk around as well as inside the models.

The authors would like to take the opportunity to thank Dr. Ok Joon Kim in helping setup the Virtual Reality Devices for the demonstrations in the VRCIM Lab at WSU and Dr. Jagan Padbidri for sharing his expertise in contact mechanics.