Gesture and Three-Dimensional Visualization
Kinnari Atit¹, Barbara Dutrow², Carol Ormand³, and Thomas F. Shipley¹
¹Temple University, Philadelphia, PA
²Louisiana State University, Baton Rouge, LA
³Carleton College, Northfield, MN
Research has shown that high spatial ability is related to achievement in the Science, Technology, Engineering, and Mathematics (STEM) disciplines (Shea, Lubinski, & Benbow, 2001). One skill that is pervasive in all of these fields is visualizing objects in three-dimensions (e.g. Kali & Orion, 1996; Keehner, Hegarty, Cohen, Khooshabeh, & Montello, 2008). For example, petroleum geologists envision the three-dimensional geometry of complex fault systems, chemists visualize the three-dimensional structures of complex molecules (e.g. Stull, Hegarty, Dixon, & Stieff, 2012), and engineers create three-dimensional structures from two-dimensional sketches and drawings (Sorby, 2009). Experts practicing and teaching in these fields report that mentally picturing an object in three-dimensions is a skill students struggle with (e.g. Riggs, Lieder, & Balliet, 2009; Rapp, Culpepper, Kirkby, & Morin, 2007). Here, we will discuss how gesturing may help with three-dimensional thinking.
People use gesture when communicating spatial information and for individual problem solving (Alibali, 2005). Common spatial activities, such as giving directions, often include gesture (e.g. Lavergne & Kimura, 1987), and entire domains of science (e.g. geology) that require communicating and understanding complex spatial ideas often make use of gesture (Liben, Christensen, & Kastens, 2010). Gestures are useful when thinking about three-dimensional spatial relationships for two reasons. First, the hand movement itself occurs in three-dimensions, and therefore allows one to represent the spatial properties of one or more objects in three-dimensions (Roth, 2000; Pozzer-Ardenghi & Roth, 2006). This multi-dimensionality makes gestures well suited to portray shape, form, space, and position. Second, gestures can show movement in real time in a way that circumvents the linearity of speech (Roth & Lawless, 2002), allowing one to represent the movement of one or more objects.
Those that work within the STEM disciplines generally use gesture because they have to reason about objects and relations that are not visible to them. For instance, expert chemists use gesture to represent the structure and movement of molecules too small to see. Becvar, Hollan, and Hutchins' (2005) observations of a biochemistry lab showed that experts used individual fingers of one hand to represent the relative locations of the loops that make up a protein molecule. Stieff and Raje (2010) found that expert chemists use gesture to visualize alternative 3D perspectives of molecules shown in diagrams. Structural geologists likely use gesture because the structures of interest are too big to see in their entirety, are occluded by other structures, or no longer exist (Frodemen, 1995). Usually only one face of a structure is visible on an outcrop (providing two-dimensional information): in this situation experts use gesture to represent information about the three-dimensional geometry of the geologic structure.
Along with providing a visual representation for objects that are not entirely visible within the space, research has shown that gesturing in general provides many benefits. First, gestures can provide a medium through which the speaker can form and articulate new ideas (Crowder, 1996; Roth, 2000). Crowder (1996) observed sixth graders during science lessons and found that they gestured differently depending on their discourse. When the students explained concepts they already knew, their gestures were timed with their speech. When they explained concepts that they were in the process of understanding, they used gesture that foreshadowed the ideas that would eventually be articulated in speech (Crowder, 1996). Second, the speaker's gestures can reflect a readiness to learn, possibly indicating to listeners that instruction at that moment would be beneficial. Church and Goldin-Meadow (1986) asked five to eight year old children to make conservation judgments and then explain their rationale. Those who provided different information in gesture than they did in speech were found to be in a transitional state and were more receptive to training than those where speech and gesture matched (Church & Goldin-Meadow, 1986). And third, gesturing can effectively reduce the speaker's cognitive load. Goldin-Meadow, Nusbaum, Kelly, and Wagner (2001) asked participants to remember lists of letters or words while they explained how to solve different math problems. Participants remembered significantly more items when gesturing than when not gesturing, indicating that gesturing may have freed up some cognitive resources during the math explanation task, allowing for more resources to be used for the memory task (Goldin-Meadow, et al., 2001).
I will present preliminary results from two large-scale studies that will investigate the use of gesture in the communication and understanding of three-dimensional information. The studies focus on two tasks that geoscience professors report challenge students: reading and interpreting topographic maps, and the use of Miller Indices to note the location and direction of planes in crystal lattices. Additionally, results from these studies may inform members of STEM disciplines about three things: 1) how best to use gesture to communicate three-dimensional information, 2) more broadly, how gestures can be used to effectively teach these and possibly other spatially complex concepts in the classroom, and 3) independent of the importance of gesture for communication, whether gesture can help an individual solve problems involving the visualization of three-dimensional information.
Alibali, M. W. (2005). Gesture in spatial cognition: expressing, thinking, and communicating about spatial information. Spatial Cognition and Computation, 5, 307-331.
Becvar, L. A., Hollan, J., & Hutchins, E. (2005). Hands as molecules: Representational gestures used for developing theory in a scientific laboratory. Semiotica, 156, 89-112.
Church, R. B., & Goldin-Meadow (1986). The mismatch between gesture and speech as an index of transitional knowledge. Cognition, 23, 43-71.
Crowder, E. M. (1996). Gestures at work in sense-making science talk. The Journal of the Learning Sciences, 5, 173-208.
Frodeman, R. (1995). Geological reasoning: Geology as an interpretive and historical science, GSA Bulletin, 107, 960-968.
Goldin-Meadow, S., Nusbaum, H., Kelly, S. D., & Wagner, S. (2001). Explaining math: Gesturing lightens the load. Psychological Science, 12, 516-522.
Kali, Y., & Orion, N. (1996). Spatial abilities of high-school students in the perception of geologic structures. Journal of Research in Science Teaching, 33, 369-391.
Keehner, M., Hegarty, M., Cohen, C., Khooshabeh, P., & Montello, D. R. (2008). Spatial reasoning with external visualizations: What matters is what you see, not whether you interact. Cognitive Science, 32, 1099-1132.
Lavergne, J., & Kimura, D. (1987). Hand movement asymmetry during speech: No effect of speaking topic. Neuropsychologia, 25, 689-693.
Liben, L. S., Christensen, A. E., & Kastens, K. A. (2010). Gestures in geology: The roles of spatial skills, expertise, and communicative context. Spatial Cognition VII, 95-111.
Pozzer-Ardenghi, L., & Roth, W. M. (2006). On performing concepts during science lectures. Science Education, 91, 96-113.
Rapp, D. N., Culpepper, S. A., Kirkby, K., & Morin, P. (2007). Fostering students' comprehension of topographic maps. Journal of Geoscience Education, 55(1), 5.
Riggs, E. M., Lieder, C. C., & Balliet, R. (2009). Geologic problem solving in the field: Analysis of field navigation and mapping by advanced undergraduates. Journal of
Geoscience Education, 57(1), 48-63.
Roth, W. M. (2000). From gesture to scientific language. Journal of Pragmatics, 32, 1683-1714.
Roth, W. M., & Lawless, D. (2002). Science, culture, and the emergence of language. Science Education, 86, 368-385.
Shea, D. L., Lubinski, D., & Benbow, C. P. (2001). Importance of assessing spatial ability in intellectually talented young adolescents: A 20-year longitudinal study. Journal of Educational Psychology, 93, 604-614.
Sorby, S. A. (2009). Educational Research in Developing 3-D Spatial Skills for Engineering Students. International Journal of Science Education, 31(3), 459-480.
Stieff, M., & Raje, S. (2010). Expert algorithmic and imagistic problem solving strategies in advanced chemistry. Spatial Cognition & Computation, 10(1), 53-81.
Stull, A. T., Hegarty, M., Dixon, B., & Stieff, M. (2012). Representational Translation
With Concrete Models in Organic Chemistry. Cognition and Instruction, 30(4), 404-434.
AAPG Search and Discovery Article #120140© 2014 AAPG Hedberg Conference 3D Structural Geologic Interpretation: Earth, Mind and Machine, June 23-27, 2013, Reno, Nevada