Triadic Synchrony: Application of Multiple Wavelet Coherence to a Small Group Conversation

By applying multiple wavelet coherence (MWC) to data from human body movements in triadic interaction, this study quantified triadic synchrony, rhythmic similarity among three interactants. Thirty-nine Japanese undergraduates were randomly assigned in a triad, and engaged in a brain-storming task. Triadic synchrony was quantified by calculating MWC to the time-series movement data collected by Kinect v2 sensor. The existence of synchrony was statistically tested by using a pseudo-synchrony paradigm. Results showed that the averaged value of MWC was higher in the experimental participant trio than in those of the pseudo trio in the frequency band of 0.5 - 1 Hz. The result supports the possible utility of applying multiple wavelet coherence to evaluate triadic synchrony in a small group interaction.

• Multiple Wavelet Coherence
• Nonverbal Behavior
• Synchrony
• Small Group
• Automated Method

1. Introduction

Synchrony has attracted the attention of psychology and communication researchers. Past work has revealed that individuals got synchronized or unsynchronized in their body posture and/or movement through the interaction. In previous studies, synchrony tends to be seemed as the subtype of interpersonal coordination. Interpersonal coordination, by definition, is when two or more individuals coordinate their behavior in a time series. The relationship between each time series can be analyzed in either time- or frequency-domains [1] ; thus, coordination can be considered as a time- and/or frequency-domain phenomenon. In the time-domain, coordination is interpreted as the extent to which the behavior co-occurred or the amount of behavior that is similar between the individuals within a predetermined time window. In comparison, in the frequency-domain, coordination is interpreted as the amount of similarity at each frequency component (i.e., cross-spectral coherence). Each instance of time- or frequency-domain coordination seems to correspond approximately to the work of [2] who differentiated interpersonal coordination into two facets: behavior matching and synchrony. The former one is currently known as behavioral mimicry [3] and lots of empirical findings are accumulated; for a review see [4] [5] . On the other hand, synchrony, as a frequency-domain coordination, has not got sufficient attention in psychology and communication research.

Over time, synchrony research, employing the analysis method from physics [6] , has revealed that temporal coordination occurs between individuals [7] . Some synchrony research has focused on a similar movement between individuals such as swinging pendulums and rocking in rocking chairs, e.g., [7] [8] ; however, the essence of synchrony is rhythm and timing. So, synchrony can be achieved even with different behaviors between individuals. In order to evaluate synchrony in the frequency domain (i.e., the similarity of rhythm and timing), spectrum analysis including Fourier and wavelet approach has been employed.

1.1. Evaluating Synchrony: Wavelet Transform

In the early stage of nonverbal research, microanalysis analyzing films of social interactions frame by frame was employed to generate time-series movement data [9] [10] . This measuring process, unfortunately, has been resource intensive. Coding behaviors is time-consuming and painstaking in itself and requires establishing reliability among coders. To address this problem, some recent studies have utilized automated techniques to generate time-series movement data; they use a depth sensor, Kinect (Microsoft) [11] , or employ video-tracking techniques [12] - [15] . Behavioral data acquired by these techniques would be less costly as well as highly reliable.

After obtaining time-series data, a spectrum analysis is conducted. Many of the previous studies employed Fourier transform and calculated coherence by using two time-series [14] , which is interpreted as the extent of synchrony. A coherence of 1 reflects a perfect correlation between the two movements, and 0 reflects no correlation [7] [8] . However, the Fourier transform that has been used in previous studies has a serious practical limitation: it assumes a stable frequency or repetitive pattern during the entire interaction [1] [16] , which is difficult to be applied to our daily conversation.

As a potent alternative to the Fourier transform, the wavelet approach; it does not require stationarity in each time series, receives attention [13] . Several studies in motor coordination have evaluated coordination by using cross-wavelet analysis. Reference [17] , for instance, collected body movement data in dance settings, and found that the cross-wavelet coherence of the trained dancers was significantly higher than that of the non-dancers, indicating that the dancers achieved a higher level of coordination with their confederate. Reference [18] performed a rhythmical sway task in the sagittal plane, and found that a light fingertip contact, i.e., haptic contact, increased coherence. In settings with more socialization, [19] used a periodic interaction task (knock-knock joking task), and calculated wavelet coherence to evaluate rhythmic similarity between two individuals. Not employing specific rhythmic task, [13] illustrated that synchrony could be evaluated by using wavelet coherence even in unstructured conversation situation.

1.2. Current Study: Evaluating Triadic Synchrony via Multiple Wavelet Coherence

Considering our daily interaction, some are dyadic, others include more than three individuals. However, previous studies employing the wavelet approach [13] [17] - [19] have shed light on the dyadic interaction; it is still unquestioned whether synchrony can be captured in a small group interaction. To handle this issue, as a first step, this study focused on triadic interaction, and tried to measure the overall synchrony from three individuals engaging a brainstorming task, a well-known small group task in an office meeting situation. As an index of triadic synchrony, I calculate a multiple wavelet coherence (MWC) [20] which seeks the resulting coherence of multiple independents on a dependent, like as multiple correlation. First, following [21] , the continuous wavelet transform (CWT) is defined as:

The CWT of a time series (x_n, n = 1, ∙∙∙, N) with uniform time steps δt, is defined as the convolution of x_n with the scaled and normalized wavelet. Also, following [20] [21] , the squared wavelet coherence (WTC), a tool for identifying possible relationships between two processes by searching frequency bands and time intervals during which they covariate, can be defined as:

Then, following [20] , MWC, like multiple correlation that is capable of seeking the resulting coherence of multiple independents on a dependent, can be described as:

To test the existence of synchrony, [2] proposed the pseudo-synchrony experimental paradigm. In this paradigm, the video clip of dyadic interaction partners (i.e., genuine pair) are isolated and re-combined in a random order. The synchrony score of these virtual data (i.e., pseudo pair) is compared to the genuine pair. In this study, our focus is on a triadic conversation; thus, movement data from genuine trio are isolated and re-combined in a random order to generate pseudo trios. The score of MWC is compared between the genuine and pseudo trio. We hypothesized that the extent of synchrony, i.e., MWC, would be higher in the genuine pairs than in the pseudo pairs.

2. Method

2.1. Ethics Statement

All procedures performed in studies involving human participants were in accordance with the ethical standards of the Japanese Society of Social Psychology. Before data collection, all the participants completed a consent form and approved data usage.

2.2. Participants

Thirty-nine Japanese undergraduates (13 females, Mean age = 20.34, SD = 1.10) participated in exchange for extra course credit. Each participant was randomly assigned to a mixed-gender trio. Their familiarity among participants was not identical; some participants knew each other before the interaction task, others did not know at all, which seemed to have a potent influence on the strength of synchrony, after their conversation they were asked to complete questionnaires regarding their familiarity with one another. Whether the familiarity was correlated to the extent of synchrony, i.e., MWC, was examined.

2.3. Procedure

First, participants were seated to each desk at the four corners of a room, and completed a consent form, then they moved to another seat that was placed in a fanwise formation (Figure 1); the settings including the distances between seats were decided based on a preliminary phase. They were instructed to engage in a brainstorming task, generating ideas for usage of a wire hanger, for six minute. Their body movement in the conversation was sensed by using Kinect v2 (Microsoft) placed 250 cm away and to the front side of the participants.

2.4. Generating Time-Series Movement Data

Time-series movement data was extracted with Brekel Pro Body v2 (Brekel). This software, activated together with Kinect v2, provided coordinate information saved as local coordinates, or in other words relative to their parent; therefore there is no need to conduct calibration for each participant. Sampling rate was 10 Hz. For each participant, coordinate points for the hands and head were captured in chronological order; the coordinate point changes between frames were calculated to compose the movement of each body part. The movement of hands and nose were added together.

Figure 1

Experimental setting

2.5. Making Virtual Data

To evaluate the significance of the extent of triadic synchrony in the genuine trio, a baseline is needed. Following the pseudo-synchrony experimental paradigm [2] , a virtual dataset was generated. Three time-series data from the genuine trio were isolated and re-combined; in each trio, the individual at the center chair was fixed (n = 13), and the other two individuals seated at the two sides were shuffled in a random order to make pseudo trios. The extent of triadic synchrony of the pseudo trios was assessed in the same manner.

2.6. Evaluating Triadic Synchrony via MWC

To evaluate triadic synchrony via MWC, Matlab 2015a (Mathworks), the wavelet toolbox [21] , and the code for calculating multiple wavelet coherence [20] were used. MWC seeks the resulting coherence of multiple independents on a dependent [20] . As a dependent, the movement of individual seated at the center chair was chosen; then the movement of other two individuals seated at the two sides was used as multiple independents (Figure 2); A1 - A3 represents each participant’s wavelet power spectrum (WPS) in a group, and B1 represents MWC of the group. Regarding parameters of wavelet transformation, default parameters of [21] were employed except for the number of order; following [1] , the order was set to eight. Morlet was used as the mother wavelet. Cone of influence (COI) area was not included for subsequent analysis [21] .

We used a MWC value under 4 Hz (over 0.25 period) because our participants’ unstructured conversation was not so active or fast [13] . The average MWC under 4 Hz across the time line was standardized by using a Fisher-Z transformation before statistical analyses. In addition, the MWC of frequency band around 0.5 Hz (i.e., 0.2 - 0.5 Hz, 0.5 - 1 Hz), characterizing unstructured conversations [13] , was calculated and compared between the genuine and pseudo trio.

3. Results

First, whether familiarity among group members was correlated to the MWC was examined. Each participant answered familiarity toward other two participant (e.g., participant A provided the familiarity score toward participant B and C), which was averaged to calculate the individual score of familiarity. Then the familiarity score from the three members in a group was averaged with respect to each group (n = 13) to calculate the group score of familiarity. The analysis of correlation indicated that the group score of familiarity was not significantly correlated to the average MWC under 4 Hz, 0.2 - 0.5 Hz, and 0.2 - 0.5 Hz (r = −0.11, −0.15, and −0.14, respectively; all ps > 0.63). Therefore, the influence of familiarity on MWC was not considered in subsequent analyses.

We compared the average value of MWC between the genuine and pseudo trios. The result of separate t-tests indicated that there was no significant difference between the genuine trios (M = 0.405, SD = 0.026) and the pseudo trios (M = 0.400, SD = 0.029) in the average MWC under 4 Hz throughout the time line (t(23.56) = 0.43, p = 0.67, d = 0.17). Also, the average MWC of 0.2 - 0.5 Hz in the genuine trios (M = 0.382, SD = 0.029) and the pseudo trios (M = 0.381, SD = 0.031) was not significantly different (t(23.78) = 0.05, p = 0.96, d = 0.01). On the other hand, the average MWC of 0.5 - 1 Hz was higher in the genuine trios (M = 0.394, SD = 0.015) than in the pseudo trios (M = 0.373, SD = 0.027), and this difference was statistically significant (t(19.04) = 2.48, p = 0.02, d = 0.97).

Figure 2

Example of MWC

4. Conclusions and Implications

This study focused on a triadic interaction, and body movement data of each member was extracted by employing an automated technique, using Kinect v2 (Microsoft) and Brekel Pro Body v2 software (Brekel). Application of multiple wavelet coherence to the triadic movement data indicated that the genuine trio who were engaged in a brainstorming task was more synchronized in the frequency band of 0.5 - 1 Hz than the pseudo trio consisting of virtual data, which supported the hypothesis and demonstrated that triadic synchrony could be captured by employing multiple wavelet approach. However, the statistically significant difference was seen in the frequency band of 0.5 - 1 Hz; there was no significant difference in overall frequency band under 4 Hz, and slower frequency band of 0.2 - 0.5 Hz. These results provide three implications.

First, findings of this study could extend the field of synchrony and/or interpersonal coordination research. Interpersonal coordination was observed at various levels of communication behavior [2] [3] [7] - [10] [13] . However, previous studies have usually shed light on the dyadic interaction, it is still unquestioned whether synchrony can be captured in a triadic interaction, a small group interaction. This study focused on a small group interaction engaging a brainstorming task; there would be other cases of triadic interaction: a family interaction, i.e., mother-father-infant interaction [22] [23] , or a clinical interaction, i.e., physician-patient-companion interaction [24] . The results of this study illustrated that triadic synchrony, in this case rhythmic similarity, could be seen even in an unstructured triadic interaction via the multiple wavelet approach. This adds new insight regarding rhythm into communication research focusing a triadic interaction in our daily settings.

Second, this study supported the validity of automated techniques to extract body movement data in social interaction situation that was studied by nonverbal and communication researchers. In the early stage of nonverbal research, generating time-series movement data or coding behaviors has been resource intensive, or even painstaking, which hindered the theoretical and/or practical development of (nonverbal) behavioral research on a group interaction. Some recent studies began to utilize automated techniques to generate time-series movement data [11] - [15] . These novel techniques could reduce the cost of conducting nonverbal research, which will enable us to obtain a better understanding of small group interaction from the perspective of body movement and nonverbal behavior.

Finally, as a limitation of this study, I need to note that only the specific frequency band (i.e., 0.5 - 1 Hz) was significantly different from the virtual data. The previous study illustrated that synchrony in an unstructured conversation was characterized around 0.5 Hz; the value of cross wavelet coherence was increased around 0.5 Hz. This study also indicated that the group members were more synchronized in the frequency band of 0.5 - 1 Hz. However, in the overall (i.e., under 4 Hz) nor the slower frequency band (i.e., 0.2 - 0.5 Hz), there was no significant difference between the genuine and pseudo trios. These may be caused by the experimental setting; this study conducted a brain storming task, which results in requiring participants to behave actively. Although participants, in a three-person group, could not interact too fast, they could not talk together slowly; they were required to generate ideas as many as they possible. Thus, the frequency band of 0.5 - 1 Hz became discriminative. However, this interpretation remains a matter of speculation because it is not clear what the frequency band around 0.5 Hz exactly represent. To develop synchrony research, further examination employing the (multiple) wavelet approach would be needed.

Acknowledgements

This research was supported by Japan Society for the Promotion of Science (Grant-in-Aid for Young Scientists (B); 15K17259).

References

1. Issartel, J., Marin, L., Gaillot, P., Bardainne, T. and Cadopi, M. (2006) A Practical Guide to Time—Frequency Analysis in the Study of Human Motor Behavior: The Contribution of Wavelet Transform. Journal of Motor Behavior, 38, 139-159. http://dx.doi.org/10.3200/JMBR.38.2.139-159

2. Bernieri, F.J. and Rosenthal, R. (1991) Interpersonal Coordination: Behavior Matching and Interactional Synchrony. In: Feldman, R.S. and Rime, B., Eds., Fundamentals of Nonverbal Behavior, Cambridge University Press, Cambridge, 401-432.

3. Chartrand, T.L. and Bargh, J.A. (1999) The Chameleon Effect: The Perception-Behavior Link and Social Interaction. Journal of Personality and Social Psychology, 76, 893-910. http://dx.doi.org/10.1037/0022-3514.76.6.893

4. Chartrand, T.L. and Lakin, J.L. (2013) The Antecedents and Consequences of Human Behavioral Mimicry. Annual Review of Psychology, 64, 285-308. http://dx.doi.org/10.1146/annurev-psych-113011-143754

5. Lakin, J.L. (2013) Behavioral Mimicry and Interpersonal Synchrony. In: Hall, J.A. and Knapp, M.L., Eds., Nonverbal Communication, De Gruyter Mouton, Berlin, 539-576. http://dx.doi.org/10.1515/9783110238150.539

6. Pikovsky, A., Rosenblum, M. and Kurths, J. (2003) Synchronization: A Universal Concept in Nonlinear Sciences. Cambridge University Press, Cambridge.

7. Schmidt, R.C. and O’Brien, B. (1997) Evaluating the Dynamics of Unintended Interpersonal Coordination. Ecological Psychology, 9, 189-206. http://dx.doi.org/10.1207/s15326969eco0903_2

8. Richardson, M.J., Marsh, K.L., Isenhower, R., Goodman, J. and Schmidt, R.C. (2007) Rocking Together: Dynamics of Intentional and Unintentional Interpersonal Coordination. Human Movement Science, 26, 867-891. http://dx.doi.org/10.1016/j.humov.2007.07.002

9. Condon, W.S. and Ogston, W.D. (1966) Sound Film Analysis of Normal and Pathological Behavior Patterns. Journal of Nervous and Mental Disease, 143, 338-347. http://dx.doi.org/10.1097/00005053-196610000-00005

10. Condon, W.S. and Ogston, W.D. (1967) A Segmentation of Behavior. Journal of Psychiatric Research, 5, 221-235. http://dx.doi.org/10.1016/0022-3956(67)90004-0

11. Frauendorfer, D., Mast, M.S., Nguyen, L. and Gatica-Perez, D. (2014) Nonverbal Social Sensing in Action: Unobtrusive Recording and Extracting of Nonverbal Behavior in Social Interactions Illustrated with a Research Example. Journal of Nonverbal Behavior, 38, 231-245. http://dx.doi.org/10.1007/s10919-014-0173-5

12. Fujiwara, K. and Daibo, I. (2014) The Extraction of Nonverbal Behaviors: Using Video Images and Speech-Signal Analysis in Dyadic Conversation. Journal of Nonverbal Behavior, 38, 377-388. http://dx.doi.org/10.1007/s10919-014-0183-3

13. Fujiwara, K. and Daibo, I. (2016) Evaluating Interpersonal Synchrony: Wavelet Transform toward an Unstructured Conversation. Frontiers in Psychology, 7, 516. http://dx.doi.org/10.3389/fpsyg.2016.00516

14. Schmidt, R.C., Morr, S., Fitzpatrick, P. and Richardson, M.J. (2012) Measuring the Dynamics of Interactional Synchrony. Journal of Nonverbal Behavior, 36, 263-279. http://dx.doi.org/10.1007/s10919-012-0138-5

15. Tschacher, W., Rees, G.M. and Ramseyer, F. (2014) Nonverbal Synchrony and Affect in Dyadic Interactions. Frontiers in Psychology, 5, 1323. http://dx.doi.org/10.3389/fpsyg.2014.01323

16. Issartel, J., Barainne, T., Gaillot, P. and Marin, L. (2014) The Relevance of the Cross-Wavelet Transform in the Analysis of Human Interaction—A Tutorial. Frontiers in Psychology, 5, 1566. http://dx.doi.org/10.3389/fpsyg.2014.01566

17. Washburn, A., DeMarco, M., de Vries, S., Ariyabuddhiphongs, K., Schmidt, R.C., Richardson, M.J. and Riley, M.A. (2014) Dancers Entrain More Effectively than Non-Dancers to Another Actor’s Movements. Frontiers in Human Neuroscience, 8, 800. http://dx.doi.org/10.3389/fnhum.2014.00800

18. Sofianidis, G., Hatzitaki, V., Grouios, G., Johannsen, L. and Wing, A. (2012) Somatosensory Driven Interpersonal Synchrony during Rhythmic Sway. Human Movement Science, 31, 553-566. http://dx.doi.org/10.1016/j.humov.2011.07.007

19. Schmidt, R.C., Nie, L., Franco, A. and Richardson, M.J. (2014) Bodily Synchronization Underlying Joke Telling. Frontiers in Human Neuroscience, 8, 633. http://dx.doi.org/10.3389/fnhum.2014.00633

20. Ng, E.K.W. and Chan, J.C.L. (2012) Geophysical Applications of Partial Wavelet Coherence and Multiple Wavelet Coherence. Journal of Atmospheric and Ocean Technology, 29, 1845-1853. http://dx.doi.org/10.1175/JTECH-D-12-00056.1

21. Grinsted, A., Moore, J.C. and Jevrejeva, S. (2004) Application of the Cross Wavelet Transform and Wavelet Coherence to Geophysical Time Series. Nonlinear Processes in Geophysics, 11, 561-566. http://dx.doi.org/10.5194/npg-11-561-2004

22. Gjerde, P.F. (1986) The Interpersonal Structure of Family Interaction Settings: Parent-Adolescent Relations in Dyads and Triads. Developmental Psychology, 22, 297-304. http://dx.doi.org/10.1037/0012-1649.22.3.297

23. Lindsey, E.W. and Caldera, Y.M. (2006) Mother-Father-Child Triadic Interaction and Mother-Child Dyadic Interaction: Gender Differences within and between Contexts. Sex Roles, 55, 511-521. http://dx.doi.org/10.1007/s11199-006-9106-z

24. Karnieli-Miller, O., Werner, P., Neufeld-Kroszynski, G. and Eidelman, S. (2012) Are You Talking to Me?! An Exploration of the Triadic Physician-Patient-Companion Communication within Memory Clinics Encounters. Patient Education and Counseling, 88, 381-390. http://dx.doi.org/10.1016/j.pec.2012.06.014