Author

  • Benjamin Wegener

    • Ostwestfalen-Lippe University of Applied Sciences and Arts, Campusallee 12, Lemgo, Germany
    • Germany

Abstract

Recently, the analysis of communication has gained attention in experimental research. One important question is whether certain types of communication affect decisions differently than others. In this regard, Houser & Xiao (2011) present an approach for the classification of natural language messages. The primary limitation of their approach is its limited applicability to large message datasets. Therefore, Penczynski (2019) extends the methodological instruments by applying a machine learning classifier to experimental communication data. This is accompanied by the problem of a dearth of machine learning knowledge among experimenters. Hence, this paper presents an approach that employs a publicly available machine learning text analysis application. This makes it possible to analyze larger datasets based on small training datasets classified beforehand by human evaluators. As a first step, I use primary communication data reported by Charness and Dufwenberg (2006) to generate both training and test datasets. Following this approach, I am able to substantially replicate the original classification results obtained by Charness and Dufwenberg. The second step again involves messages from Charness and Dufwenberg as training data, while I take messages from a related trust game published by Deck et al. (2013) as a test, dataset. Promisingly, I am also able to replicate the classification results obtained by the external evaluators, as reported by Deck et al. The findings suggest that machine learning can be used to analyze large message datasets, both if the artificial intelligence is trained with data from the very same experiment and if it is trained with message data from a comparable experiment.

Keywords

References

  • Agranov, M. & Yariv, L. (2018). Collusion through communication in auctions, Games and Economic Behavior , 107, 93-108.
  • Ahn, L. v. & Dabbish, L. (2004). Labeling images with a computer game. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 319-326.
  • Braun, D., Hernandez-Mendez, A., Matthes, F. & Langen, M. (2017). Evaluating natural language understanding services for conversational question answering systems. Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, 174-185.
  • Charness, G. (2006). Promises and partnership. Econometrica, 74(6), 1579-1601.
  • Chen, T. & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 785-794.
  • Cohen, J. (1960). A Coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46.
  • Deck, C., Servátka, M. & Tucker, S. (2013). An examination of the effect of messages on cooperation under double-blind and single-blind payoff procedures. Experimental Economics, 16(4), 597-607.
  • Fischer, C. & Normann, H. T. (2019). Collusion and bargaining in asymmetric Cournot duopoly - An experiment. European Economic Review, 111, 360-379.
  • Hastie, T., Friedman, J. & Tibshirani, R. (2009). The elements of statistical learning: Data mining, inference, and prediction. New York: Springer.
  • Houser, D. & Xiao, E. (2011). Classification of natural language messages using a coordination game. Experimental Economics, 14(1), 1-14.
  • Huang, L. & Xiao, E. (2018). Peer effects in public support for Pigouvian taxation. Working Paper.
  • Huerta, J. (2008). Relative rank statistics for dialog analysis. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, 965-972.
  • Isaac, R. & Walker, J. (1988). Communication and free-riding behavior: The voluntary contribution mechanism. Economic Inquiry, 26(4), 585-608.
  • Ismayilov, H. & Potters, J. (2016). Why do promises affect trustworthiness, or do they? Experimental Economics, 19(2), 382-393.
  • Khan, A., Baharudin, B., Lee, L. H. & Khan, K. (2010). A review of machine learning algorithms for textdocuments classification. Journal of Advances in Information Technology, 1(1), 4-20.
  • Landis, J. & Koch, G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics , 33(2), 363-374.
  • Lundquist, T., Ellingsen, T., Gribbe, E. & Johannesson, M. (2009). The aversion to lying. Journal of Economic Behavior & Organization, 70(1-2), 81-92.
  • Mitchell, T. (1997). Machine Learning. New York: McGraw-Hill.
  • Moellers, C., Normann, H. T. & Snyder, C. M. (2017). Communication in vertical markets: Experimental evidence. International Journal of Industrial Organization, 50, 214-258.
  • Nielsen, K., Bhattacharya, P., Kagel, J. & Sengupta, A. (2019). Teams promise but do not deliver. Games and Economic Behavior, 117, 420-432.
  • Penczynski, S. P. (2019). Using machine learning for communication classification. Experimental Economics, 22(4), 1002-1029.
  • Ribeiro, M., Singh, S. & Guestrin, C. (2016). Why should I trust you? Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135-1144.
  • Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computer Surveys, 34(1), 1-47.
  • Snow, R., O'Connor, B., Jurafsky, D. & Ng, A. (2008). Cheap and fast - But is it good?: Evaluating non-expert annotations for natural language tasks. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, 254-263.
  • Uthus, D. & Aha, D. (2013). Multiparticipant chat analysis: A survey. Artificial Intelligence, (199-200), 106-121.

Downloads

No downloads have been recorded for this article yet. Figures appear here once readers begin downloading the full text.

How to Cite

Wegener, B. (2021). How to Analyze Communication Data from Laboratory Experiments Without Being a Machine Learning Specialist. Journal of Economics and Behavioral Studies, 13(1(J), 32-56. https://doi.org/10.22610/jebs.v13i1(J).3083

Share this article

License

Copyright (c) 2021 Benjamin Wegener (Author)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Author (s) should affirm that the material has not been published previously. It has not been submitted and it is not under consideration by any other journal. At the same time author (s) need to execute a publication permission agreement to assume the responsibility of the submitted content and any omissions and errors therein. After submission of revised paper in the light of suggestions of the reviewers, the editorial team  edits and formats manuscripts to bring uniformity and standardization in published material.

This work will be licensed under Creative Commons Attribution 4.0 International (CC BY 4.0) and under condition of the license, users are free to read, copy, remix, transform, redistribute, download, print, search or link to the full texts of articles and even build upon their work as long as they credit the author for the original work. Moreover, as per journal policy author (s) hold and retain copyrights without any restrictions.

Keywords

Section

Research Paper

Published in

Vol. 13 No. 1(J) (2021) Money, Communication & Experimental Behaviour View issue contents

Published

2021-04-07

Submissions

Have research to publish? Journal of Economics and Behavioral Studies welcomes submissions.

Make a Submission