Abstract
Video sharing platforms like YouTube are increasingly targeted by aggression and hate attacks. Prior work has shown how these attacks often take place as a result of "raids," i.e., organized efforts by ad-hoc mobs coordinating from third-party communities. Despite the increasing relevance of this phenomenon, however, online services often lack effective countermeasures to mitigate it. Unlike well-studied problems like spam and phishing, coordinated aggressive behavior both targets and is perpetrated by humans, making defense mechanisms that look for automated activity unsuitable. Therefore, the de-facto solution is to reactively rely on user reports and human moderation. In this paper, we propose an automated solution to identify YouTube videos that are likely to be targeted by coordinated harassers from fringe communities like 4chan. First, we characterize and model YouTube videos along several axes (metadata, audio transcripts, thumbnails) based on a ground truth dataset of videos that were targeted by raids. Then, we use an ensemble of classifiers to determine the likelihood that a video will be raided with very good results (AUC up to 94%). Overall, our work provides an important first step towards deploying proactive systems to detect and mitigate coordinated hate attacks on platforms like YouTube.
- S. Agarwal and A. Sureka. A Focused Crawler for Mining Hate and Extremism Promoting Videos on YouTube. In ACM Hypertext, 2014.Google ScholarDigital Library
- N. Aggarwal, S. Agrawal, and A. Sureka. Mining YouTube metadata for detecting privacy invading harassment and misdemeanor videos. In PST, 2014.Google ScholarCross Ref
- S. Al-Azani and E.-S. M. El-Alfy. Using word embedding and ensemble learning for highly imbalanced data sentiment analysis in short arabic text. Procedia Computer Science, 109, 2017.Google Scholar
- S. Alhabash, J. hwan Baek, C. Cunningham, and A. Hagerstrom. To comment or not to comment?: How virality, arousal level, and commenting behavior on YouTube videos affect civic behavioral intentions. Computers in Human Behavior, 51, 2015.Google Scholar
- D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473, 2014.Google Scholar
- A. Ben-David and A. Matamoros-Fernández. Hate speech and covert discrimination on social media: Monitoring the Facebook pages of extreme-right political parties in Spain. International Journal of Communication, 10, 2016.Google Scholar
- F. Benevenuto, G. Magno, T. Rodrigues, and V. Almeida. Detecting spammers on twitter. In CEAS, 2010.Google Scholar
- S. Bhattacharyya, S. Jha, K. Tharakunnel, and J. C. Westland. Data mining for credit card fraud: A comparative study. Decision Support Systems, 50(3), 2011.Google Scholar
- P. Burnap and M. L. Williams. Us and them: identifying cyber hate on Twitter across multiple protected characteristics. EPJ Data Science, 5, 2016.Google Scholar
- E. Chandrasekharan, U. Pavalanathan, A. Srinivasan, A. Glynn, J. Eisenstein, and E. Gilbert. You can't stay here: The efficacy of reddit's 2015 ban examined through hate speech. Proceedings of the ACM on Human-Computer Interaction,1(CSCW), 2017.Google ScholarDigital Library
- D. Chatzakou, N. Kourtellis, J. Blackburn, E. De Cristofaro, G. Stringhini, and A. Vakali. Hate is not Binary: Studying Abusive Behavior of #GamerGate on Twitter. In ACM Hypertext, 2017.Google ScholarDigital Library
- D. Chatzakou, N. Kourtellis, J. Blackburn, E. De Cristofaro, G. Stringhini, and A. Vakali. Mean Birds: Detecting Aggression and Bullying on Twitter. In International ACM Web Science Conference, 2017.Google ScholarDigital Library
- M. Chau and J. Xu. Mining communities and their relationships in blogs: A study of online hate groups. International Journal of Human-Computer Studies, 65(1), 2007.Google ScholarDigital Library
- J. Cheng, M. Bernstein, C. Danescu-Niculescu-Mizil, and J. Leskovec. Anyone can become a troll: Causes of trolling behavior in online discussions. In Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW), 2017.Google ScholarDigital Library
- S. Chess and A. Shaw. A conspiracy of fishes, or, how we learned to stop worrying about #GamerGate and embrace hegemonic masculinity. Journal of Broadcasting & Electronic Media, 2015.Google Scholar
- M. Conway and L. McInerney. Jihadi Video and Auto-radicalisation: Evidence from an Exploratory YouTube Study. Springer Berlin Heidelberg, 2008.Google Scholar
- M. Dadvar, R. Trieschnigg, and F. de Jong. Experts and Machines against Bullies: A Hybrid Approach to Detect Cyberbullies. In Canadian AI, 2014.Google ScholarCross Ref
- S. Datta, C. Phelan, and E. Adar. Identifying misaligned inter-group links and communities. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 2017.Google ScholarDigital Library
- E. Denton. YouTuber Jaclyn Hill Reveals She's "Scared" of Her Channel Because the Comments Are So Mean. https://bit.ly/2HXSTa4, 2015.Google Scholar
- T. G. Dietterich. Ensemble Methods in Machine Learning. In First International Workshop on Multiple Classifier Systems, 2000.Google Scholar
- D. B. Eichenberger. Speech activity detection: Application-specific tuning and context-based neural approaches. Bachelor thesis, Universitat Politècnica de Catalunya, July 2016.Google Scholar
- M. Ekman. The dark side of online activism: Swedish right-wing extremist video activism on YouTube. MedieKultur: Journal of media and communication research, 30(56), 2014.Google Scholar
- M. ElSherief, V. Kulkarni, D. Nguyen, W. Y. Wang, and E. Belding. Hate lingo: A target-based linguistic analysis of hate speech in social media. In International Conference on Web and Social Media (ICWSM), 2018.Google Scholar
- M. ElSherief, S. Nilizadeh, D. Nguyen, G. Vigna, and E. Belding. Peer to Peer Hate: Hate Speech Instigators and Their Targets. In International Conference on Web and Social Media (ICWSM), 2018.Google Scholar
- B. Feldman. Can 10,000 Moderators Save YouTube? http://nymag.com/selectall/2017/12/can-10-000-moderators-save-youtube.html, 2017.Google Scholar
- M. Fernández-Delgado, E. Cernadas, S. Barro, and D. Amorim. Do we need hundreds of classifiers to solve real world classification problems? JMLR, 15(1), 2014.Google Scholar
- P. Gerbaudo. Social media and populism: an elective affinity? Media, Culture & Society, 40(5), 2018.Google Scholar
- P. B. Gerstenfeld, D. R. Grant, and C.-P. Chiang. Hate online: A content analysis of extremist Internet sites. Analyses of social issues and public policy, 3(1), 2003.Google Scholar
- P. Geurts, D. Ernst, and L. Wehenkel. Extremely randomized trees. Machine learning, 63(1), 2006.Google Scholar
- J. Glaser, J. Dixit, and D. P. Green. Studying hate crime with the internet: What makes racists advocate racial violence? Journal of Social Issues, 58(1), 2002.Google ScholarCross Ref
- J. J. Godfrey, E. C. Holliman, and J. McDaniel. SWITCHBOARD: Telephone Speech Corpus for Research and Development. In ICASSP, 1992.Google ScholarCross Ref
- P. Gomez. YouTube and Instagram Stars Explain How to Protect Your Kids from Online Bullying. http://people.com/social-media-stars/protect-kids-online-bullying-youtube-instagram/, 2017.Google Scholar
- M. Green, A. Bobrowicz, and C. S. Ang. The lesbian, gay, bisexual and transgender community online: discussions of bullying and self-disclosure in YouTube videos. Behaviour & Information Technology, 2015.Google ScholarCross Ref
- D. W. Grigg. Cyber-aggression: Definition and concept of cyberbullying. Australian Journal of Guidance and Counselling, 20(2), 2010.Google ScholarCross Ref
- G. E. Hine, J. Onaolapo, E. De Cristofaro, N. Kourtellis, I. Leontiadis, R. Samaras, G. Stringhini, and J. Blackburn. Kek, Cucks, and God Emperor Trump: A Measurement Study of 4chan's Politically Incorrect Forum and Its Effects on the Web. In AAAI ICWSM, 2017.Google Scholar
- B.-J. P. Hsu and J. R. Glass. Iterative language model estimation: efficient data structure & algorithms. In Interspeech, 2008.Google ScholarCross Ref
- M. Hussin, S. Frazier, and J. K. Thompson. Fat stigmatization on YouTube: A content analysis. Body Image, 8(1), 2011.Google Scholar
- A. Israni, S. Erete, and C. L. Smith. Snitches, Trolls, and Social Norms: Unpacking Perceptions of Social Media Use for Crime Prevention. In Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW), 2017.Google ScholarDigital Library
- A. K. Jain, R. P. W. Duin, and J. Mao. Statistical pattern recognition: A review. IEEE Transactions on pattern analysis and machine intelligence, 22(1), 2000.Google Scholar
- S. Jhaver, S. Ghoshal, A. Bruckman, and E. Gilbert. Online harassment and content moderation: The case of blocklists. ACM Transactions on Computer-Human Interaction (TOCHI), 25(2), 2018.Google ScholarDigital Library
- L. Jönson. Flaming motivation in YouTube users as a function of the traits Disinhibition seeking, Assertiveness and Anxiety? Technical report, University of Twente, 2013.Google Scholar
- A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In IEEE Conference on Computer Vision and Pattern Recognition, 2015.Google ScholarCross Ref
- I. Kayes, N. Kourtellis, D. Quercia, A. Iamnitchi, and F. Bonchi. The Social World of Content Abusers in Community Question Answering. In WWW, 2015.Google ScholarDigital Library
- S. Kumar, W. L. Hamilton, J. Leskovec, and D. Jurafsky. Community Interaction and Conflict on the Web. In The Web Conference (WWW), 2018.Google Scholar
- K. Kwon and A. Gruzd. Is Aggression Contagious Online? A Case of Swearing on Donald Trump's Campaign Videos on YouTube. In Hawaii International Conference on System Sciences, 2017.Google ScholarCross Ref
- K. H. Kwon and A. Gruzd. Is offensive commenting contagious online? Examining public vs interpersonal swearing in response to Donald Trump's YouTube campaign videos. Internet Research, 2017.Google ScholarCross Ref
- P. G. Lange. Commenting on YouTube rants: Perceptions of inappropriateness or civic engagement? Journal of Pragmatics, 2014.Google ScholarCross Ref
- J. Luque, C. Segura, A. Sánchez, M. Umbert, and L. A. Galindo. The Role of Linguistic and Prosodic Cues on the Prediction of Self-Reported Satisfaction in Contact Centre Phone Calls. In Proc. Interspeech 2017, 2017.Google ScholarCross Ref
- S. K. Maity, A. Chakraborty, P. Goyal, and A. Mukherjee. Opinion Conflicts: An Effective Route to Detect Incivility in Twitter. Proc. ACM Hum.-Comput. Interact., 2(CSCW), 2018.Google ScholarDigital Library
- S. Marathe and K. P. Shirsat. Approaches for Mining YouTube Videos Metadata in Cyber bullying Detection. International Journal of Engineering Research & Technology, 4, 2015.Google Scholar
- P. J. Moor, A. Heuvelman, and R. Verleur. Flaming on YouTube. Computers in Human Behavior, 26(6), 2010.Google Scholar
- B. Moser. How YouTube Became the Worldwide Leader in White Supremacy. https://newrepublic.com/article/144141/youtube-became-worldwide-leader-white-supremacy, 2017.Google Scholar
- S. Nilizadeh, F. Labrèche, A. Sedighian, A. Zand, J. Fernandez, C. Kruegel, G. Stringhini, and G. Vigna. POISED: Spotting Twitter Spam Off the Beaten Paths. In ACM SIGSAC Conference on Computer and Communications Security (CCS), 2017.Google ScholarDigital Library
- C. Nobata, J. Tetreault, A. Thomas, Y. Mehdad, and Y. Chang. Abusive Language Detection in Online User Content. In WWW, 2016.Google ScholarDigital Library
- Nobullying.com. Youtube Bullying. https://nobullying.com/youtube-bullying/, 2018.Google Scholar
- A. O'Connor. Suicide Draws Attention to Gay Bullying. https://well.blogs.nytimes.com/2011/09/21/suicide-of-gay-teenager-who-urged-hope/, 2011.Google Scholar
- A. Oksanen, D. Garcia, A. Sirola, M. Näsi, M. Kaakinen, T. Keipi, and P. Räsänen. Pro-Anorexia and Anti-Pro-Anorexia Videos on YouTube: Sentiment Analysis of User Responses. Journal of Medical Internet Research, 17, 2015.Google Scholar
- A. Olteanu, C. Castillo, J. Boy, and K. R. Varshney. The effect of extremist violence on hateful speech online. In International Conference on Web and Social Media (ICWSM), 2018.Google Scholar
- M. Oppenheim. Jessi Slaughter on becoming a meme and falling victim to trolls after infamous YouTube video. https://ind.pn/2IjeRnB, 2016.Google Scholar
- J. Y. Park, J. Jang, A. Jaimes, C.-W. Chung, and S.-H. Myaeng. Exploring the User-generated Content (UGC) Uploading Behavior on Youtube. In WWW Companion, 2014.Google ScholarDigital Library
- S. Perez. YouTube promises to increase content moderation and other enforcement staff to 10K in 2018. https://goo.gl/2te7HV, 2018.Google Scholar
- D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely. The Kaldi Speech Recognition Toolkit. In IEEE Workshop on Automatic Speech Recognition and Understanding, 2011.Google Scholar
- A. Romano. Reddit just banned one of its most toxic forums. But it won't touch The_Donald. https://www.vox.com/culture/2017/11/13/16624688/reddit-bans-incels-the-donald-controversy, 2017.Google Scholar
- C. Rossow. Amplification Hell: Revisiting Network Protocols for DDoS Abuse. In Network and Distributed Systems Security Symposium (NDSS), 2014.Google Scholar
- J. Salminen, H. Almerekhi, M. Milenkovic, S.-g. Jung, J. An, H. Kwak, and B. J. Jansen. Anatomy of online hate: developing a taxonomy and machine learning models for identifying and classifying hate in online news media. In International Conference on Web and Social Media (ICWSM), 2018.Google Scholar
- M. Sewell. Ensemble learning. RN, 11(02), 2008.Google Scholar
- P. Sobkowicz and A. Sobkowicz. Dynamics of hate based Internet user networks. The European Physical Journal B, 73(4), 2010.Google Scholar
- D. Soni and V. K. Singh. See No Evil, Hear No Evil: Audio-Visual-Textual Cyberbullying Detection. Proc. ACM Hum.-Comput. Interact., 2(CSCW), 2018.Google ScholarDigital Library
- R. Speer, J. Chin, and C. Havasi. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. In AAAI, 2017.Google ScholarDigital Library
- N. Statt. YouTube is facing a full-scale advertising boycott over hate speech. https://www.theverge.com/2017/3/24/15053990/google-youtube-advertising-boycott-hate-speech, 2017.Google Scholar
- G. Stringhini, P. Mourlanne, G. Jacob, M. Egele, C. Kruegel, and G. Vigna. Evilcohort: detecting communities of malicious accounts on online services. In USENIX Security Symposium, 2015.Google ScholarDigital Library
- A. Sureka, P. Kumaraguru, A. Goyal, and S. Chhabra. Mining YouTube to Discover Extremist Videos, Users and Hidden Communities. In AIRS, 2010.Google Scholar
- O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: Lessons learned from the 2015 MSCOCO image captioning challenge. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2016.Google Scholar
- J. Vitak, K. Chadha, L. Steiner, and Z. Ashktorab. Identifying Women's Experiences With and Strategies for Mitigating Negative Effects of Online Harassment. In Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW), 2017.Google ScholarDigital Library
- G. Wang, J. Hao, J. Ma, and H. Jiang. A comparative assessment of ensemble learning for credit scoring. Expert systems with applications, 38(1), 2011.Google Scholar
- A. Weaver, A. Zelenkauskaite, and L. Samson. The (Non)Violent World of Youtube: Content Trends in Web Video. Journal of Communication, 62(6), 2012.Google ScholarCross Ref
- D. H. Wolpert. Stacked generalization. Neural networks, 5(2), 1992.Google Scholar
- K. Yurieff. Google's CEO knows YouTube must do better at policing hate. https://edition.cnn.com/2019/06/17/tech/youtube-lgbt-google-ceo-sundar-pichai/index.html, 2019.Google Scholar
- S. Zannettou, B. Bradlyn, E. De Cristofaro, H. Kwak, M. Sirivianos, G. Stringini, and J. Blackburn. What is gab: A bastion of free speech or an alt-right echo chamber. In CyberSafety, 2018.Google ScholarDigital Library
- J. Zhang, C. Danescu-Niculescu-Mizil, C. Sauper, and S. J. Taylor. Characterizing Online Public Discussions through Patterns of Participant Interactions. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 2018.Google ScholarDigital Library
Recommendations
The Doppelgänger Bot Attack: Exploring Identity Impersonation in Online Social Networks
IMC '15: Proceedings of the 2015 Internet Measurement ConferencePeople have long been aware of malicious users that impersonate celebrities or launch identity theft attacks in social networks. However, beyond anecdotal evidence, there have been no in-depth studies of impersonation attacks in today's social networks. ...
Consequences of Connectivity: Characterizing Account Hijacking on Twitter
CCS '14: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications SecurityIn this study we expose the serious large-scale threat of criminal account hijacking and the resulting damage incurred by users and web services. We develop a system for detecting large-scale attacks on Twitter that identifies 14 million victims of ...
Comments