Organization
Activefence
Description
This report aims to help Trust & Safety teams better understand the challenges presented by the GenAI ecosystem, by providing: • A primer on GenAI functionality and its impact on Trust & Safety; • Examples of GenAI exploitation, resulting in the mass creation of dangerous content across a range of abuse areas; • A review and analysis of emerging trends in child predator activity to create GenAI-produced CSAM, and the dissemination of new exploitative methodologies that enable additional threat actors to engage in online harm at scale; • A regulatory overview examining platform accountability for GenAI-produced malicious content.
File
Generative AI The New Attack Vector for Trust and Safety.pdf
Link
https://www.activefence.com/research/genai-new-attack-vector/
Tags
Artificial intelligenceChild Sexual Abuse Material (CSAM)Generative AI
Type
Report
Year
2023
/w=32,quality=90,fit=scale-down){kind=small}
Artificial intelligence-produced child sexual abuse material: Insights from
Dark Web forum posts
Artificial intelligence-produced child sexual abuse material: Insights from
Dark Web forum posts
Anglia Ruskin University
Report
Child Sexual Abuse Material (CSAM)Generative AI
Stella Polaris Summit 2024 Summary
Stella Polaris Summit 2024 Summary
World Childhood Foundation
Other publication
Generative AI - A New Threat for Online Child Sexual
Exploitation and Abuse
Generative AI - A New Threat for Online Child Sexual
Exploitation and Abuse
Bracket Foundation and UNICRI Centre for AI and Robotics
Report
PreventionArtificial intelligenceGenerative AIChild Sexual Abuse Material (CSAM)
Global Threat Assessment 2023
Global Threat Assessment 2023
WeProtect Global Alliance
Report
Child Sexual Abuse Material (CSAM)Prevention
Childhood Activity Report
Childhood Activity Report
World Childhood Foundation
Report
Activity report /VerksamhetsberättelseSweden-focused
Child Sexual Abuse on the Internet: Report on the Analysis of Technological Factors that Affect the Creation and Sharing of Child Sexual Abuse Material on the Internet.
Child Sexual Abuse on the Internet: Report on the Analysis of Technological Factors that Affect the Creation and Sharing of Child Sexual Abuse Material on the Internet.
BI Norwegian Business School, Norwegian University of Science and Technology
Report
InternetPolicyDetectionprosecutionPost-crime effortsCriminal investigation
Guarding the Guardians: Automated Analysis of Online Child Sexual Abuse
Guarding the Guardians: Automated Analysis of Online Child Sexual Abuse
Research (peer reviewed)
Criminal investigationNatural Language Processing
Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and Abusive Texts
Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and Abusive Texts
Cornell University
Research (peer reviewed)
Machine learningNatural Language Processing
Black and Latinx Primary caregiver considerations for developing and implementing a machine learning-based model for detecting child abuse and neglect with implications for racial bias reduction: A qualitative study
Black and Latinx Primary caregiver considerations for developing and implementing a machine learning-based model for detecting child abuse and neglect with implications for racial bias reduction: A qualitative study
University of Pennsylvania Columbia University
Report
PreventionMachine learning
BS-SC Model: A Novel Method for Predicting Child Abuse Using Borderline-SMOTE Enabled Stacking Classifier
BS-SC Model: A Novel Method for Predicting Child Abuse Using Borderline-SMOTE Enabled Stacking Classifier
B. S. Abdur Rahman Crescent Institute of Science and Technology
Research (peer reviewed)
PreventionDetectionClustering/Classification
Applications of artificial intelligence in predicting the risk of child abuse: A literature review
Applications of artificial intelligence in predicting the risk of child abuse: A literature review
King Faisal Specialist Hospital and Research Centre Princess Nora bint Abdul Rahman University Mississippi State University
Research (peer reviewed)
DetectionChild-focusedNeural networks
Determining Child Sexual Abuse Posts based on Artificial
Intelligence
Determining Child Sexual Abuse Posts based on Artificial
Intelligence
Technological University Dublin
Report
Child Sexual Abuse Material (CSAM)Clustering/Classification
Identifying Online Child Sexual Texts in Dark Web through
Machine Learning and Deep Learning Algorithms
Identifying Online Child Sexual Texts in Dark Web through
Machine Learning and Deep Learning Algorithms
Technological University Dublin
Report
Child Sexual Abuse Material (CSAM)Machine learningDeep Learning
Discovering Child Sexual Abuse Material Creator´s Behaviors and
Preferences on the Dark Web
Discovering Child Sexual Abuse Material Creator´s Behaviors and
Preferences on the Dark Web
Technological University Dublin
Research (peer reviewed)
Child Sexual Abuse Material (CSAM)PerpetratorsClustering/Classification
Generative AI: New Attack Vector for Trust & Safety
Generative AI: New Attack Vector for Trust & Safety
Activefence
Report
Artificial intelligenceChild Sexual Abuse Material (CSAM)Generative AI
Child predator abuse of generative AI
Child predator abuse of generative AI
Activefence
Report
Generative AIChild Sexual Abuse Material (CSAM)Artificial intelligence
How AI is being abused to create child sexual abuse imagery
How AI is being abused to create child sexual abuse imagery
Internet Watch Foundation
Report
Artificial intelligencePerpetratorsPreventionChild Sexual Abuse Material (CSAM)Generative AI
…det var en känsla av att han befann sig i rummet bredvid
…det var en känsla av att han befann sig i rummet bredvid
World Childhood Foundation
Report
Sweden-focusedprosecutionInternet
Sperm hunting on optical microscope slides for forensic analysis with deep convolutional networks – a feasibility study
Sperm hunting on optical microscope slides for forensic analysis with deep convolutional networks – a feasibility study
Zurich Institute of Forensic Medicine
Research (peer reviewed)
Perpetrators
Developing automated methods to detect and match face and voice biometrics in child sexual abuse videos
Developing automated methods to detect and match face and voice biometrics in child sexual abuse videos
San Jose State University
Research (peer reviewed)
Child Sexual Abuse Material (CSAM)Perpetrators
Child Sexual Abuse Material Online: The Perspective of Online Investigators on Training and Support
Child Sexual Abuse Material Online: The Perspective of Online Investigators on Training and Support
Griffith University
Research (peer reviewed)
Child Sexual Abuse Material (CSAM)prosecutionPerpetrators
Child Online Safety Toolkit
Child Online Safety Toolkit
5Rights Foundation
Report
PreventionNeural networks
Child Abuse Risk Prediction and Prevention Framework using AI and Dark Web
Child Abuse Risk Prediction and Prevention Framework using AI and Dark Web
Adhiyamaan College of Engineering
Research (peer reviewed)
Child Sexual Abuse Material (CSAM)PreventionNeural networks
Applying Artificial Intelligence for Age Estimation in Digital Forensic Investigations
Applying Artificial Intelligence for Age Estimation in Digital Forensic Investigations
University of Warwick
Research (peer reviewed)
Artificial intelligenceNeural networks
Hand-Based Person Identification Using Global and Part-Aware Deep Feature Representation Learning
Hand-Based Person Identification Using Global and Part-Aware Deep Feature Representation Learning
Lancaster University
Research (peer reviewed)
Artificial intelligenceprosecutionChild Sexual Abuse Material (CSAM)Machine learning
Out of the Shadows Index
Out of the Shadows Index
The Economist Intelligence Unit
Report
PreventionChild Sexual Abuse Material (CSAM)
Developing machine learning-based models to help identify child abuse and neglect: key ethical challenges and recommended solutions
Developing machine learning-based models to help identify child abuse and neglect: key ethical challenges and recommended solutions
Columbia University
Research (peer reviewed)
EthicsDetectionPrevention
ReDirection survey report
ReDirection survey report
Suojellaan Lapsia
Report
Perpetrators
Unga sex och internet #MeToo
Unga sex och internet #MeToo
Stiftelsen Allmänna Barnahus
Report
PreventionChild Sexual Abuse Material (CSAM)Child-focused
Internetrelaterade sexuella övergrepp mot barn
Internetrelaterade sexuella övergrepp mot barn
Riksrevisionen
Report
PreventionChild Sexual Abuse Material (CSAM)prosecution
How SARs fight online child sexual abuse by flagging financial crime
How SARs fight online child sexual abuse by flagging financial crime
Napier
Other publication
PreventionChild Sexual Abuse Material (CSAM)prosecution
Predicting Prolific Live Streaming of Child Sexual Abuse
Predicting Prolific Live Streaming of Child Sexual Abuse
Australian Institute of Criminology
Research (peer reviewed)
Financial transactionsArtificial intelligenceSupervised learningMachine learningCriminal investigationChild Sexual Abuse Material (CSAM)
Automated Biometric Identification using Dorsal Hand Images and Convolutional Neural Networks
Automated Biometric Identification using Dorsal Hand Images and Convolutional Neural Networks
Auckland University of Technology
Research (peer reviewed)
Neural networksPerpetrators
Multimodal Virtual Avatars for Investigative Interviews with Children
Multimodal Virtual Avatars for Investigative Interviews with Children
Oslo Metropolitan University
Research (peer reviewed)
Criminal investigationprosecution
Early Detection of Sexual Predators in Chats
Early Detection of Sexual Predators in Chats
Humboldt Universitat zu Berlin
Research (peer reviewed)
Natural Language ProcessingClustering/Classification
When “Sweetie” is not so Sweet: Artificial Intelligence and its Implications for Child Pornography
When “Sweetie” is not so Sweet: Artificial Intelligence and its Implications for Child Pornography
Report
Child Sexual Abuse Material (CSAM)prosecution
Keeping Children Safe Online With Limited
Resources: Analyzing What is Seen and Heard
Keeping Children Safe Online With Limited
Resources: Analyzing What is Seen and Heard
Singidunum University
Research (peer reviewed)
PreventionClustering/ClassificationNeural networks
Suspicious activity detection using deep learning in secure assisted living IoT environments
Suspicious activity detection using deep learning in secure assisted living IoT environments
Nalla Malla Engineering College, Galgotias University, Vellore Institute of Technology
Research (peer reviewed)
Child Sexual Abuse Material (CSAM)Neural networks
Prediction of the development of depression and post-traumatic stress disorder in sexually abused children using a random forest classifier
Prediction of the development of depression and post-traumatic stress disorder in sexually abused children using a random forest classifier
Uskudar University Medical Faculty, Istanbul, Turkey
Research (peer reviewed)
Clustering/ClassificationMachine learningPost-crime effortsChild-focusedPost-traumatic stress (PTS)
NetClean Report 2020 - Covid-19 Impact
NetClean Report 2020 - Covid-19 Impact
NetClean
Report
PreventionChild Sexual Abuse Material (CSAM)
Illegal Online Sexual Behavior During the COVID-19 Pandemic
Illegal Online Sexual Behavior During the COVID-19 Pandemic
Karolinska Institutet
Research (peer reviewed)
PreventionPerpetratorsChild Sexual Abuse Material (CSAM)Financial transactions
Sexuella övergrepp via nätet lika allvarliga som IRL
Sexuella övergrepp via nätet lika allvarliga som IRL
Göteborgs Universitet
Research (peer reviewed)
Child-focusedChild Sexual Abuse Material (CSAM)Prevention
Summary Paper on Online Child Sexual Exploitation
Summary Paper on Online Child Sexual Exploitation
ECPAT
Report
PreventionChild Sexual Abuse Material (CSAM)Perpetrators
Curbing the Surge
in Online Child Abuse
Curbing the Surge
in Online Child Abuse
European Parliament
Report
PreventionNeural networks
Action to End child Sexual Abuse and Exploitation
Action to End child Sexual Abuse and Exploitation
Unicef
Report
Child Sexual Abuse Material (CSAM)Prevention
Brief version: Action to End child Sexual Abuse and Exploitation
Brief version: Action to End child Sexual Abuse and Exploitation
Unicef
Report
Child Sexual Abuse Material (CSAM)Prevention
An Integrative Review of Historical Technology and Countermeasure Usage Trends in Online Child Sexual Exploitation Material Offender
An Integrative Review of Historical Technology and Countermeasure Usage Trends in Online Child Sexual Exploitation Material Offender
University of Edinburgh and George Mason University
Research (peer reviewed)
Child Sexual Abuse Material (CSAM)PerpetratorsCriminal investigationPrevention
How Facial Recognition Is Helping Fight Child Sexual Abuse
How Facial Recognition Is Helping Fight Child Sexual Abuse
Griffeye
Other publication
Child Sexual Abuse Material (CSAM)Criminal investigationMachine learningArtificial intelligence
Can Artificial Intelligence Achieve Human-Level Performance? A Pilot Study of Childhood Sexual Abuse Detection in Self-Figure Drawings
Can Artificial Intelligence Achieve Human-Level Performance? A Pilot Study of Childhood Sexual Abuse Detection in Self-Figure Drawings
University of Haifa
Research (peer reviewed)
Artificial intelligencePerpetratorsCriminal investigationSupervised learningNeural networksMachine learningChild-focusedChild Sexual Abuse Material (CSAM)
Estimation of the Development of Depression and PTSD in Children Exposed to Sexual Abuse and Development of Decision Support Systems by Using Artificial Intelligence
Estimation of the Development of Depression and PTSD in Children Exposed to Sexual Abuse and Development of Decision Support Systems by Using Artificial Intelligence
Inonu University
Research (peer reviewed)
Child-focusedPost-traumatic stress (PTS)Post-crime efforts
AI Roundtable 2019 Summary
AI Roundtable 2019 Summary
World Childhood Foundation
Report
PreventionArtificial intelligence
Artificial Intelligence: combating online sexual abuse of children
Artificial Intelligence: combating online sexual abuse of children
Bracket Foundation
Report
PreventionNeural networksChild Sexual Abuse Material (CSAM)
Out of the Shadows Whitepaper
Out of the Shadows Whitepaper
The Economist Intelligence Unit
Report
PreventionChild Sexual Abuse Material (CSAM)
What Works to Prevent Sexual Violence Against Children
What Works to Prevent Sexual Violence Against Children
Together for girls
Report
Child Sexual Abuse Material (CSAM)PreventionFinancial transactions
Child Online Safety:
Minimizing the Risk of Violence,
Abuse and Exploitation Online.
Child Online Safety:
Minimizing the Risk of Violence,
Abuse and Exploitation Online.
ITU/UNESCO Broadband Commission for Sustainable Development
Report
PreventionPerpetrators
A Practitioner Survey Exploring the Value of Forensic Tools, AI, Filtering, & Safer Presentation for Investigating Child Sexual Abuse Material (CSAM)
A Practitioner Survey Exploring the Value of Forensic Tools, AI, Filtering, & Safer Presentation for Investigating Child Sexual Abuse Material (CSAM)
University of New Haven / Digital Forensic Research Workshop
Research (peer reviewed)
Artificial intelligencePreventionNeural networks
Therabot: An Adaptive Therapeutic Support Robot
Therabot: An Adaptive Therapeutic Support Robot
Institute of Electrical and Electronics Engineers (IEEE) and Mississippi State University
Research (peer reviewed)
Post-crime effortsChild-focusedRoboticsPost-traumatic stress (PTS)
Offender strategies for engaging children in online sexual activity
Offender strategies for engaging children in online sexual activity
Department of Psychology, University of Gothenburg
Research (peer reviewed)
PreventionChild Sexual Abuse Material (CSAM)
Study on the Effects of New Information Technologies on the Abuse and Exploitation
of Children
Study on the Effects of New Information Technologies on the Abuse and Exploitation
of Children
UNODC
Report
PreventionChild Sexual Abuse Material (CSAM)
Predictive Risk Modelling to Prevent Child Maltreatment and Other Adverse Outcomes for Service Users
Predictive Risk Modelling to Prevent Child Maltreatment and Other Adverse Outcomes for Service Users
University of Queensland
Research (peer reviewed)
PreventionNeural networks