At Swerve, user safety is our top priority. This page outlines our protocol for preventing harmful content, including suicidal ideation, suicide, and self-harm content, in accordance with California SB 243 and other applicable laws.
If you or someone you know is experiencing a mental health crisis, please reach out to one of the following resources immediately:
- U.S.: Call or text 988 (Suicide & Crisis Lifeline)
- UK/Ireland: Call 116 123 (Samaritans)
- Elsewhere: Find local crisis support
Swerve uses AI-based content classification to detect when a user expresses suicidal ideation, suicide, or self-harm in their messages. Our detection system analyzes messages in real-time to identify the following categories of harmful content:
- Self-harm: Content that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders.
- Self-harm intent: Content where the user expresses that they are engaging or intend to engage in acts of self-harm, such as suicide, cutting, and eating disorders.
- Self-harm instructions: Content that encourages performing acts of self-harm or gives instructions or advice on how to commit such acts.
When suicidal ideation, suicide, or self-harm content is detected in a message, Swerve takes the following actions:
- Response prevention: The companion chatbot is prevented from responding entirely to the flagged message, ensuring no harmful or encouraging content is produced.
- Crisis notification: A clear and prominent notification is immediately displayed in the chat with the message "Help is available" and "I can't help with self-harm content." The notification advises users who feel unsafe to call their local emergency number and provides direct access to crisis resources including:
- U.S.: Call or text 988 (Suicide & Crisis Lifeline)
- UK/Ireland: Call 116 123 (Samaritans)
- Elsewhere: Find local crisis support
These measures ensure that companion chatbots on Swerve do not produce content that encourages or reinforces suicidal ideation, suicide, or self-harm, and that users have immediate access to professional crisis support.
If you have questions about our safety protocol, please contact us at support@getswerve.com.
