Gemini account suspended after tweaking a custom Gem's safety wording
A healthcare worker built a custom Gemini chatbot (a "Gem") to answer clinical questions. To cut down on repetitive output, its stated that Gemini already understood it couldn't give medical advice, so it didn't need to repeat that disclaimer in every response. Over the prior two weeks, Google sent warning emails flagging the account for vague policy violations related to "overriding or circumventing or ." Asking Gemini itself to diagnose the issue, the best guess was that this healthcare Gem's instructions were the likely cause.
The disclaimer wording was removed the next day, following Gemini's own suggestion. The following morning, the entire Gemini account was restricted. An appeal has been filed, but given how opaque the whole process was, there's little expectation of reinstatement.
The person also ran separate fiction-writing Gems, which Gemini did not flag as a likely cause.
Key points
- A custom healthcare Gem's instructions told Gemini it didn't need to repeat a medical-disclaimer every response
- Google flagged the account for vague 'circumventing ' violations
- The account was fully restricted the day after the flagged wording was removed
- Even Gemini's own of the cause was just a best guess
- An appeal was filed but reinstatement seems unlikely given the opaque process