Content area
Full text
Enterprise IT execs know well the dangers of relying too much on third-parties, how automated decision systems need to alwayshave a human in the loop, and the dangers of telling customers too much/too little when policy violations require an account shutdown. But a saga that played out Tuesday between Anthropic and the CEO of a Swiss cybersecurity company brings it all into a new and disturbing context.
The tale began when Tom Hoffman, the CEO of a Swiss cybersecurity company called Wicked Design, on Monday received an alert from Anthropic’s system that the company’s entire account had been cancelled due to an automated review that supposedly found unspecified policy violations.
Hoffman knew the best way to get that addressed was with social media support, so he wrote about it on LinkedIn. He also alerted some people who worked at Anthropic, including the head of product legal. The latter responded, saying he’d flagged the issue internally for review. “We are working on the automated ban stuff,” said the Anthropic attorney. (Hoffman shared multiple Anthropic screen captures with Computerworld.)
Within a day, the account was restored — sort of. The new alert told him: “Earlier this week, your account was disabled by an automated system for being in violation of our Terms of Service or Acceptable Use Policy. Upon further investigation, we believe this was an error and your account has been reinstated. We apologize for the inconvenience and for your patience.”
Hoffman briefly celebrated, logged in and found that most of the account — 80% of company projects and data, he said — was missing.
When he asked Anthropic’s automated system how to restore those files, the system replied, “I understand how frustrating this must be. When a user is removed and later re-added to an organization, previous projects and their associated chats are not restored — even if you use the same email address. Unfortunately, there’s no way to restore or...