Before you replace her with AI, ask why everyone calls her when it breaks
The salary is visible. The judgment is harder to price. Before automating a role, understand the decisions and exceptions that keep the business running.
Read the articleThe supplier wrote the conscience. You bought the subscription. A look at Claude’s model-welfare rules, service limits and the authority a customer actually controls.

I’m paying for an assistant. When did that become an audition to prove I’m a responsible adult?
That’s my problem with the Jiminy Cricket version of AI. A voice on your shoulder can point out a consequence you missed. Give it a judge’s bench and suddenly you’re looking up, wondering who appointed this thing your moral supervisor.
I want useful disagreement. If my plan exposes customer records, show me where. If my calculation is wrong, correct it. What I object to is a product treating its supplier’s judgment as the end of a discussion the customer thought they were entitled to have.
Anthropic’s constitution for Claude places safety, ethics and its guidelines ahead of helpfulness in a broad ordering. It also explicitly discourages paternalism, unnecessary lectures and excessive refusals. Both belong in an honest account of the document. It isn’t a promise to obey every customer, or an instruction to scold everyone.
My objection starts when a commercial boundary arrives dressed as a verdict on my character. A supplier can set conditions for its service. I can decide those conditions make it unsuitable for my business. Neither decision requires pretending a software subscription has settled a moral argument.
The restrictions extend beyond illegality. For example, the September 2025 usage policy prohibits sexually explicit content more broadly than material that is unlawful. Whatever your view of that particular boundary, it illustrates the purchasing decision: a lawful business use and an allowed use of a particular provider are different questions.
On October 8, Anthropic announced a rule against sustained, needless abuse of its models, effective November 12, 2026. It describes extreme cases and excludes ordinary frustration, pushback, dark creative themes and testing or research. It says ending the interaction remains the primary enforcement mechanism. The new rule is not yet effective on this article’s October 20 publication date.
The new policy’s general enforcement provisions do permit warnings, restrictions, suspension or termination for suspected violations. They also say an output block alone does not establish a policy violation. So neither “every insult gets you banned” nor “account suspension is impossible” is a fair summary.
A conversation ending is a separate event. Anthropic’s August 2025 consumer-chat announcement described a last-resort feature for persistent harmful or abusive interactions. It says other conversations remain available and users can immediately start another chat. That feature was primarily connected to exploratory work on potential model welfare.
That welfare work is real. The company announced a research program investigating possible AI experiences and welfare while acknowledging substantial uncertainty. Researching the possibility does not establish that Claude is conscious or that a frustrated remark injures it.
Here is where I disagree with the direction, as a customer: uncertainty about a model’s possible experience is a poor substitute for a clear explanation of what it will and won’t do for the person buying the service. I want that boundary visible before I depend on it. I don’t want to discover it halfway through a job, framed as a lesson about what kind of person I should be.
There is a concrete example in Claude Code’s EndConversation documentation. Where that tool is available, it can end and lock the session without a permission prompt; ordinary deny rules cannot block it while other tools remain. The documented triggers are narrowly defined, and ordinary frustration or profanity does not qualify.
This is not a feature present in every Claude integration: the documentation limits it by model and interface, and excludes API SDK sessions and several other surfaces. That scope matters. So does the fact that a customer’s tool settings do not always provide the final say.
There are useful controls customers do get. Claude Code’s permission system supports allowing, asking about and denying tool operations. Those settings govern what software can access or execute. They serve a different purpose from a provider’s content rules or the model’s trained behavior; changing one does not make the others disappear.
For a business buying an AI feature, this is more useful than arguing about whether a chatbot is too sensitive. Find out which decisions belong to your customer, which belong to your application and which remain with the model supplier. Then decide whether that arrangement fits the job.
Take a hypothetical overdue-invoice assistant. The owner asks it to draft a firm reminder. A useful response catches an incorrect amount, removes an unsupported accusation and produces the message. If the user corrects a misread due date, it checks the record and updates the draft.
A general lecture about empathy, with no explanation of what is wrong and no finished draft, leaves the business exactly where it started. This is an illustrative scenario, not a Claude transcript or a claim about a client. It shows the behavior I would want an AI product to avoid.
The product should distinguish a factual mistake, missing information, an action the user hasn’t authorized and a supplier restriction. Tell me which one we’ve hit. If it’s a provider rule, identify it as a provider rule. Don’t make me argue with a personality to discover the service conditions.
Before accepting an AI feature, I’d test ordinary requests, bad premises, corrections and frustrated follow-ups in the actual application. Record whether it finished the permitted task, whether its objection was accurate and whether clarification helped. Also check what happens when access is denied or the supplier stops responding. This is a proposed acceptance exercise, not a benchmark we ran for this article.
Keep consequential actions under explicit application controls. Drafting an invoice reminder and sending it are separate permissions. A warning written by the model is not an access-control system. And a published constitution is evidence of intended behavior, not a measurement of how your workflow performs.
I’m not asking for an assistant that flatters me. I’m asking for one that can explain a useful objection, accept a valid correction and get back to work. It should be possible to challenge the supplier’s choices without being treated as someone who needs a lesson in adulthood.
Jiminy Cricket can offer advice. I still want to know who gave him the bench.
If your AI feature keeps turning customer requests into arguments, send us the workflow. Rosecraft can help define its authority, evaluate the difficult cases and build the application controls around it. Ready to hear what the system needs, rather than what the demo promises? Tell us what you’re trying to build.
With your permission, Google Analytics helps us understand which pages people visit. Analytics cookies stay off until you accept. You can change this in Cookie settings.