Not on the plan most people are using.
That is the short answer, and it is worth sitting with, because the difference between a personal login and a business one is the single largest factor here and almost nobody has checked which their team is on. This article ends with how to check yours.
The Australian Bureau of Statistics counted 12% of Australian businesses using AI in 2024 to 2025, up from 1% three years earlier. Surveys that ask employees rather than employers come back far higher. Some of that gap is method, because the two are counting different things. Not all of it.
I say a version of this to business owners often enough that it has become a stock line: you tell your people to go and use AI, they go and do it, and some of them have no real idea what they are doing. They might be leaking private information without knowing.
What actually happens to what you type in?
Three questions hide inside that one. Is it used to train the model, how long is it kept, and who can reach it.
The Australian Signals Directorate, writing for small business with its New Zealand counterpart and the Council of Small Business Organisations Australia, answers the first one directly. Some providers may use customer-submitted data to train or refine their models, it says, and this “can depend on the configuration settings or the type of subscription”. Information entered “could potentially be reused or disclosed in unexpected contexts later”.
The specifics, as they stand today. On ChatGPT’s consumer plans, free, Plus and Pro, OpenAI’s help centre says the product improves by further training on the conversations people have with it, unless you opt out. On Claude’s consumer plans, free, Pro and Max, it is a choice rather than a default: Anthropic asked users to decide. Allow training and conversations are retained for five years. Decline, and deleted conversations are purged within 30 days instead.
Worth being precise about that last point, because it is widely misread. Declining does not make your chat history disappear after a month. It sits in the account until someone deletes it. The 30 days is how long deletion takes to reach the back end.
Does the plan you are on change the answer?
Yes. More than anything else you could do this month.
OpenAI’s enterprise privacy page states that on its business, enterprise, education and API products it does not train on your data by default. Anthropic’s commercial tiers, team and enterprise and API, were explicitly excluded from the consumer terms change above.
So the practical question is not “is ChatGPT safe”. It is which account your bookkeeper was signed into on Tuesday.
This is where it comes apart in smaller businesses. Someone starts on a personal subscription because they wanted to try it, it works, and it quietly becomes part of how they do their job. There was never a moment where a decision got made. The firm now has client information going through an account it does not own, cannot see, and cannot switch off when that person leaves.
One honest qualifier, because it cuts against my own advice. A business plan changes the vendor’s default, not the laws the vendor operates under. In May 2025 a US court ordered OpenAI to preserve output logs it would otherwise have deleted, including for free, Plus, Pro and Team subscribers. Enterprise, education and zero-retention API customers were outside it. That order ended in September 2025, but data already preserved stayed preserved. Published terms are a policy, and a policy is not the same thing as a guarantee.
Has this actually gone wrong in Australia?
Twice, publicly, and neither was caught by a security alert.
The older case is the more uncomfortable. In September 2024 the Victorian information commissioner published an investigation into a child protection worker’s use of ChatGPT to help draft a Protection Application Report for the Children’s Court, arising from an incident reported in December 2023. The worker said they had removed identifying details. The regulator concluded, on the balance of probabilities, that client names had been entered, and found the department in breach of two Information Privacy Principles.
What gave it away was not a monitoring tool. It was the writing. The draft mischaracterised evidence in the case badly enough that a departmental lawyer noticed the language about a week later, in a report going to a court about a child’s safety. The privacy failure and the accuracy failure arrived together, which is the part most accounts miss.
The scale is the detail I would take from it. The department found that almost 900 staff had accessed the ChatGPT website over six months, close to 13% of its workforce. Nobody had decided that. It had simply happened.
Then, in March 2025, a former temporary staff member at the NSW Reconstruction Authority uploaded a spreadsheet to ChatGPT, which was not an approved tool there. It held more than 12,000 rows of applicant data from a flood recovery program. Investigation later established that 2,031 people had some of their data uploaded: names, contact details, dates of birth, and sensitive personal and health information. The NSW Privacy Commissioner was notified, and investigators found no evidence the data had been accessed by anyone else or appeared publicly. ASD’s small business guidance now carries an anonymised example that reads unmistakably like this one.
Is this a privacy problem or a security problem?
Both. The privacy half is the one with a regulator attached.
The OAIC’s guidance on commercially available AI products says that, as a matter of best practice, organisations should not enter personal information, and particularly sensitive information, into publicly available generative AI tools. That is the regulator’s own position in its own words, and it has stood since late 2024.
Whether the Privacy Act applies to you turns on your turnover and on what you do. The small business exemption still sits at $3 million, but the carve-outs catch more firms than owners expect, and since 1 July 2026 the anti-money-laundering reforms have brought accountants, lawyers, conveyancers and real estate businesses into scope for the personal information they handle under those obligations. The thresholds, and what the December 2026 obligation on automated decisions actually reaches, are in whether Australian businesses still need AI governance. That article is about what is required. This one is about what is happening.
There is a fair objection to all this, and you should have it. Notifiable data breaches reported to the OAIC hit an all-time high in 2025, 1,205 notifications, up 8% on the year before. AI is not among the causes. It has never been a category in that reporting at all.
Two reasons not to take much comfort from it. The federal scheme has no AI cause code, so a breach that began with an AI tool gets recorded as human error and disappears into a number that rose sharply last year anyway. And both Australian cases above sat with state regulators, Victorian and NSW, so neither was ever in the OAIC’s figures to begin with. The Victorian investigation added the sharpest point: the regulator recorded that it was impossible to verify what had actually been typed in, because organisations have no visibility of what staff put into these tools. That is a gap in the measurement, not a finding about the risk.
What about the tools nobody approved?
You cannot manage what you are pretending is not happening.
A survey of 500 Australian technology decision-makers by Josys, a vendor that sells software for exactly this problem, found 36% reporting staff uploading sensitive company information to AI tools and 24% reporting customer personal information going the same way. Read it with that interest in mind. The figure I find more telling is a quieter one: only 30% of firms under 250 staff felt equipped to assess AI risk, against 42% of larger ones. The survey is from September 2025, so treat it as a direction rather than a current number.
The instinct is to ban it. That fails in a specific way: the tool moves to a personal phone, the work carries on, and you lose the only advantage you had, which was being able to see it.
Does the vendor’s security handle this for you?
No, and this is the assumption I most often have to take apart.
Vendor certifications tell you the vendor runs a tight operation. They say nothing about which of your client folders somebody connected to it last month. The vendor secures their side. What reaches the tool from your side is yours to decide, and no product makes that decision for you.
The vendor’s own controls do not always hold either. In early 2026 Microsoft fixed a defect in which Copilot Chat surfaced content from confidential-labelled emails in users’ own Sent Items and Drafts, going around sensitivity labels those customers had configured. Microsoft has not published a post-incident report or said how many tenants were affected. The controls you set are a claim about how a product should behave, and products have bugs.
That gap widens as soon as you connect anything. Once a tool can see your files or your mail, what it can reach stops being theoretical, and permissions are always easier to widen than to pull back. I have written about scoping that in deciding what to let an AI system touch.
How do you actually check your own setup?
Half an hour, and you can do it yourself today.
Find out what people are signed into. Ask, and make it safe to answer honestly, because the goal is an accurate picture rather than a culprit. Then check the expense claims and the company card statements for the last three months. Personal AI subscriptions show up there as small recurring charges, and they are the clearest evidence of work happening on accounts you do not control.
Open the settings on each account. In ChatGPT it is under Settings, in the data controls section, where the toggle governs whether your conversations improve the model. In Claude it sits under Settings, in privacy. Both take under a minute and both tell you what has been true for however long that account has existed.
Check whether you are already paying for a better option. If you run Microsoft 365 or Google Workspace, the assistant bundled with your plan has different data handling from the consumer app of the same name, and it may be sitting there switched off. Ask your IT provider which tier you hold and what its terms actually say, rather than assuming the brand name tells you.
Then do the four things that stick. Move everyone doing client work onto a business plan. Write down what may not go in, as a paragraph rather than a policy: names, health information, financial detail, anything under a confidentiality clause. Ask what each tool is currently connected to and remove anything nobody needed. Name someone who owns it, including the bad weeks.
ASD’s own list for small business runs along the same lines: know what data you hold, read the vendor’s handling terms, write down what may not be uploaded, train people, and strip identifying detail before it goes anywhere.
All of that is about the setup. What it does not settle is what the output then decides, and that is where exposure changes if AI is anywhere near decisions about people. The AI risk assessment takes one use case through both and ends on a decision: proceed, proceed with conditions, or not yet.
Getting a whole team to work the same way afterwards is its own problem, and I have covered it in getting a team using AI consistently. If you would rather work through your own setup with someone, that is what AI training is for, or you can just get in touch.
FAQ
Is it safe to put client information into ChatGPT?
On a personal plan, treat it as unsafe. Consumer ChatGPT may use conversations to improve the model unless you have opted out, and the account is not yours to control. On the business and enterprise tiers OpenAI states it does not train on your data by default, which is a materially different position. Separately, the Australian privacy regulator recommends as best practice not entering personal information, and particularly sensitive information, into publicly available generative AI tools at all.
Does ChatGPT train on what I type?
On the consumer plans, yes, unless you turn it off in the data controls section of Settings. On the business, enterprise, education and API products, training on your data is off by default. Retention is a separate question from training, and a court order in 2025 required OpenAI to keep logs it would otherwise have deleted, on consumer and team plans alike, which is worth knowing before you treat any published retention period as a promise.
Does Claude train on my conversations?
On the consumer plans it depends on a choice made when the account was set up. Allowing it means conversations are retained for five years. Declining means deleted conversations are purged within 30 days, which is not the same as your history clearing itself. The commercial tiers, team and enterprise and API, were excluded from that change.
Should we ban AI tools at work?
I would not. A ban moves the same behaviour onto personal devices and personal accounts, where you have no visibility and nothing to switch off. A permitted tool on a business plan, with a short written rule about what may go into it, gets you further than a prohibition everyone quietly works around.
Someone has already put client data into a personal account. What now?
Do not delete anything yet, and do not start with blame or you will not hear about the next one. First preserve the record: export or screenshot the conversation, because it is the only evidence of what actually went in, and Australian regulators have found that organisations usually cannot reconstruct this afterwards. Then assess whether anyone’s personal information was involved, which is the point to take advice rather than guess, and note that the notifiable data breach scheme gives you 30 days to complete that assessment once you suspect a breach. Clean-up, settings and moving the person onto a business plan come after that, not before.
We are a small business under the turnover threshold. Does any of this apply?
Your Privacy Act position depends on turnover and on what you do, and the exemption has carve-outs that catch more firms than owners expect, including professions brought in by the 2026 anti-money-laundering reforms. But the commercial exposure does not track the legal one. A client who finds their information went into a tool they were never told about is not going to check whether you were technically covered.
General information only, not legal advice. Current as at September 2026: vendor terms and retention periods in this area change frequently, and the position may have moved by the time you read it.
