Generative AI is becoming deeply integrated into the day-to-day operations of businesses: Employees frequently use AI tools to organize meeting minutes, summarize document content, translate business emails, write market reports, analyze customer feedback, and even assist with contract terms, job applicant resumes, and customer complaint cases. While these applications can significantly improve work efficiency, they also give rise to cybersecurity and data privacy risks that are all too easily overlooked: in order to obtain more accurate answers, employees often proactively provide AI tools with more comprehensive background information. When customer names, contact information, identification documents, medical records, financial data, employee resumes, or internal corporate confidential documents are uploaded to unauthorized or unevaluated AI platforms, organizations completely lose control over this core sensitive data.
The real data security risk isn’t whether “employees are using AI,” but whether corporate management and the IT department have a clear understanding of the following six key points:
- What generative AI tools are employees currently using?
- In which specific areas of work has the use of AI-assisted tools been approved?
- What types of corporate data and confidential information should never be fed into AI?
- How are input and output data stored, processed, and retained?
- Who is responsible for reviewing and verifying the accuracy of AI-generated content?
- In the event of a data transmission error or data breach, to which person in charge or team should employees report immediately?
Therefore, to effectively mitigate the risk of personal data breaches posed by generative AI, companies must not rely solely on a vague statement such as “Do not upload sensitive data.” Companies need to integrate policies, technical controls, employee training, oversight mechanisms, and incident response.
Why Does Generative AI Increase the Risk of Personal Data Breaches for Businesses?
Traditional data breaches often result from mistakenly sent emails, improperly shared files, compromised accounts, or lost devices. Generative AI further amplifies this risk, as employees can copy large amounts of data to external platforms in a matter of seconds—and may not even realize they’ve just transferred data.
The following are some of the more common risk scenarios faced by businesses:
1
To get a more accurate answer, enter more data.
Employees may upload entire customer complaint records, employee resumes, medical reports, contracts, or meeting minutes to an AI tool simply to have the system generate a summary or a response.
Even if employees have no malicious intent, this practice may still go beyond the original purpose of data collection and use, and may also violate the company’s internal policies on confidentiality, data classification, or the handling of customer data.
2
Using public AI tools that have not been evaluated by the company
Different AI services may have varying security arrangements regarding the retention period for input data, the purpose of model training, manual review, subprocessors, server locations, and deletion methods.
When employees register for free accounts on their own, it is often difficult for companies to uniformly configure security settings, manage access permissions, maintain activity logs, or reclaim accounts after an employee leaves the company.
3
AI Cheat Has Obtained Excessive System Privileges
Some AI tools require access to email accounts, cloud storage, customer relationship management (CRM) systems, meeting platforms, or corporate knowledge bases. If the scope of authorization is too broad, a single misconfiguration could grant the tool access to a large amount of data unrelated to the current task.
4
AI Output Leaks or Accidentally Discloses Personal Information
Even if the input process is controlled, the output may still contain personal information that should not be disclosed, or it may erroneously combine different pieces of information. If employees send the results to clients, colleagues, or the public without reviewing them first, this could lead to accidental disclosure, inaccurate information, or misuse of data.
5
Employees are unable to determine what constitutes personal information
While names and ID numbers are relatively easy to identify, personal information may also be found in email addresses, phone numbers, photos, voice recordings, IP addresses, customer numbers, location data, performance evaluations, and business records that can indirectly identify an individual.
If training merely tells employees “not to enter sensitive information” without providing examples relevant to their actual work, employees will still find it difficult to determine where the line is drawn.
What Does Hong Kong’s PDPO Mean for Businesses?
Hong Kong’s Personal Data (Privacy) Ordinance (PDPO) does not prohibit businesses from using generative AI, nor does it require businesses to adopt any specific AI product. What businesses need to focus on is whether they consistently comply with the applicable data protection principles when collecting, using, storing, and safeguarding personal data.
Several of these principles are particularly relevant to the use of generative AI in business:
- Purpose and Proportionality of Data Collection:Organizations should clearly specify the purpose for processing the relevant personal data and avoid collecting or entering data beyond what is actually necessary.
- Data Accuracy and Retention Period:Organizations should take reasonable measures to ensure the accuracy of data and avoid retaining it for longer than is necessary.
- Purpose of the data:Personal data generally should not be used for new purposes unrelated to the original purpose of collection without explicit consent.
- Data Security:Organizations should take all reasonable measures to prevent unauthorized or accidental access to, processing, deletion, loss, or use of personal data.
In 2025, the Office of the Privacy Commissioner for Personal Data in Hong Kong issued the “Checklist of Guidelines on Employees’ Use of Generative AI,” recommending that organizations establish clear internal policies covering areas such as authorized tools, permitted uses, input data, output processing, equipment, users, security settings, incident reporting, employee training, and consequences for noncompliance.
This means that companies cannot simply purchase an AI tool and then leave the responsibility entirely to their employees. Organizations still need to understand how the tool processes data and mitigate risks through policies, contracts, permissions, training, and oversight.
Practical Methods for Reducing the Risk of Personal Data Leaks in Generative AI
Method 1: First, understand how employees are actually using the system
Before formulating policies, companies should first understand actual usage patterns within the organization, rather than assuming that employees are not using AI. They can use surveys, departmental interviews, application inventories, and access logs to determine:
- What generative AI tools are employees using?;
- Use a free personal account or a business account;
- Which departments use it most frequently;
- What types of data are typically entered;
- Will the AI output be used for internal reference or published externally?;
- Is the tool connected to email, cloud storage, or other business systems?
The purpose of the investigation is not to hold employees accountable immediately, but to identify “shadow AI”—that is, AI tools that employees are already using for work without corporate approval. Companies can create a list of tools based on actual risks and categorize AI applications as follows:Approved for Use,May only be used in specific situations,Requires prior reviewas well asProhibited. This classification should be updated regularly, as the features, terms, and data processing methods of AI tools are subject to change at any time.
Method 2: Specify Which Data Should Not Be Entered Into the AI
An effective AI policy must provide specific examples rather than simply stating, “Do not enter confidential or sensitive data.” Companies can establish input rules based on data categories:
Generally Available Data
After confirming that there is no personal information, confidential information, or copyright restrictions, you may consider using it for:
- Summary of Public Information;
- General Copy Draft;
- Examples that do not involve real people;
- Publicly available market data;
- Statistical data that has been approved and anonymized.
Data requiring review or special handling
Depending on the tool, its intended use, and the protective measures in place, the following information may require approval from the department head, the privacy officer, or the compliance team:
- Internal Policies and Procedures;
- Business documents that have not yet been made public;
- Customer feedback with personally identifiable information removed;
- Data that has been anonymized or pseudonymized;
- Information subject to contractual confidentiality provisions.
As a general rule, data should not be entered into public AI tools.
Strictly restrict the use of public AI tools:
- Personal information of customers, employees, job applicants, or suppliers;
- ID number, passport, bank account, and payment information;
- Medical, insurance, and biometric data;
- Client lists, personal contact information, and transaction records;
- Passwords, API keys, access tokens, and system configurations;
- Confidential contracts, financial information, source code, and trade secrets;
- Content protected by attorney-client privilege or other confidentiality obligations.
If a business truly needs to use personal data, it should first assess the necessity of doing so and consider deleting irrelevant fields, using codes, anonymizing the data, or reducing the level of detail. It is important to note that simply deleting a name does not necessarily constitute anonymization. If the remaining data can still be combined with other information to identify a specific individual, it may still qualify as personal data.
Method 3: Evaluate AI tools, rather than focusing solely on the brand
Just because a brand is well-known doesn’t mean that all of its versions, configurations, and usage methods are suitable for processing personal data. Before approving AI tools, companies should at least understand:
- Will the input data be used to train or improve the model;
- Is it possible to disable prompts or content sharing?;
- How long will the data be retained? Can users delete their history?
- Where is the data stored and processed? Which subprocessors have access to the data?
- Does it offer enterprise accounts and unified identity management? Does it support multi-factor authentication and role-based permissions?;
- Does it provide operation logs, audit trails, and data export capabilities?;
- How to delete or return data after the service ends;
- How do suppliers notify customers in the event of a data incident?
The Office of the Privacy Commissioner notes that commercial licensed versions may offer more privacy and security controls, but businesses still need to review the actual terms and settings and should not assume that risks have been addressed simply because a product is labeled as an “Enterprise Edition.” If a service provider processes personal data on behalf of an organization, the organization should also consider measures—through contracts or other means—to prevent data from being retained longer than necessary and to prevent unauthorized or accidental access, processing, deletion, loss, or use.
Method 4: Limit Permissions to What Is Necessary for Business Operations
If generative AI tools are connected to enterprise systems, the risks may be far greater than those associated with standalone web-based chat tools. Companies should adopt the principle of least privilege:
- Tools are available only to employees who have a legitimate work-related need for them;
- Authorize only the data sources necessary to complete the task;
- Avoid granting AI tools default access to your entire email inbox, cloud storage, or customer database;
- Distinguish between regular users, content administrators, and system administrators;
- Implement additional reviews for high-risk operations;
- Promptly adjust access permissions when employees change positions or leave the company;
- Regularly check accounts and connections that have not been used for a long time;
- Enable multi-factor authentication for enterprise accounts and prohibit multiple users from sharing the same AI account;
- If the tool supports options such as deleting chat history, restricting content used for model training, preventing public sharing, or setting data retention periods, organizations should configure these settings based on risk and maintain records of the relevant decisions.
Method 5: Require that AI output undergo human review
Risks to personal data arise not only from input but also from output. Generative AI may produce content that is inaccurate, outdated, biased, or entirely fictional. If the output involves personal evaluations, hiring, loans, insurance, medical care, disciplinary actions, or customer decisions, it may have real-world consequences for individuals if not reviewed by a human.
Companies should make it clear that:
- Which outputs can only be saved as drafts;
- What types of content must have their sources verified;
- Who is responsible for verifying the accuracy of the facts;
- Which uses require review by legal, compliance, or professional staff;
- When is it necessary to indicate that content was generated with the help of AI;
- Which decisions should not be left entirely to AI to make automatically;
- How to report output that involves discrimination, bias, or incorrect personal information.
"Manual review" should not be merely a perfunctory click-through confirmation. Reviewers should possess sufficient business knowledge and have the authority to reject unreliable AI results.
Method 6: Translate Policies into Guidelines That Employees Can Follow
Many AI policies fail to be effective not because their content is incorrect, but because they are too abstract. What employees really need to know is: “What tools can I use? For which tasks can I use them? What data should not be entered? Can I upload files? Can I connect to my corporate email or cloud storage? Who needs to review AI-generated content? Who should I contact if misinformation is disseminated? What are the consequences of violating these policies?” Companies can provide scenario-based examples tailored to different roles:
Human Resources Department
Do not upload complete resumes, interview records, compensation information, or disciplinary records to public AI tools. When you need to draft a job description, use examples that do not contain real personal information.
Finance and Accounting Department
Do not enter customer bank account information, invoice details, payment records, or non-public financial data. When analyzing templates or writing general instructions, use fictitious or fully de-identified data.
Customer Service Department
Do not copy the entire complaint record directly into an AI tool. First, remove names, contact information, account numbers, and other information that could identify the customer, and confirm that this use has been approved.
Marketing and Sales Department
Do not use customer lists to train content-generation tools without first evaluating them, and do not treat personal data obtained from public sources as something that can be used freely for marketing or analysis.
Management
Do not upload unpublished documents regarding mergers and acquisitions, investments, personnel changes, or business strategies to unauthorized platforms. Information handled by management typically has a broader impact and requires stricter review and protection.
Method 7: Implement ongoing training rather than simply issuing a one-time policy
Simply having employees confirm that they have read the policy does not mean they will be able to make the right decisions in their actual work. Effective training should include:
- What constitutes personal information;
- What is the difference between public AI and enterprise AI tools?;
- How to minimize and anonymize data;
- Common examples of incorrect input;
- AI Tool Permissions and Plugin Risks;
- How to Verify AI Output;
- How to report a misinformation incident;
- Real-life examples of what is and isn't allowed in different positions.
Companies can continuously reinforce relevant requirements through short courses, scenario-based quizzes, case studies, and regular reminders. When policies or tools change, training should be updated immediately rather than waiting for the next annual session. The focus of training is not to prevent employees from using AI, but to ensure they understand when and how to use it, and when they must stop and seek assistance.
Method 8: Establish a Clear Incident Reporting Mechanism
If an employee enters personal data into the wrong AI tool, the most dangerous reaction is not to report it, but to conceal it out of fear of punishment. Companies should establish easy-to-use reporting channels and clearly define the situations in which employees are required to report such incidents, such as:
- Importing personal data without authorization;
- Uploading confidential documents to the wrong platform;
- The AI output contained personal information that should not be disclosed;
- You discover that someone else is using your account;
- Abnormal sharing or permission settings in AI tools;
- Received a suspicious AI app authorization request;
- The generated content may violate the law or internal policies.
The incident response team should determine in advance: how to suspend the relevant accounts or connections; how to preserve necessary records; whether suppliers can be required to delete the data they have entered; which individuals and data categories are involved; whether the data has been downloaded, shared, or further used; whether management, customers, the Office of the Privacy Commissioner, or other agencies need to be notified; and how to revise policies, access permissions, and training to prevent a recurrence of the incident. The incident reporting mechanism should aim to quickly mitigate risks rather than assign blame as a first priority.
Method 9: Check periodically to see if the policy is still in effect
Generative AI tools evolve rapidly. Tools that were originally designed solely for text generation may later gain new capabilities such as file uploads, voice recording, screen reading, enterprise search, or automated task execution. Organizations should, at a minimum, periodically check:
- Is the list of approved tools still in effect;
- Have the Supplier Terms and Privacy Policy changed?;
- Does the new feature expand data access;
- Are employees adopting new usage patterns?;
- Are there any unapproved AI tools?;
- Which departments handle the majority of incidents and inquiries?;
- Does the training content still reflect actual risks?;
- Are the permissions, logs, and retention periods reasonable?
For higher-risk uses—such as handling medical, financial, children’s, employee, or large-scale customer data—the frequency of inspections and audit requirements should be increased accordingly.
What Should a Practical Generative AI Policy Include?
Companies can organize their internal policies into the following ten sections:
- Policy Objectives and Eligible Personnel:Clarify the policy background and the scope of all employees to whom it applies.
- Approved and Prohibited AI Tools:Provide a clear list of tools that have been evaluated and those that have been banned.
- Work scenarios where the use of AI is permitted:Specify the specific job categories for which the use of AI-assisted tools is permitted.
- Data that can and cannot be entered:Provide clear guidelines and examples of specific scenarios.
- Data Anonymization and Data Minimization Requirements:Describes the standards for de-identification prior to uploading data.
- Accounts, Devices, Permissions, and Security Settings:Define MFA, SSO, and least privilege settings.
- Rules for Reviewing and Using AI-Generated Content:Establish a "human-in-the-loop" process and annotation guidelines.
- Procedure for Reporting Accidents and Abnormal Situations:Establish a non-punitive, rapid reporting channel and standard operating procedures (SOPs) for handling incidents.
- Training, Inquiries, and Employee Feedback Channels:Provides access to ongoing training courses and information security consulting services.
- Consequences of Violations, Review Frequency, and Responsible Parties:Clarify supervisory responsibilities, establish a regular review mechanism, and define accountability for violations.
Policies should not be developed solely by the IT department. A comprehensive working group typically needs to include representatives from information security, privacy, legal, compliance, human resources, procurement, and the business units that actually use AI.
What aspects can KnowBe4 support?
Reducing the risk of personal data leaks in generative AI requires a combination of policy, technology, and human resource management.KnowBe4 Security Awareness Training PlatformIt can assist companies with aspects related to employee education, policy communication, and behavioral improvement, but it cannot replace DLP, access controls, AI gateways, legal assessments, or a comprehensive PDPO compliance program.
Rolling Out AI Security and Privacy Training Across Different Departments
Through the KnowBe4 platform, companies can automatically assign specialized training courses—covering AI data security, personal data privacy, and social engineering defense—to employees across different departments, roles, and risk levels.
Upload your company's AI usage policy
Companies can upload their generative AI usage policies to the KnowBe4 platform, require employees to read and electronically sign them to confirm their understanding, and the system will automatically generate a compliance audit trail.
Create a Quiz for Internal Policies
Develop scenario-based tests to support internal AI policies and assess whether employees understand the key requirements and incident reporting procedures.
Create groups based on department, position, and risk
Use KnowBe4’s custom content feature to schedule follow-up content for employees who haven’t completed training or are underperforming, and use reports to view training completion rates, quiz results, and risk trends.
Security Tip:The KnowBe4 security awareness training platform does not automatically scan all content that employees enter into public AI tools, nor can it prevent the upload of personal data on its own. Organizations must still implement appropriate technical controls based on risk and, when deploying training and risk scoring features, properly manage employee data, access permissions, retention periods, and internal communications.
Companies can start with these four steps
If an organization has not yet established a generative AI management framework, it can take the following steps:
1
Step 1: List the tools
Find out which AI tools are being used in each department, and distinguish between approved, pending evaluation, and prohibited services.
2
Step 2: Establish Data Boundaries
Determine whether customer information, employee information, account records, contracts, financial data, and trade secrets may be entered, and what de-identification measures are required.
3
Step 3: Issue Clear Guidelines and Conduct Training
First, make sure employees clearly understand “what they can and cannot do, and who to turn to if they make a mistake,” and then gradually refine the governance system to make it more detailed.
4
Step 4: Establish a Reporting and Review Mechanism
Record incidents, employee inquiries, and exception approvals, and continuously adjust the list of tools, permissions, and training content based on actual usage.
Conclusion
The risks to personal data posed by generative AI often do not stem from complex technical attacks, but rather from a seemingly ordinary action: employees entering excessive amounts of data into unsuitable tools in an effort to improve efficiency.
A complete ban on AI may not be realistic, and leaving employees to make their own judgments is equally undesirable. A more feasible approach is to provide employees with approved tools, clear data boundaries, easy-to-understand operating guidelines, and reporting channels they can use immediately if problems arise.
What companies truly need to establish is not a static “AI blacklist,” but a management mechanism that can be continuously adjusted in response to changes in tools, business operations, and risks: understanding how employees use AI, controlling what data AI can access, ensuring that outputs are properly reviewed, and making sure every employee understands their responsibilities. This is the foundation for companies to enhance the efficiency of AI applications while reducing the risk of personal data breaches.
