AI Is Your Coworker Now. Can You Trust It?
Generative AI tools such as OpenAI’s ChatGPT and Microsoft’s Copilot are becoming part of everyday business life. But they come with privacy and security considerations you should know about.
Generative AI tools such as OpenAI’s ChatGPT and Microsoft’s Copilot are rapidly evolving, fueling concerns that the technology could open the door to multiple privacy and security issues, particularly in the workplace.
In May, privacy campaigners dubbed Microsoft’s new Recall tool a potential “privacy nightmare” due to its ability to take screenshots of your laptop every few seconds. The feature has caught the attention of UK regulator the Information Commissioner’s Office, which is asking Microsoft to reveal more about the safety of the product launching soon in its Copilot PCs.
Concerns are also mounting over OpenAI’s ChatGPT, which has demonstrated screenshotting abilities in its soon-to-launch macOS app that privacy experts say could result in the capture of sensitive data.
The US House of Representatives has banned the use of Microsoft’s Copilot among staff members after it was deemed by the Office of Cybersecurity to be a risk to users due to “the threat of leaking House data to non-House approved cloud services.”
Meanwhile, market analyst Gartner has cautioned that “using Copilot for Microsoft 365 exposes the risks of sensitive data and content exposure internally and externally.” And last month, Google was forced to make adjustments to its new search feature, AI Overviews, after screenshots of bizarre and misleading answers to queries went viral.
Overexposed
For those using generative AI at work, one of the biggest challenges is the risk of inadvertently exposing sensitive data. Most generative AI systems are “essentially big sponges,” says Camden Woollven, group head of AI at risk management firm GRC International Group. “They soak up huge amounts of information from the internet to train their language models.”
AI companies are “hungry for data to train their models,” and are “seemingly making it behaviorally attractive” to do so, says Steve Elcock, CEO and founder at software firm Elementsuite. This vast amount of data collection means there’s the potential for sensitive information to be put “into somebody else’s ecosystem,” says Jeff Watkins, chief product and technology officer at digital consultancy xDesign. “It could also later be extracted through clever prompting.”
At the same time, there’s the threat of AI systems themselves being targeted by hackers. “Theoretically, if an attacker managed to gain access to the large language model (LLM) that powers a company's AI tools, they could siphon off sensitive data, plant false or misleading outputs, or use the AI to spread malware,” says Woollven.
Consumer-grade AI tools can create obvious risks. However, an increasing number of potential issues are arising with “proprietary” AI offerings broadly deemed safe for work such as Microsoft Copilot, says Phil Robinson, principal consultant at security consultancy Prism Infosec.
“This could theoretically be used to look at sensitive data if access privileges have not been locked down. We could see employees asking to see pay scales, M&A activity, or documents containing credentials, which could then be leaked or sold.”
Another concern centers around AI tools that could be used to monitor staff, potentially infringing their privacy. Microsoft’s Recall feature states that “your snapshots are yours; they stay locally on your PC” and “you are always in control with privacy you can trust.”
Yet “it doesn’t seem very long before this technology could be used for monitoring employees,” says Elcock.
Self-Censorship
Generative AI does pose several potential risks, but there are steps businesses and individual employees can take to improve privacy and security. First, do not put confidential information into a prompt for a publicly available tool such as ChatGPT or Google’s Gemini, says Lisa Avvocato, vice president of marketing and community at data firm Sama.
When crafting a prompt, be generic to avoid sharing too much. “Ask, ‘Write a proposal template for budget expenditure,’ not ‘Here is my budget, write a proposal for expenditure on a sensitive project,’” she says. “Use AI as your first draft, then layer in the sensitive information you need to include.”
If you use it for research, avoid issues such as those seen with Google’s AI Overviews by validating what it provides, says Avvocato. “Ask it to provide references and links to its sources. If you ask AI to write code, you still need to review it, rather than assuming it’s good to go.”
Microsoft has itself stated that Copilot needs to be configured correctly and the “least privilege”—the concept that users should only have access to the information they need—should be applied. This is “a crucial point,” says Prism Infosec’s Robinson. “Organizations must lay the groundwork for these systems and not just trust the technology and assume everything will be OK.”
It’s also worth noting that ChatGPT uses the data you share to train its models, unless you turn it off in the settings or use the enterprise version.
Read more:
archive.ph/v2xkc