Gen AI tools pose risks for sensitive data disclosure
Gen AI tools expose sensitive data risks as 8.5 percent of prompts may include critical information
Add bookmark
PEX Network’s weekly news bulletin rounds up the latest research, reports and publications in operational excellence (OPEX), digital transformation, artificial intelligence (AI) and automation, business process management (BPM), process mining and process intelligence and more.
This week includes:
- Gen AI tools pose risks for sensitive data disclosure, study finds
- Automation gains priority while AI use remains fragmented, shows report
- Google’s DeepMind expands AI teams to accelerate research and development
Don't miss any news, updates or insider tips from PEX Network by getting them delivered to your inbox. Sign up to our newsletter and join our community of experts.
Gen AI tools pose risks for sensitive data disclosure, study finds
A recent study by Harmonic Security reveals that nearly 8.5 percent of prompts entered into generative AI (GenAI) tools, such as Microsoft Copilot, OpenAI’s ChatGPT, Google Gemini, Anthropic’s Claude and Perplexity, may expose sensitive information. Analyzing tens of thousands of prompts in Q4 2024, researchers identified customer data, employee records, legal and financial details, security configurations and sensitive source code among the types of information potentially disclosed. Alarmingly, nearly half (45.8 percent) of these cases involved customer billing and authentication details, while 26.8 percent included employee-specific data like payroll information and performance reviews.
Harmonic also highlights the widespread use of free-tier Gen AI services, which often lack enterprise-grade security. Over 63 percent of ChatGPT users and 75 percent of Claude users rely on free plans, despite the associated risks of sensitive data being used to train AI models. Harmonic's CEO, Alastair Paterson, warns of the dangers of data leakage, stating, “While most GenAI usage is benign, the potential exposure of sensitive information requires proactive measures.” He emphasizes the need for real-time monitoring, prompt-level visibility and employee training to mitigate risks.
The study highlights the importance of organizations adopting stricter controls over Gen AI use, such as mandating paid plans that don't train on input data and creating workflows to manage interactions with AI tools. Harmonic Security’s recommendations also include enhanced monitoring and departmental guidelines to ensure secure, responsible use of Gen AI. These steps aim to safeguard critical business data while enabling employees to harness the potential of AI responsibly.
Automation gains priority while AI use remains fragmented, shows report
Automation is a top priority for marketers entering 2025, with 17 percent more identifying it as critical compared to previous periods, according to Mediaocean's “2025 H1 Advertising Outlook Report.” While automation investments are closely linked to generative AI, many organizations continue to adopt AI in a piecemeal fashion. The report, based on a survey of nearly 700 marketing and advertising professionals, highlights that generative AI (63 percent) has overtaken connected television (CTV) as the leading consumer trend. However, the most common AI applications remain tactical, focusing on data analysis (47 percent), market research (38 percent), copywriting (32 percent) and image generation (22 percent).
Despite this momentum, the lack of synchronization between creative and media processes is a glaring gap. Only 14 percent of respondents report full alignment between these functions, highlighting a disconnect that hampers the potential for more integrated strategies. Brian Wieser, CEO of Madison and Wall, advocates for a shift from short-term efficiency gains to comprehensive ecosystem-wide AI integration, emphasizing that true value lies in holistic implementation. Meanwhile, digital media continues to dominate advertising budgets, with significant planned increases in social media, digital display/video and CTV spending.
The report also notes declining concerns over a cookie-less future, with only 31 percent of marketers citing it as a priority compared to 39 percent in 2023. However, as Google introduces consumer opt-out mechanisms, advertisers are exploring multi-ID measurement across devices and browsers to enhance campaign effectiveness. This evolution, combined with the growing importance of synchronized workflows, highlights the need for marketers to adopt forward-looking strategies to stay competitive in a transforming digital landscape.
Google’s DeepMind expands AI teams to accelerate research and development
Google is restructuring its AI operations, transferring more teams to its DeepMind division to accelerate development and bridge the gap between research and implementation. According to Logan Kilpatrick, product lead for Google’s AI Studio platform, the AI Studio team and the team responsible for the API of Google’s Gemini models will now operate under DeepMind. Formed in 2023 through a merger of Google’s DeepMind and Google Brain teams, DeepMind has driven many of Google’s recent AI innovations, including the Gemini series of models.
This shift reflects Google’s ongoing strategy to streamline AI development. The reshuffling aims to strengthen collaboration and improve accessibility. Jaana Dogan, an engineer on one of the reassigned teams, stated the move will make DeepMind’s work more publicly available through better APIs, open-source initiatives and enhanced tools. Google CEO Sundar Pichai emphasized the importance of scaling Gemini for consumer applications in 2025, citing “strong momentum” but also acknowledging a need to “close the gap and establish leadership” in the AI landscape.
The consolidation of developer-focused AI teams follows Google’s earlier integration of its Gemini-powered chatbot team and responsible AI groups into DeepMind. These changes highlight Google’s urgency to accelerate innovation in AI, reflecting Pichai’s message that “the stakes are high” as the company seeks to solidify its position as an industry leader.
READ MORE PEX RESEARCH & REPORTS NEWS
The 26th Annual OPEX Week: Business Transformation World Summit | 2025 Brochure
The renowned OPEX Week: Business Transformation World Summit is returning to Miami on January 27-29, 2025, for an electric gathering of transformation leaders and pioneers shaping the industry. Celebrating 26 years of excellence, we're back with a brand-new festival theme, a show-stopping line-up and fresh content stages. We're gearing up to help you deliver ground-breaking, human-centric transformation in the age of AI.
To view this content, please fill out the form to register and become a member.
Or, if you're already a member, sign in below to view.
Please note: That all fields marked with an asterisk (*) are required.
