Google Online Security Blog

I/O 2024: What’s new in Android security and privacy

May 15, 2024

Stronger protections against fraud and scams

We’re also bringing additional protections to fight fraud and scams in Android 15 with two key enhancements to safeguard your information and privacy from bad apps:

Protecting One-time Passwords from Malware: With the exception of a few types of apps, such as wearable companion apps, one-time passwords are now hidden from notifications, closing a common attack vector for fraud and spyware.
Expanded Restricted Settings: To help protect more sensitive permissions that are commonly abused by fraudsters, we’re expanding Android 13’s restricted settings, which require additional user approval to enable permissions when installing an app from an Internet-sideloading source (web browsers, messaging apps or file managers).

Protecting against screen-sharing social engineering attacks

We’re also tightening controls for screen sharing in Android 15 to limit social engineering attacks that try to view your screen and steal information, while introducing new safeguards to further shield your sensitive information:

Automatically Hidden Notifications and One-time Passwords (OTPs): During screen sharing, private notification content will be hidden, preventing remote viewers from seeing details in a user's notifications. Apps that post OTPs in notifications will be automatically protected from remote viewers when you’re screen sharing, helping thwart attempts to steal sensitive data.
Safer Logins: Your screen will be hidden when you enter credentials like usernames, passwords and credit card numbers during a screen-share session.
Choose What You Share: Currently available on Pixel, other Android devices will also have the ability to share just one app's content rather than your whole screen to help preserve your screen privacy.

Having clear content sharing indicators is important for users to understand when their data is visible. A new, more prominent screen indicator coming to Android devices later this year will always let you know when screen sharing is active, and you can stop sharing with a simple tap.

Advanced cellular security to fight fraud and surveillance

We’re adding new advanced cellular protections in Android 15 to defend against abuse by criminals using cell site simulators to snoop on users or send them SMS-based fraud messages.

Cellular Cipher Transparency: We’ll notify you if your cellular network connection is unencrypted, potentially exposing voice and SMS traffic to radio interception, and potentially visible to others. This can help warn users if they’re being targeted by criminals who are trying to intercept their traffic or inject a fraud SMS message.
Identifier Disclosure Transparency: We’ll help at risk-users like journalists or dissidents by alerting them if a potential false cellular base station or surveillance tool is recording their location using a device identifier.

These features require device OEM integration and compatible hardware. We are working with the Android ecosystem to bring these features to users. We expect OEM adoption to progress over the next couple of years.

More security tools for developers to fight fraud and scams

Safeguarding apps from scams and fraud is an ongoing battle for developers. The Play Integrity API lets developers check that their apps are unmodified and running on a genuine Android device so that they can detect fraudulent or risky behavior and take actions to prevent attacks and abuse. We’ve updated the API with new in-app signals to help developers secure their apps against new threats:

Risk From Screen Capturing or Remote Access: Developers can check if there are other apps running that could be capturing the screen, creating overlays, or controlling the device. This is helpful for apps that want to hide sensitive information from other apps and protect users from scams.
Risk From Known Malware: Developers can check if Google Play Protect is active and the user device is free of known malware before performing sensitive actions or handling sensitive data. This is particularly valuable for financial and banking apps, adding another layer of security to protect user information.
Risk From Anomalous Devices: Developers can also opt-in to receive recent device activity to check if a device is making too many integrity checks, which could be a sign of an attack.

Developers can decide how their apps respond to these signals, such as prompting the user to close risky apps or turn on Google Play Protect before continuing.

Upgraded policies and tools for developers to enhance user privacy

We’re working to make photo permissions even more private for users. Starting this year, apps on Play must demonstrate that they require broad access to use the photo or video permissions. Google Play will start enforcing this policy in August. We’ve updated photo picker, Android’s preferred solution for granting individual access to photos and videos without requiring broad permissions. Photo picker now includes support for cloud storage services like Google Photos. It’s much easier to find the right photo by browsing albums and favorites. Coming later this year, photo picker will support local and cloud search as well.

Always evolving our multi-layered protections

Android's commitment to user safety is unwavering. We're constantly evolving our multi-layered user protections – combining the power of advanced AI with close partnerships across OEMs, the Android ecosystem, and the security research community. Building a truly secure Android experience is a collaborative effort, and we'll continue to work tirelessly to safeguard your device and data.

Posted by Dave Kleidermacher, VP Engineering, Android Security and Privacy Our commitment to user safety is a top priority for Android. We’ve been consistently working to stay ahead of the world’s scammers, fraudsters and bad actors. And as their tactics evolve in sophistication and scale, we continually adapt and enhance our advanced security features and AI-powered protections to help keep Android users safe. In addition to our new suite of advanced theft protection features to help keep your device and data safe in the case of theft, we’re also focusing increasingly on providing additional protections against mobile financial fraud and scams.
Today, we’re announcing more new fraud and scam protection features coming in Android 15 and Google Play services updates later this year to help better protect users around the world. We’re also sharing new tools and policies to help developers build safer apps and keep their users safe. Google Play Protect live threat detection Google Play Protect now scans 200 billion Android apps daily, helping keep more than 3 billion users safe from malware. We are expanding Play Protect’s on-device AI capabilities with Google Play Protect live threat detection to improve fraud and abuse detection against apps that try to cloak their actions. With live threat detection, Google Play Protect’s on-device AI will analyze additional behavioral signals related to the use of sensitive permissions and interactions with other apps and services. If suspicious behavior is discovered, Google Play Protect can send the app to Google for additional review and then warn users or disable the app if malicious behavior is confirmed. The detection of suspicious behavior is done on device in a privacy preserving way through Private Compute Core, which allows us to protect users without collecting data. Google Pixel, Honor, Lenovo, Nothing, OnePlus, Oppo, Sharp, Transsion, and other manufacturers are deploying live threat detection later this year.

Stronger protections against fraud and scams We’re also bringing additional protections to fight fraud and scams in Android 15 with two key enhancements to safeguard your information and privacy from bad apps:

Protecting One-time Passwords from Malware: With the exception of a few types of apps, such as wearable companion apps, one-time passwords are now hidden from notifications, closing a common attack vector for fraud and spyware.

Expanded Restricted Settings: To help protect more sensitive permissions that are commonly abused by fraudsters, we’re expanding Android 13’s restricted settings, which require additional user approval to enable permissions when installing an app from an Internet-sideloading source (web browsers, messaging apps or file managers).

We are continuing to develop new, AI-powered protections, like the scam call detection capability that we’re testing, which uses on-device Gemini-Nano AI to warn users in real-time when it detects conversation patterns commonly associated with fraud and scams. Protecting against screen-sharing social engineering attacks

We’re also tightening controls for screen sharing in Android 15 to limit social engineering attacks that try to view your screen and steal information, while introducing new safeguards to further shield your sensitive information:

Automatically Hidden Notifications and One-time Passwords (OTPs): During screen sharing, private notification content will be hidden, preventing remote viewers from seeing details in a user's notifications. Apps that post OTPs in notifications will be automatically protected from remote viewers when you’re screen sharing, helping thwart attempts to steal sensitive data.

Safer Logins: Your screen will be hidden when you enter credentials like usernames, passwords and credit card numbers during a screen-share session.

Choose What You Share: Currently available on Pixel, other Android devices will also have the ability to share just one app's content rather than your whole screen to help preserve your screen privacy.

Advanced cellular security to fight fraud and surveillance
We’re adding new advanced cellular protections in Android 15 to defend against abuse by criminals using cell site simulators to snoop on users or send them SMS-based fraud messages.

Cellular Cipher Transparency: We’ll notify you if your cellular network connection is unencrypted, potentially exposing voice and SMS traffic to radio interception, and potentially visible to others. This can help warn users if they’re being targeted by criminals who are trying to intercept their traffic or inject a fraud SMS message.

Identifier Disclosure Transparency: We’ll help at risk-users like journalists or dissidents by alerting them if a potential false cellular base station or surveillance tool is recording their location using a device identifier.

More security tools for developers to fight fraud and scams Safeguarding apps from scams and fraud is an ongoing battle for developers. The Play Integrity API lets developers check that their apps are unmodified and running on a genuine Android device so that they can detect fraudulent or risky behavior and take actions to prevent attacks and abuse. We’ve updated the API with new in-app signals to help developers secure their apps against new threats:

Risk From Screen Capturing or Remote Access: Developers can check if there are other apps running that could be capturing the screen, creating overlays, or controlling the device. This is helpful for apps that want to hide sensitive information from other apps and protect users from scams.

Risk From Known Malware: Developers can check if Google Play Protect is active and the user device is free of known malware before performing sensitive actions or handling sensitive data. This is particularly valuable for financial and banking apps, adding another layer of security to protect user information.

Risk From Anomalous Devices: Developers can also opt-in to receive recent device activity to check if a device is making too many integrity checks, which could be a sign of an attack.

Developers can decide how their apps respond to these signals, such as prompting the user to close risky apps or turn on Google Play Protect before continuing. Upgraded policies and tools for developers to enhance user privacy We’re working to make photo permissions even more private for users. Starting this year, apps on Play must demonstrate that they require broad access to use the photo or video permissions. Google Play will start enforcing this policy in August. We’ve updated photo picker, Android’s preferred solution for granting individual access to photos and videos without requiring broad permissions. Photo picker now includes support for cloud storage services like Google Photos. It’s much easier to find the right photo by browsing albums and favorites. Coming later this year, photo picker will support local and cloud search as well.

Always evolving our multi-layered protections Android's commitment to user safety is unwavering. We're constantly evolving our multi-layered user protections – combining the power of advanced AI with close partnerships across OEMs, the Android ecosystem, and the security research community. Building a truly secure Android experience is a collaborative effort, and we'll continue to work tirelessly to safeguard your device and data.

Google and Apple deliver support for unwanted tracking alerts in Android and iOS

May 13, 2024

With this new capability, Android users will now get a “Tracker traveling with you” alert on their device if an unknown Bluetooth tracking device is seen moving with them over time, regardless of the platform the device is paired with.

If a user gets such an alert on their Android device, it means that someone else’s AirTag, Find My Device network-compatible tracker tag, or other industry specification-compatible Bluetooth tracker is moving with them. Android users can view the tracker’s identifier, have the tracker play a sound to help locate it, and access instructions to disable it. Bluetooth tag manufacturers including Chipolo, eufy, Jio, Motorola, and Pebblebee have committed that future tags will be compatible.

Google’s Find My Device is secure by default and private by design. Multi-layered user protections, including first of its kind safety-first protections, help mitigate potential risks to user privacy and safety while allowing users to effectively locate and recover lost devices. This cross-platform collaboration — an industry first, involving community and industry input — offers instructions and best practices for manufacturers, should they choose to build unwanted tracking alert capabilities into their products. Google and Apple will continue to work with the Internet Engineering Task Force via the Detecting Unwanted Location Trackers working group to develop the official standard for this technology.

Google and Apple have worked together to create an industry specification – Detecting Unwanted Location Trackers – for Bluetooth tracking devices that makes it possible to alert users across both Android and iOS if such a device is unknowingly being used to track them. This will help mitigate the misuse of devices designed to help keep track of belongings. Google is now launching this capability on Android 6.0+ devices, and today Apple is implementing this capability in iOS 17.5.
With this new capability, Android users will now get a “Tracker traveling with you” alert on their device if an unknown Bluetooth tracking device is seen moving with them over time, regardless of the platform the device is paired with. If a user gets such an alert on their Android device, it means that someone else’s AirTag, Find My Device network-compatible tracker tag, or other industry specification-compatible Bluetooth tracker is moving with them. Android users can view the tracker’s identifier, have the tracker play a sound to help locate it, and access instructions to disable it. Bluetooth tag manufacturers including Chipolo, eufy, Jio, Motorola, and Pebblebee have committed that future tags will be compatible. Google’s Find My Device is secure by default and private by design. Multi-layered user protections, including first of its kind safety-first protections, help mitigate potential risks to user privacy and safety while allowing users to effectively locate and recover lost devices. This cross-platform collaboration — an industry first, involving community and industry input — offers instructions and best practices for manufacturers, should they choose to build unwanted tracking alert capabilities into their products. Google and Apple will continue to work with the Internet Engineering Task Force via the Detecting Unwanted Location Trackers working group to develop the official standard for this technology.

Your Google Account allows you to create passkeys on your phone, computer and security keys

May 2, 2024

Last year, Google launched passkey support for Google Accounts. Passkeys are a new industry standard that give users an easy, highly secure way to sign-in to apps and websites. Today, we announced that passkeys have been used to authenticate users more than 1 billion times across over 400 million Google Accounts.

As more users encounter passkeys, we’re often asked questions about how they relate to security keys, how Google Workspace administrators can configure passkeys for the user accounts that they manage, and how they relate to the Advanced Protection Program (APP). This post will seek to clarify these topics.

Passkeys and security keys

Passkeys are an evolution of security keys, meaning users get the same security benefits, but with a much simplified experience. Passkeys can be used in the Google Account sign-in process in many of the same ways that security keys have been used in the past — in fact, you can now choose to store your passkey on your security key. This provides users with three key benefits:

Stronger security. Users typically authenticate with passkeys by entering their device’s screen lock PIN, or using a biometric authentication method, like a fingerprint or a face scan. By storing the passkey on a security key, users can ensure that passkeys are only available when the security key is plugged into their device, creating a stronger security posture.

Flexible portability. Today, users rely on password managers to make passkeys available across all of their devices. Security keys provide an alternate way to use your passkeys across your devices: by bringing your security keys with you.

Simpler sign-in. Passkeys can act as a first- and second-factor, simultaneously. By creating a passkey on your security key, you can skip entering your password. This replaces your remotely stored password with the PIN you used to unlock your security key, which improves user security. (If you prefer to continue using your password in addition to using a passkey, you can turn off “Skip password when possible” in your Google Account security settings.)

Passkeys bring strong and phishing-resistant authentication technology to a wider user base, and we’re excited to offer this new way for passkeys to meet more user needs.

Google Workspace admins have additional controls and choice

Google Workspace accounts have a domain level “Allow users to skip passwords at sign-in by using passkeys” setting which is off by default, and overrides the corresponding user-level configuration. This retains the need for a user’s password in addition to presenting a passkey. Admins can also change that setting and allow users to sign-in with just a passkey.

When the domain-level setting is off, end users will still see a “use a security key” button on their “passkeys and security keys” page, which will attempt to enroll any security key for use as a second factor only. This action will not require the user to set up a PIN for their security key during registration. This is designed to give enterprise customers who have deployed legacy security keys additional time to make the change to passkeys, with or without a password.

Passkeys for Advanced Protection Program (APP) users

Since the introduction of passkeys in 2023, users enrolled in APP have been able to add any passkey to their account and use it to sign in. However users are still required to present two security keys when enrolling into the program. We will be updating the enrollment process soon to enable a user with any passkey to enroll in APP. By allowing any passkey to be used (rather than only hardware security keys) we expect to reach more high risk users who need advanced protection, while maintaining phishing-resistant authentication.

Sriram Karra and Christiaan Brand, Google product managers

Passkeys and security keys

Stronger security. Users typically authenticate with passkeys by entering their device’s screen lock PIN, or using a biometric authentication method, like a fingerprint or a face scan. By storing the passkey on a security key, users can ensure that passkeys are only available when the security key is plugged into their device, creating a stronger security posture.

Flexible portability. Today, users rely on password managers to make passkeys available across all of their devices. Security keys provide an alternate way to use your passkeys across your devices: by bringing your security keys with you.

Simpler sign-in. Passkeys can act as a first- and second-factor, simultaneously. By creating a passkey on your security key, you can skip entering your password. This replaces your remotely stored password with the PIN you used to unlock your security key, which improves user security. (If you prefer to continue using your password in addition to using a passkey, you can turn off “Skip password when possible” in your Google Account security settings.)

Passkeys bring strong and phishing-resistant authentication technology to a wider user base, and we’re excited to offer this new way for passkeys to meet more user needs.

Google Workspace admins have additional controls and choice

Passkeys for Advanced Protection Program (APP) users

Detecting browser data theft using Windows Event Logs

April 30, 2024

The issue with the 4693 event is that while it is generated if there is DPAPI activity on the system, it unfortunately does not contain information about which process was performing the DPAPI activity, nor does it contain information about which particular secret is being accessed. This is because the Execution ProcessID field in the event will always be the process id of lsass.exe because it is this process that manages the encryption keys for the system, and there is no entry for the description of the data.

It was for this reason that, in recent versions of Windows a new event type was added to help identify the process making the DPAPI call directly. This event was added to the Microsoft-Windows-Crypto-DPAPI stream which manifests in the Event Log in the Applications and Services Logs > Microsoft > Windows > Crypto-DPAPI part of the Event Viewer tree.

The new event is called DPAPIDefInformationEvent and has id 16385, but unfortunately is only emitted to the Debug channel and by default this is not persisted to an Event Log, unless Debug channel logging is enabled. This can be accomplished by enabling it directly in powershell:

&dollar;log = &grave;&NewLine; New-Object System&period;Diagnostics&period;Eventing&period;Reader&period;EventLogConfiguration &grave;&NewLine; Microsoft-Windows-Crypto-DPAPI/Debug&NewLine;&dollar;log&period;IsEnabled = &dollar;True&NewLine;&dollar;log&period;SaveChanges()&NewLine;

Once this log is enabled then you should start to see 16385 events generated, and these will contain the real process ids of applications performing DPAPI operations. Note that 16385 events are emitted by the operating system even for data not flagged with CRYPTPROTECT_AUDIT, but to identify the data as owned by the browser, the data description is essential. 16385 events are described later.

You will also want to enable Audit Process Creation in order to be able to know a current mapping of process ids to process names — more details on that later. You might want to also consider enabling logging of full command lines.

Step 2: Collect the events

The events you want to collect are:

From Security log:

4688: "A new process was created."

From Microsoft-Windows-Crypto-DPAPI/Debug log: (enabled above)

16385: "DPAPIDefInformationEvent"

These should be collected from all workstations, and persisted into your enterprise logging system for analysis.

Step 3: Write detection logic to detect theft.

With these two events is it now possible to detect when an unauthorized application calls into DPAPI to try and decrypt browser secrets.

The general approach is to generate a map of process ids to active processes using the 4688 events, then every time a 16385 event is generated, it is possible to identify the currently running process, and alert if the process does not match an authorized application such as Google Chrome. You might find your enterprise logging software can already keep track of which process ids map to which process names, so feel free to just use that existing functionality.

Let's dive deeper into the events.

A 4688 event looks like this - e.g. here is Chrome browser launching from explorer:

<Event xmlns="http&colon;//schemas&period;microsoft&period;com/win/2004/08/events/event">&NewLine; <System>&NewLine; <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{...}" />&NewLine; <EventID>4688</EventID>&NewLine; <Version>2</Version>&NewLine; <Level>0</Level>&NewLine; <Task>13312</Task>&NewLine; <Opcode>0</Opcode>&NewLine; <Keywords>0x8020000000000000</Keywords>&NewLine; <TimeCreated SystemTime="2024-03-28T20&colon;06&colon;41&period;9254105Z" />&NewLine; <EventRecordID>78258343</EventRecordID>&NewLine; <Correlation />&NewLine; <Execution ProcessID="4" ThreadID="54256" />&NewLine; <Channel>Security</Channel>&NewLine; <Computer>WIN-GG82ULGC9GO&period;contoso&period;local</Computer>&NewLine; <Security />&NewLine; </System>&NewLine; <EventData>&NewLine; <Data Name="SubjectUserSid">S-1-5-18</Data>&NewLine; <Data Name="SubjectUserName">WIN-GG82ULGC9GO&dollar;</Data>&NewLine; <Data Name="SubjectDomainName">CONTOSO</Data>&NewLine; <Data Name="SubjectLogonId">0xe8c85cc</Data>&NewLine; <Data Name="NewProcessId">0x17eac</Data>&NewLine; <Data Name="NewProcessName">C&colon;\Program Files\Google\Chrome\Application\chrome&period;exe</Data>&NewLine; <Data Name="TokenElevationType">&percnt;&percnt;1938</Data>&NewLine; <Data Name="ProcessId">0x16d8</Data>&NewLine; <Data Name="CommandLine">"C&colon;\Program Files\Google\Chrome\Application\chrome&period;exe" </Data>&NewLine; <Data Name="TargetUserSid">S-1-0-0</Data>&NewLine; <Data Name="TargetUserName">-</Data>&NewLine; <Data Name="TargetDomainName">-</Data>&NewLine; <Data Name="TargetLogonId">0x0</Data>&NewLine; <Data Name="ParentProcessName">C&colon;\Windows\explorer&period;exe</Data>&NewLine; <Data Name="MandatoryLabel">S-1-16-8192</Data>&NewLine; </EventData>&NewLine;</Event>&NewLine;

The important part here is the NewProcessId, in hex 0x17eac which is 97964.

A 16385 event looks like this:

<Event xmlns="http&colon;//schemas&period;microsoft&period;com/win/2004/08/events/event">&NewLine; <System>&NewLine; <Provider Name="Microsoft-Windows-Crypto-DPAPI" Guid="{...}" />&NewLine; <EventID>16385</EventID>&NewLine; <Version>0</Version>&NewLine; <Level>4</Level>&NewLine; <Task>64</Task>&NewLine; <Opcode>0</Opcode>&NewLine; <Keywords>0x2000000000000040</Keywords>&NewLine; <TimeCreated SystemTime="2024-03-28T20&colon;06&colon;42&period;1772585Z" />&NewLine; <EventRecordID>826993</EventRecordID>&NewLine; <Correlation ActivityID="{777bf68d-7757-0028-b5f6-7b775777da01}" />&NewLine; <Execution ProcessID="1392" ThreadID="57108" />&NewLine; <Channel>Microsoft-Windows-Crypto-DPAPI/Debug</Channel>&NewLine; <Computer>WIN-GG82ULGC9GO&period;contoso&period;local</Computer>&NewLine; <Security UserID="S-1-5-18" />&NewLine; </System>&NewLine; <EventData>&NewLine; <Data Name="OperationType">SPCryptUnprotect</Data>&NewLine; <Data Name="DataDescription">Google Chrome</Data>&NewLine; <Data Name="MasterKeyGUID">{4df0861b-07ea-49f4-9a09-1d66fd1131c3}</Data>&NewLine; <Data Name="Flags">0</Data>&NewLine; <Data Name="ProtectionFlags">16</Data>&NewLine; <Data Name="ReturnValue">0</Data>&NewLine; <Data Name="CallerProcessStartKey">32651097299526713</Data>&NewLine; <Data Name="CallerProcessID">97964</Data>&NewLine; <Data Name="CallerProcessCreationTime">133561300019253302</Data>&NewLine; <Data Name="PlainTextDataSize">32</Data>&NewLine; </EventData>&NewLine;</Event>&NewLine;

The important parts here are the OperationType, the DataDescription and the CallerProcessID.

For DPAPI decrypts, the OperationType will be SPCryptUnprotect.

Each Chromium based browser will tag its data with the product name, e.g. Google Chrome, or Microsoft Edge depending on the owner of the data. This will always appear in the DataDescription field, so it is possible to distinguish browser data from other DPAPI secured data.

Finally, the CallerProcessID will map to the process performing the decryption. In this case, it is 97964 which matches the process ID seen in the 4688 event above, showing that this was likely Google Chrome decrypting its own data! Bear in mind that since these logs only contain the path to the executable, for a full assurance that this is actually Chrome (and not malware pretending to be Chrome, or malware injecting into Chrome), additional protections such as removing administrator access, and application allowlisting could also be used to give a higher assurance of this signal. In recent versions of Chrome or Edge, you might also see logs of decryptions happening in the elevation_service.exe process, which is another legitimate part of the browser's data storage.

To detect unauthorized DPAPI access, you will want to generate a running map of all processes using 4688 events, then look for 16385 events that have a CallerProcessID that does not match a valid caller – Let's try that now.

Testing with a python password stealer

We can test that this works with a public script to decrypt passwords taken from a public blog. It generates two events, as expected:

Here is the 16385 event, showing that a process is decrypting the "Google Chrome" key.

<Event xmlns="http&colon;//schemas&period;microsoft&period;com/win/2004/08/events/event">&NewLine; <System>&NewLine; < &period;&period;&period; >&NewLine; <EventID>16385</EventID>&NewLine; < &period;&period;&period; >&NewLine; <TimeCreated SystemTime="2024-03-28T20&colon;28&colon;13&period;7891561Z" />&NewLine; < &period;&period;&period; >&NewLine; </System>&NewLine; <EventData>&NewLine; <Data Name="OperationType">SPCryptUnprotect</Data>&NewLine; <Data Name="DataDescription">Google Chrome</Data>&NewLine; < &period;&period;&period; >&NewLine; <Data Name="CallerProcessID">68768</Data>&NewLine; <Data Name="CallerProcessCreationTime">133561312936527018</Data>&NewLine; <Data Name="PlainTextDataSize">32</Data>&NewLine; </EventData>&NewLine;</Event>

Since the data description being decrypted was "Google Chrome" we know this is an attempt to read Chrome secrets, but to determine the process behind 68768 (0x10ca0), we need to correlate this with a 4688 event.

Here is the corresponding 4688 event from the Security Log (a process start for python3.exe) with the matching process id:

<Event xmlns="http&colon;//schemas&period;microsoft&period;com/win/2004/08/events/event">&NewLine; <System>&NewLine; < &period;&period;&period; >&NewLine; <EventID>4688</EventID>&NewLine; < &period;&period;&period; >&NewLine; <TimeCreated SystemTime="2024-03-28T20&colon;28&colon;13&period;6527871Z" />&NewLine; < &period;&period;&period; >&NewLine; </System>&NewLine; <EventData>&NewLine; < &period;&period;&period; >&NewLine; <Data Name="NewProcessId">0x10ca0</Data>&NewLine; <Data Name="NewProcessName">C&colon;\python3\bin\python3&period;exe</Data>&NewLine; <Data Name="TokenElevationType">&percnt;&percnt;1938</Data>&NewLine; <Data Name="ProcessId">0xca58</Data>&NewLine; <Data Name="CommandLine">"c&colon;\python3\bin\python3&period;exe" steal&lowbar;passwords&period;py</Data>&NewLine; < &period;&period;&period; >&NewLine; <Data Name="ParentProcessName">C&colon;\Windows\System32\cmd&period;exe</Data>&NewLine; </EventData>&NewLine;</Event>

In this case, the process id matches the python3 executable running a potentially malicious script, so we know this is likely very suspicious behavior, and should trigger an alert immediately! Bear in mind process ids on Windows are not unique so you will want to make sure you use the 4688 event with the timestamp closest, but earlier than, the 16385 event.

Summary

This blog has described a technique for strong detection of cookie and credential theft. We hope that all defenders find this post useful. Thanks to Microsoft for adding the DPAPIDefInformationEvent log type, without which this would not be possible.

Accelerating incident response using generative AI

April 26, 2024

Introduction

As security professionals, we're constantly looking for ways to reduce risk and improve our workflow's efficiency. We've made great strides in using AI to identify malicious content, block threats, and discover and fix vulnerabilities. We also published the Secure AI Framework (SAIF), a conceptual framework for secure AI systems to ensure we are deploying AI in a responsible manner.

Today we are highlighting another way we use generative AI to help the defenders gain the advantage: Leveraging LLMs (Large Language Model) to speed-up our security and privacy incidents workflows.

Incident management is a team sport. We have to summarize security and privacy incidents for different audiences including executives, leads, and partner teams. This can be a tedious and time-consuming process that heavily depends on the target group and the complexity of the incident. We estimate that writing a thorough summary can take nearly an hour and more complex communications can take multiple hours. But we hypothesized that we could use generative AI to digest information much faster, freeing up our incident responders to focus on other more critical tasks - and it proved true. Using generative AI we could write summaries 51% faster while also improving the quality of them.

Our incident response approach

When suspecting a potential data incident, for example,we follow a rigorous process to manage it. From the identification of the problem, the coordination of experts and tools, to its resolution and then closure. At Google, when an incident is reported, our Detection & Response teams work to restore normal service as quickly as possible, while meeting both regulatory and contractual compliance requirements. They do this by following the five main steps in the Google incident response program:

Identification: Monitoring security events to detect and report on potential data incidents using advanced detection tools, signals, and alert mechanisms to provide early indication of potential incidents.
Coordination: Triaging the reports by gathering facts and assessing the severity of the incident based on factors such as potential harm to customers, nature of the incident, type of data that might be affected, and the impact of the incident on customers. A communication plan with appropriate leads is then determined.
Resolution: Gathering key facts about the incident such as root cause and impact, and integrating additional resources as needed to implement necessary fixes as part of remediation.
Closure: After the remediation efforts conclude, and after a data incident is resolved, reviewing the incident and response to identify key areas for improvement.
Continuous improvement: Is crucial for the development and maintenance of incident response programs. Teams work to improve the program based on lessons learned, ensuring that necessary teams, training, processes, resources, and tools are maintained.

Google’s Incident Response Process diagram flow

Leveraging generative AI

Our detection and response processes are critical in protecting our billions of global users from the growing threat landscape, which is why we’re continuously looking for ways to improve them with the latest technologies and techniques. The growth of generative AI has brought with it incredible potential in this area, and we were eager to explore how it could help us improve parts of the incident response process. We started by leveraging LLMs to not only pioneer modern approaches to incident response, but also to ensure that our processes are efficient and effective at scale.

Managing incidents can be a complex process and an additional factor is effective internal communication to leads, executives and stakeholders on the threats and status of incidents. Effective communication is critical as it properly informs executives so that they can take any necessary actions, as well as to meet regulatory requirements. Leveraging LLMs for this type of communication can save significant time for the incident commanders while improving quality at the same time.

Humans vs. LLMs

Given that LLMs have summarization capabilities, we wanted to explore if they are able to generate summaries on par, or as well as humans can. We ran an experiment that took 50 human-written summaries from native and non-native English speakers, and 50 LLM-written ones with our finest (and final) prompt, and presented them to security teams without revealing the author.

We learned that the LLM-written summaries covered all of the key points, they were rated 10% higher than their human-written equivalents, and cut the time necessary to draft a summary in half.

Comparison of human vs LLM content completeness

Comparison of human vs LLM writing styles

Managing risks and protecting privacy

Leveraging generative AI is not without risks. In order to mitigate the risks around potential hallucinations and errors, any LLM generated draft must be reviewed by a human. But not all risks are from the LLM - human misinterpretation of a fact or statement generated by the LLM can also happen. That is why it’s important to ensure there is human accountability, as well as to monitor quality and feedback over time.

Given that our incidents can contain a mixture of confidential, sensitive, and privileged data, we had to ensure we built an infrastructure that does not store any data. Every component of this pipeline - from the user interface to the LLM to output processing - has logging turned off. And, the LLM itself does not use any input or output for re-training. Instead, we use metrics and indicators to ensure it is working properly.

Input processing

The type of data we process during incidents can be messy and often unstructured: Free-form text, logs, images, links, impact stats, timelines, and code snippets. We needed to structure all of that data so the LLM “knew” which part of the information serves what purpose. For that, we first replaced long and noisy sections of codes/logs by self-closing tags (<Code Section/> and <Logs/>) both to keep the structure while saving tokens for more important facts and to reduce risk of hallucinations.

During prompt engineering, we refined this approach and added additional tags such as <Title>, <Actions Taken>, <Impact>, <Mitigation History>, <Comment> so the input’s structure becomes closely mirrored to our incident communication templates. The use of self-explanatory tags allowed us to convey implicit information to the model and provide us with aliases in the prompt for the guidelines or tasks, for example by stating “Summarize the <Security Incident>”.

Sample {incident} input

Prompt engineering

Once we added structure to the input, it was time to engineer the prompt. We started simple by exploring how LLMs can view and summarize all of the current incident facts with a short task:

Caption: First prompt version

Limits of this prompt:

The summary was too long, especially for executives trying to understand the risk and impact of the incident
Some important facts were not covered, such as the incident’s impact and its mitigation
The writing was inconsistent and not following our best practices such as “passive voice”, “tense”, “terminology” or “format”
Some irrelevant incident data was being integrated into the summary from email threads
The model struggled to understand what the most relevant and up-to-date information was

For version 2, we tried a more elaborate prompt that would address the problems above: We told the model to be concise and we explained what a well-written summary should be: About the main incident response steps (coordination and resolution).

Second prompt version

Limits of this prompt:

The summaries still did not always succinctly and accurately address the incident in the format we were expecting
At times, the model lost sight of the task or did not take all the guidelines into account
The model still struggled to stick to the latest updates
We noticed a tendency to draw conclusions on hypotheses with some minor hallucinations

For the final prompt, we inserted 2 human-crafted summary examples and introduced a <Good Summary> tag to highlight high quality summaries but also to tell the model to immediately start with the summary without first repeating the task at hand (as LLMs usually do).

Final prompt

This produced outstanding summaries, in the structure we wanted, with all key points covered, and almost without any hallucinations.

Workflow integration

In integrating the prompt into our workflow, we wanted to ensure it was complementing the work of our teams, vs. solely writing communications. We designed the tooling in a way that the UI had a ‘Generate Summary’ button, which would pre-populate a text field with the summary that the LLM proposed. A human user can then either accept the summary and have it added to the incident, do manual changes to the summary and accept it, or discard the draft and start again.

UI showing the ‘generate draft’ button and LLM proposed summary around a fake incident

Quantitative wins

Our newly-built tool produced well-written and accurate summaries, resulting in 51% time saved, per incident summary drafted by an LLM, versus a human.

Time savings using LLM-generated summaries (sample size: 300)

The only edge cases we have seen were around hallucinations when the input size was small in relation to the prompt size. In these cases, the LLM made up most of the summary and key points were incorrect. We fixed this programmatically: If the input size is smaller than 200 tokens, we won’t call the LLM for a summary and let the humans write it.

Evolving to more complex use cases: Executive updates

Given these results, we explored other ways to apply and build upon the summarization success and apply it to more complex communications. We improved upon the initial summary prompt and ran an experiment to draft executive communications on behalf of the Incident Commander (IC). The goal of this experiment was to ensure executives and stakeholders quickly understand the incident facts, as well as allow ICs to relay important information around incidents. These communications are complex because they go beyond just a summary - they include different sections (such as summary, root cause, impact, and mitigation), follow a specific structure and format, as well as adhere to writing best practices (such as neutral tone, active voice instead of passive voice, minimize acronyms).

This experiment showed that generative AI can evolve beyond high level summarization and help draft complex communications. Moreover, LLM-generated drafts, reduced time ICs spent writing executive summaries by 53% of time, while delivering at least on-par content quality in terms of factual accuracy and adherence to writing best practices.

What’s next

We're constantly exploring new ways to use generative AI to protect our users more efficiently and look forward to tapping into its potential as cyber defenders. For example, we are exploring using generative AI as an enabler of ambitious memory safety projects like teaching an LLM to rewrite C++ code to memory-safe Rust, as well as more incremental improvements to everyday security workflows, such as getting generative AI to read design documents and issue security recommendations based on their content.

Security Blog