Back to blogBlogs

When AI Scales the Attack, Privacy-Enhancing Technologies Can Strengthen the Defence

Jayesh KenaudekarCo-founder & CTOOct 25, 202611 min read
When AI Scales the Attack, Privacy-Enhancing Technologies Can Strengthen the Defence

1. The New Reality: Attacks at Machine Speed

Cybersecurity has always been a race between attackers and defenders. Artificial intelligence is changing the pace of that race.

AI systems can help attackers automate reconnaissance, generate convincing phishing messages, identify potential vulnerabilities and process large volumes of stolen information. Tasks that once required considerable time and manual effort can increasingly be performed at scale, with less human intervention.

The concern is not simply that AI makes attacks more sophisticated. It is that it can make them faster, cheaper to execute and easier to repeat across many targets.

At the same time, organizations are embedding AI into their own operations. Employees use AI assistants to work with documents, customer records and internal knowledge. Automated agents retrieve information from enterprise systems, call APIs and execute multi-step workflows. These capabilities create real productivity gains, but they also introduce new pathways through which sensitive information can be exposed.

A compromised account, a vulnerable application, a poorly configured AI assistant or a successful prompt-injection attack may provide a route to information that was never intended to leave its original context.

In this environment, the consequences of a security failure can extend beyond the initial intrusion. Once sensitive information has been copied, extracted or processed by an unauthorized system, containing the breach becomes considerably more difficult.

This raises an important question: What happens when an attacker gets past the controls designed to keep them out?

Cyber defence needs to address not only how access is prevented, but also how much damage becomes possible when prevention fails.

2. Beyond the Perimeter: Rethinking Cyber Defence

Traditional cybersecurity controls remain essential. Firewalls, endpoint detection and response, identity and access management, network segmentation, encryption and zero-trust architecture all play important roles in preventing, detecting and containing attacks.

However, controlling access to a system is not always the same as controlling what happens to the data once access has been granted.

Consider an employee who is authorized to access customer records. If their account is compromised, an attacker may inherit some of those permissions. Similarly, an AI application with legitimate access to internal documents might disclose sensitive information because of a malicious instruction, an overly broad permission or a flaw in its design.

The system may not have been breached through a conventional network intrusion. The failure may instead occur through an identity, an application or a legitimate data-processing workflow.

This is where a more data-centric approach to security becomes important.

Rather than relying exclusively on the security of the systems that store and process information, organizations can introduce controls designed to reduce exposure of the information itself.

Encryption protects data at rest and in transit. Privacy-enhancing technologies (PETs) extend this principle to additional scenarios, including computation, analytics and controlled data sharing. Depending on the technology, sensitive information can remain encrypted during computation, be processed within an isolated execution environment, or be transformed so that certain operations can be performed without exposing the original records.

The objective is not to replace perimeter security or zero trust. It is to add another layer of protection when data must be processed, accessed or used beyond its original environment.

In other words, security should not end when access is granted.

3. PETs as a New Layer of Cyber Defence

Privacy-enhancing technologies are a family of techniques that help organizations extract value from data while reducing unnecessary exposure of the underlying information.

They are not a single product, and they do not all solve the same problem. Their effectiveness depends on the threat model, implementation and type of data processing involved.

Several technologies are particularly relevant to modern cyber defence.

Homomorphic encryption: Computing without exposing plaintext

Conventional encryption protects information while it is stored or transmitted, but applications generally need to decrypt data before performing ordinary computation on it.

Homomorphic encryption changes this model by enabling supported computations to be performed directly on encrypted data. The authorized recipient can decrypt the result without the computing party necessarily seeing the underlying plaintext.

For example, a financial institution may want to calculate risk statistics over sensitive customer information without giving an external analytics provider direct access to the underlying records.

With a suitable homomorphic encryption scheme, selected calculations can be performed on ciphertext, reducing the need to expose raw data during processing.

From a security perspective, this can limit the amount of sensitive information available to an analytics service if that service is compromised. It does not, however, eliminate risks associated with compromised keys, information leaked through outputs or weaknesses in the surrounding application.

Homomorphic encryption can also introduce significant computational overhead, so it is most practical when the workload and supported operations are carefully selected.

Secure multiparty computation: Protecting inputs during joint computation

Secure multiparty computation (MPC) allows multiple parties to compute a function over their combined inputs without requiring each participant to reveal its private input to the others.

Imagine two organizations that need to calculate a shared statistic but cannot disclose their respective customer records. An MPC protocol can allow them to obtain the agreed result while keeping their individual inputs protected under the protocol's security assumptions.

This is useful when data cannot simply be centralized in a shared environment.

In cyber defence, MPC can support selected privacy-preserving computations involving sensitive indicators, records or statistics. Its value lies in enabling a computation without requiring every participant to receive the underlying data.

As with other PETs, the security properties depend on the protocol, the participants, the implementation and the information revealed by the final result.

Confidential computing: Protecting data while it is in use

Confidential computing uses hardware-based trusted execution environments (TEEs) to isolate workloads and protect data and code during processing.

This addresses a longstanding gap in conventional encryption: applications often need plaintext data in memory while performing a computation.

A properly configured confidential computing environment can help protect that processing from unauthorized access by other software, privileged infrastructure components or cloud operators, within the guarantees and limitations of the hardware and platform.

For example, an organization could run a sensitive analytics workload inside a confidential virtual machine without making its raw data directly available to the surrounding infrastructure.

Attestation mechanisms can also help verify that a workload is running in an expected environment before sensitive data or decryption keys are released.

Confidential computing is particularly relevant to cloud-based AI and analytics, where organizations want to use scalable infrastructure without granting the entire underlying platform unrestricted access to sensitive information.

It is not a universal guarantee. Vulnerabilities in hardware, side-channel attacks, compromised applications, malicious inputs and weak key-management practices can still undermine security.

Tokenization and privacy-preserving data transformation

Sometimes the most effective approach is to avoid exposing direct identifiers in the first place.

Tokenization replaces sensitive values with alternative identifiers, while techniques such as pseudonymization, private set intersection and oblivious pseudorandom functions can support particular matching or comparison tasks without straightforwardly revealing the original identifiers to every participant.

For example, two organizations may need to determine whether their datasets contain overlapping customer identifiers without exchanging complete customer lists.

Depending on the protocol, the organizations can identify matches while limiting what each party learns about non-matching records.

However, tokenization is not automatically anonymization. If tokens can be linked back to individuals, or if additional information makes re-identification possible, the data may remain sensitive. Stable tokens, weak keys and poorly designed matching protocols can also introduce security risks.

The key principle is to select a technology based on the specific information that must be protected and the information that each participant is permitted to learn.

Data-centric encryption: Keeping control attached to the data

Encryption can also be combined with access policies designed to govern who may decrypt information and under what conditions.

For example, attribute-based encryption can associate decryption rights with defined attributes or policies. This can help restrict access to an encrypted document even after it has been copied or moved beyond its original storage environment.

Such controls can complement identity-based security by bringing enforcement closer to the data itself.

Their effectiveness still depends on sound policy design, reliable key management, revocation mechanisms and the assumptions of the underlying system. Encryption cannot prevent every form of misuse by a person or application that is legitimately authorized to decrypt and view the information.

Taken together, these technologies offer a broader security model: reduce unnecessary exposure, protect sensitive computation and enforce appropriate access to data wherever feasible.

4. Making PETs Practical in an AI-Driven Threat Landscape

The potential of PETs is significant, but organizations should avoid treating them as a universal solution to cybersecurity.

Each technology addresses a different part of the problem. The practical challenge is to identify where sensitive data is exposed, what an attacker could gain from that exposure and which controls can meaningfully reduce the risk.

Consider an organization deploying an AI assistant to support fraud investigations. The assistant may need to analyze financial records, transaction histories and case documents. Simply placing the model behind an authenticated API does not guarantee that sensitive information will remain protected throughout the workflow.

A stronger architecture could combine several layers:

  • Least-privilege access: Give the assistant access only to the records and operations required for its task.

  • Data minimization: Remove unnecessary identifiers and sensitive fields before information reaches the model.

  • Confidential computing: Where appropriate, run sensitive processing inside an attested, isolated environment.

  • Encryption and key controls: Protect stored and transmitted information, restrict access to decryption keys and separate key management from application administration where practical.

  • Output controls: Inspect model responses and generated reports for sensitive information that should not be disclosed.

  • Monitoring and containment: Log access and relevant actions, detect anomalous behaviour, and maintain the ability to revoke credentials or stop compromised workflows.

Other PETs may be appropriate for different requirements. Homomorphic encryption or MPC could protect particular analytical operations, while tokenization may reduce the amount of directly identifying information an application handles.

The goal is not to use every available technology. It is to design a system in which sensitive information is exposed only when necessary, to the minimum extent required, under clearly defined security assumptions.

Organizations should also assess performance, integration complexity, operational costs and recovery procedures before deploying PETs in production. A technically sophisticated control that cannot be maintained or correctly configured may create a false sense of security rather than improve resilience.

5. Building Resilience for the AI Era

The growth of AI does not make established cybersecurity practices obsolete. It makes their correct implementation more important while exposing the limitations of relying on any single defensive layer.

Organizations still need strong identity controls, secure software development, endpoint protection, vulnerability management, incident response and continuous monitoring. They also need governance over how employees, applications and AI agents access and use sensitive information.

PETs add another dimension to this approach by changing how data can be processed, shared and protected.

For security leaders, several questions are worth asking:

  1. Where does sensitive data become exposed? Map not only where information is stored, but also where it is decrypted, processed, copied and passed to AI systems.

  2. What happens if an identity or workload is compromised? Evaluate the potential damage if an attacker gains legitimate application permissions or access to an internal processing environment.

  3. Can the required task be performed with less exposure? Consider whether the use case can rely on tokenized data, encrypted computation, secure multiparty computation or confidential computing.

  4. Can access remain constrained after data moves? Examine whether encryption and access policies can continue to restrict who may decrypt and use sensitive information.

  5. How will the controls be validated? Test the complete workflow, including key management, output leakage, monitoring, recovery and failure scenarios.

These questions help move cybersecurity beyond a narrow focus on blocking access towards a broader goal: limiting the consequences of unauthorized access and reducing unnecessary exposure throughout the data lifecycle.

No technology can guarantee that every breach will be prevented or that every compromised system will be harmless. PETs themselves introduce implementation and operational challenges, and they cannot compensate for poor security practices.

Their promise is more specific, but important: they can reduce the amount of sensitive information that must be exposed for legitimate work to happen.

Conclusion: Protect What Matters, Even When Defences Fail

As AI makes it easier to automate both legitimate work and malicious activity, organizations need to reconsider how they protect the information that powers their operations.

A perimeter can be bypassed. An identity can be compromised. An application can behave in unexpected ways. And once sensitive information is exposed in plaintext, recovering control over it can be difficult or impossible.

Privacy-enhancing technologies offer ways to reduce this exposure by protecting computation, limiting the disclosure of inputs and controlling access to sensitive information. Used alongside zero trust, encryption, monitoring and sound operational security, they can strengthen the resilience of systems that handle valuable data.

The future of cyber defence will not be determined by a single technology. It will depend on how effectively organizations combine prevention, detection, containment and data protection.

The question is no longer only how to keep attackers out. It is also how to ensure that, when a control fails, the data remains as protected as possible.

1 view

Want to talk through how this applies to your data infrastructure?

Talk to the team