The missing piece in image scanning

Introduction

In the dynamic landscape of cloud-native cybersecurity, image scanning has become essential to ensuring the safety and integrity of cloud workloads and digital assets. Historically, image scanners focus on finding vulnerabilities (CVEs) that may be the cause of exploits in Kubernetes workloads. However, there’s a significant gap that often goes unnoticed. This gap is the lack of comprehensive scanning for malware, viruses, crypto miners, and other malicious threats. These types of threats can be caught if just look for them.

In this blog post, we explore the missing piece in most scanners today, and the urgent need to go deeper than the image scanning that is available today.

Image scanning – the beginning, not the end

The rise of cloud-native workloads has intensified the need for security, which directly correlates to the rise in SBOM and vulnerability scanning. While vital for identifying weaknesses in software components, these scans overlook a critical threat vector: evolving attackers embedding malware, viruses, ransomware and even crypto miners directly into seemingly innocent images.

Unfortunately, this growing threat goes largely unnoticed. Most image scanning tools, focused on traditional issues, lack the capacity to effectively detect and neutralize malicious code. This leaves organizations vulnerable and unaware that a seemingly harmless image could leave you open to a devastating attack.

When mentioning image scanning we would be remiss not to mention image signing.

Ensuring your images are signed

Container image signing is the process of adding a signature to a container image. This signature ensures the authenticity and integrity of the image. The purpose of signing is to verify that the image has not been tampered with or modified in any way since it was originally signed. By checking the signature, we can establish trust in the image’s origin and content.

Container image signing has gained traction due to security concerns. Tools like Cosign, Notary, and Docker Content Trust facilitate image signing. Organizations are increasingly adopting these practices to enhance supply chain security. End-user organizations are or should be getting on board with this and be checking image signatures to ensure provenance and security.

If you’ve come this far and are nodding your head, thinking “I’ve got this”, we’ll be the security person that says “yes, and there still may be a gap in your security”

Bridging the gap

While an image may be free of CVEs, it could still harbor malware. Signature-based scans (distinct from SBOM signatures), systematically search for the characteristic patterns of malware within a database of identified malicious programs. These scans are particularly crucial if image signing is not enabled within your cluster.

When looking at this from the KDR (Kubernetes Detection and Response) perspective it is still not a silver bullet. When a runtime threat detection tool looks for threats, it gathers as much information as it can in order to make a clear determination, whether there is a threat or not. 

For instance, even when a malicious behavior pattern is identified, one of the factors considered is whether the flagged binary is part of the base image or not. If the binary is part of the base image, the tool treats the threat as less severe and may even classify it as a false positive, attributing the initially flagged behavior to normal application activity. In contrast, if the binary has deviated from the base image, it immediately raises high-priority alerts.

Signature scanning capabilities

To get complete coverage, we recommend signature scanning, like the ones anti-virus does on your home PC. While this is very 90s, it is still applicable today. 

In the example below we leveraged another open-source project – ClamAV – in a PoC that adds support for node scanning. ClamAV is an open-source antivirus engine known for detecting and neutralizing a diverse range of malware, viruses, and cyber threats. It utilizes both signature-based and heuristic methods. 

We have created a custom signature database that is unique for Kubernetes and wanted to see if we can effectively detect malwares inside workloads. 

Here’s an example of how we use ClamAV to detect malware. In our proof of concept we added Kubernetes context to the results, based on the eBPF datastream, to easily enable further analysis of the threat.

In the result, you can see that we detected a virus. We can see the signature of the virus, in addition to enriched Kubernetes-specific data that guides through research and elimination of the threat.

We utilized ClamAV capabilities of supporting a custom signature database and created one that suits the needs of cloud-native environments.

Conclusion

While “shift left” security practices, like image scanning, provide substantial value. However, practitioners must be careful not to overlook threats that slip through the cracks. Comprehensive image analysis extending beyond just CVEs to also cover malware, viruses, and other malicious code is critical for robust cloud-native security postures. Thus ensuring speed and agility without sacrificing protection.

Check out the ARMO’s KDR Github repo for more information about KDR.