It’s hard to ignore the buzz around AI these days. From automating processes to enabling hyper-personalized user experiences, AI promises to reshape industries at lightning speed. But let’s be honest—AI adoption comes with its share of cybersecurity challenges. The more we rely on complex AI systems and MLOps pipelines, the bigger the risks get.
In November 2024, I sat down with Srinivas Tummalapenta, CTO for IBM Security Services, to unpack these challenges. Srinivas offered a thoughtful perspective on how organizations can secure their AI ecosystems, particularly focusing on MLSecOps (Machine Learning Security Operations). It’s a holistic strategy that extends the familiar DevSecOps principles into the AI world—and it couldn’t come at a more critical time.
Why Securing AI is Different
Securing AI systems isn’t as straightforward as patching vulnerabilities in your typical development cycles. With AI, the attack surface expands to include data pipelines, model integrity, and the unpredictable nature of machine learning behavior. In Srinivas’ words, securing AI requires us to go beyond traditional software development lifecycles and think about additional layers—what IBM refers to as DataSecOps and ModelSecOps.
Let’s break this down. When you train an AI model, the data itself becomes a security asset. A poisoned dataset—one that contains malicious inputs, sensitive information, or toxic content like hate speech—can completely derail the AI’s performance and safety. Similarly, the models you use (especially open-source models) need to be scanned for hygiene and reliability before being deployed.
Srinivas summed it up best:
"You now need to scan and verify the data pipeline for sensitive or harmful inputs. At the same time, you bring in new models into your pipeline, clean them, ensure their hygiene, and then integrate them with your verified training dataset."
This isn’t a one-and-done process either—it’s continuous. Both the data and the models evolve over time, so their security must evolve too.
Introducing MLSecOps: The AI Security Pipeline
One of the highlights of our conversation was Srinivas’ breakdown of the MLSecOps pipeline. If you’re familiar with DevSecOps, this builds on the same principles but adapts them to AI workflows. Here’s what it looks like in practice:
- DataSecOps: This focuses on securing the data pipeline. Before you even train a model, you need to cleanse the data to ensure it doesn’t contain sensitive, proprietary, or toxic information. That means scanning for things like personal data, hate speech, trademarks, or abuse that could taint the training process.
- ModelSecOps: Just like code repositories, models—including open-source large language models (LLMs)—need hygiene checks. You must scan these models to ensure they’re free from vulnerabilities or unexpected behaviors before they enter production.
- Integration with DevSecOps: Once the verified models and datasets are ready, they connect back into your traditional DevSecOps pipelines. APIs tie everything together, enabling AI-driven applications to run securely.
The end goal is a secure and auditable AI lifecycle that keeps your applications safe while giving you complete visibility into the model’s evolution.
Why Continuous Monitoring Matters
If there’s one thing I took away from this discussion, it’s that security for AI systems is never static. Both data and models are constantly changing. AI systems learn and adapt, which makes them more powerful—but also harder to secure.
For example:
- Training datasets get updated with new inputs.
- Models are retrained or fine-tuned for different use cases.
- Applications built on these models evolve over time.
According to Srinivas, organizations must implement continuous inspection and hygiene checks for both data and models. This isn’t just about catching bad actors who might be trying to inject poisoned inputs; it’s also about maintaining accountability.
He explained:
"You need to track the lineage of both data and models as part of your governance process. If an audit happens or something fails, can you backtrack and prove what processes were followed? Can you explain how the model behaved on a specific date?"
This level of traceability and auditability is essential. It’s not just about proving compliance—it’s about understanding how your AI systems evolve and ensuring they remain reliable over time.
Emerging Risks: Open-Source Models and AI Supply Chains
One of the trickiest challenges in securing AI pipelines is dealing with open-source models. These models have become incredibly popular, from sources like Hugging Face (which now hosts about 1.2 million models), especially in industries racing to deploy AI solutions quickly. But Srinivas raised a valid point: when you use open-source models, you’re inheriting risks.
How can you be sure the model you’re using is free from vulnerabilities or biases? How do you verify its integrity? Without rigorous scanning and hygiene processes, open-source models can become a ticking time bomb for organizations.
This extends to the broader AI supply chain as well. Just like software supply chains have become prime targets for cyberattacks, the same will hold true for AI pipelines. Attackers will look for weak links—whether that’s in the data, the models, or the applications themselves. A robust MLSecOps process helps mitigate these risks by ensuring every component is verified and secure.
The Future of AI Security
As AI adoption accelerates, organizations can’t afford to treat security as an afterthought. Securing AI systems requires a paradigm shift—one that integrates DataSecOps and ModelSecOps into your overall DevSecOps strategy.
Srinivas’ insights highlight the need for a proactive, lifecycle-based approach to AI security. By continuously monitoring data, scanning models, and maintaining traceability, organizations can build trust and resilience into their AI ecosystems.
At AppSOC, we’re deeply aligned with this mission and provide solutions across the entire AI and application lifecycles to ensure that AI systems remain a force for good—not a source of risk.
Final Thoughts
AI security is no longer a niche concern. It’s a fundamental part of how we build, deploy, and govern AI systems in today’s rapidly changing digital landscape. Conversations like this one with Srinivas remind me of how critical it is to stay ahead of emerging threats—and to approach AI adoption with security baked in at every step.
The stakes are high, but with the right frameworks in place—like MLSecOps—we can secure our path to AI adoption.
Watch Blog Video
