Fever pitch
AI and LLM adoption have hit the top of investment priorities for most large enterprises. About 83% see the adoption and acceleration of AI/ML use cases as a top priority over the next 24-36 months to drive business value. Most don’t want to create custom LLMs but instead want to use off-shelf models they can call out to via an API. Most enterprise use cases will likely involve using more than one foundational model to deliver impactful business outcomes. A smaller percentage are looking to use fine-tuning techniques and custom data sets to train these models. Security and protecting the proprietary nature of these datasets are top-of-mind for them.
Why AI will require Multi-Cloud Infrastructure to be successful
Most of the prevailing discussion has centered around the infrastructure required to run AI workloads – GPUs and high-speed networking within these clusters. ALl the major CSPs have world-class infrastructure and command enough scale to deliver this infrastructure at efficient market pricing. However as we click deeper into the enterprise grade use cases that will be enabled by these GenAI models (LLMs and multi-modals) multi-cloud infrastructure becomes table stakes for the following reasons:
- Distributed LLM Stack footprint – Data stores in S3, Vector embeddings in Pinecone, API calls and or data ingestion to Open AI, embedding models from huggingface, validation services – Ubiquitous connectivity to these diverse resources needs a robust hybrid multi-cloud infrastructure.
- Flexibility – Enterprises need to be able to leverage best in breed LLM services from the different CSPs. Diverse business use cases might need different levels of fine-tuning or performance from an AI service. Since CSPs vary in these capabilities MCN infrastructure is a key component in enabling this flexibility for enterprises.
- Prevent vendor lock in and cost arbitrage – CSPs charge differently for AI/ML workloads (training versus inference) and enterprises need to take advantage of these cost differentials. As GPU supply catches up to demand, enterprises need to be able to take advantage of future price reductions.
- Multi-cloud reliability for business continuity in case of regional failures.
- Day N operations with deep application level visibility to solve AppSecOps, NetDevOps and Finops use cases.
Fig 1: Sample Enterprise AI services
But there are hurdles …. It’s a new connectivity and threat landscape.
Customers are concerned with several security and compliance issues that must be addressed before the wide-scale adoption of LLMs for enterprise-grade use cases.
Fig 2: Rendering of a pipeline that is answering a user query with contexts from certain documents:
Even in this simple rendering, multiple PaaS services, data sources, users, LLM APIs need to be connected in a zero-trust, zero-data leak, highly available architecture. Traditional L3MCN architectures are inadequate and will likely hinder adoption. To accelerate AI adoption, enterprises need an app-aware, multi-cloud networking architecture that addresses all the connectivity, security and observability challenges mentioned above.
Key concerns with operating a multi-cloud AI infrastructure include:-
- Secure Connectivity – End-to-end Private connectivity to enterprise data stores and workloads. Creating a multi- region, multi-cloud private connectivity infrastructure is complex to create and manage. Prosimo’s full stack transit enables this end-to-end private connectivity use -case.
- Responding to a new class of security threats– Injection attacks, Model/data set poisoning, embeddings, prompt leaks, etc. will require a multi-cloud infrastructure that can observe and secure not only network interfaces but application interfaces (such as APIs and prompts )
- Uniform, multi-cloud policy semantics to drive privacy and compliance mandates to prevent non-compliance with regulatory or corporate data provenance guidelines.
- Enforce guardrails to check for offensive prompts, responses, or data leaks.
To address these concerns holistically, any cloud networking solution must meet the following requirements:
Fig 3 : Key requirements for multi-cloud network interconnecting GenAI workloads
Prosimo’s AI suite for MCN
Prosimo’s AI Suite for Multi-Cloud networking solves all these challenges holistically and provides foundational capabilities that enable flexibility to adapt to evolving AI workloads.
Fig 4 : Capabilities of Prosimo Multi-Cloud Networking for AI
- Secure private connectivity for all your cloud workloads
- Seamless app-layer connectivity and segmentation for your workloads, PaaS services while solving for native CSP limitations.
- App-aware traffic routing Policy based steering to different foundational models based on query type /source etc.
- Steer responses or prompts for deeper inspection by guardrails.
- Elastic infrastructure for real-time query performance.
- Rich policy-based app-layer segmentation.
- Honor compliance and regulatory mandates with ease.
Announcing our LLM AIOps assistant Nebula
The LLM is as good as the data it’s trained on. L3 only observability data will limit the LLM to insights based on that gross metric. Either that or use a pipeline to correlate with an external tool. Avoiding this needless instrumentation is one of the key benefits of a full stack app aware network- because of its deep visibility. Every session, every API call, every access is logged in the system and can be viewed through the Prosimo console serving as a multi-cloud single pane of glass. However, customers have been limited to the set of data visualization and filters that the Prosimo Console natively builds to gain insights from this data. Alternatively, they have had to rely on exporting the data set to external tools to analyze further.
Today we are proud to announce Nebula, our LLM based AIOps assistant. Customers can now query this rich operational dataset to instantly answer queries about cost chargebacks, performance anomalies, and how traffic is flowing between networks or applications. Imagine answering a GDPR compliance query to check if data from an S3 bucket in Germany ever crossed a regional boundary, with a simple NLP interface, in a matter of seconds. Or figure out how many overlapping IP subnets exist in your multi-cloud footprint, which targets have received the most traffic, or which workloads have accessed the internet. Customers in early beta have reported significant improvements in MTTR with Nebula.
The deep visibility that is inherent with the Prosimo full stack is key to powering Nebula, which is then able to use the power of LLMs to provide instantaneous answers and insights into finops ,secops or netops queries.
Fig 5 : Sample Insight from Prosimo dashboard showing GenAI network transactions
Prosimo Full Stack continues to Innovote
Prosimo full stack multi-cloud networking is the only solution that comprehensively addresses the complex connectivity and security needs of GenAI workloads. Prosimo AI Suite for multi-cloud networking builds on foundational app-aware capabilities such as Cross-cloud connect, PaaS connect, Dynamic Fqdn resolution, Service core and Adaptive firewall insertion. The Prosimo architecture is a robust framework that can accelerate the enterprise adoption of GenAI use cases while adhering to enterprise-grade security and compliance mandates. The full stack architecture allows us to incrementally add security and connectivity capabilities that save customers time and represent significant cost savings. Insights and Nebula will continue to answer complex observability questions about the multi-cloud infrastructure and enrich those observations by making recommendations that help you make intelligent decisions about your infrastructure.
Included at no additional cost to our customers, Prosimo AI Suite for multi-cloud networking is a future proof architecture that accelerates the adoption of AI use cases in the enterprise while significantly improving Day 2 operational benchmarks by offering Nebula- the industry’s only LLM based AIOps assistant.
Fig 6 : AI suite extends Prosimo’s Full stack MCN architecture
