Empowering AI Agents with Real-Time Data
Building Better AI Agents Through Real-Time Data Access
In my previous post, I explored how the Generative AI era is upending traditional data processing paradigms—shifting us from the familiar terrain of SQL-centric systems into AI-centric tools, where unstructured, multimodal data reigns supreme. While the buzz surrounds unstructured data formats like sales calls, PDFs, and videos, we mustn't forget that structured operational data remains essential to enterprise intelligence. This might seem like a step backward from the cutting-edge world of multimodal AI, but I believe it represents one of the most pressing challenges facing teams building agents and other AI applications.
This disconnect is more than a technical hiccup; it's a strategic blind spot. AI applications are hamstrung by an operational data gap, frustratingly disconnected from business-critical data stored in siloed systems like CRMs and transactional databases. Traditional ETL pipelines and batch processes ensure that any data they receive is stale, unsuitable for dynamic AI applications. The effort required to integrate AI with real-time data sources often produces brittle solutions that demand significant engineering resources.
Addressing this chasm is imperative and would unlock many applications: AI chatbots could instantly access live order statuses and customer histories, slashing customer wait times. Real-time analytics tools could integrate live data across sales and supply chains, enhancing business agility. Autonomous AI agents could make informed decisions based on up-to-the-minute operational context. For businesses building AI applications, bridging this gap isn't just about technical elegance—it's about enabling the kind of responsive, context-aware AI systems that customers increasingly expect.
Data Integration Toolbox: Trading Elegance for Expedience
The tools available to AI teams aiming to bridge the operational data gap range from sophisticated to makeshift. Retrieval-Augmented Generation (RAG), prominent in AI integration, offers an effective solution for unstructured data but falters with the complex relationships inherent in operational databases. While RAG excels at processing documents and general knowledge, it often reduces rich, structured data to flat embeddings, losing the relationships that make relational databases powerful.
Traditional methods offer limited improvement. ETL pipelines and data warehouses remain primary means to connect operational systems with AI applications. Yet their batch-processing nature and inherent latency are increasingly inadequate in an era that demands real-time responsiveness. Function calling and AI agents present a more modern approach, providing purpose-built integrations for specific use cases. However, these solutions can be costly: they deliver impressive results for narrow applications but require significant engineering resources and lead to maintenance challenges that grow with each new integration.
The limitations of current solutions highlight a broader issue: balancing scalability and fidelity. Custom pipelines and ad hoc connectors can offer precise, real-time access to operational data but are difficult to scale across multiple data sources. Fine-tuned LLMs provide another option but come with their own trade-offs: high computational costs, the need for fine-tuning data, and the difficulty of keeping models in sync with rapidly changing operational data. In my experience, many teams end up combining these approaches, resulting in complex architectures that function but feel more like temporary fixes than permanent solutions.
Snow Leopard: Bridging the AI-Operational Data Divide
Snow Leopard offers a practical solution to a problem that has hindered many AI implementations: accessing live operational data without overhauling existing infrastructures. Rather than requiring enterprises to redesign their data architectures, Snow Leopard introduces an intelligent layer that communicates directly with existing systems using their native protocols. This approach enables AI applications to access up-to-date data without the latency and complexity introduced by traditional ETL processes.
What sets Snow Leopard apart is its commitment to connecting AI systems with data where it already resides. By employing intelligent query routing and native integrations with systems ranging from SQL databases to REST APIs, it eliminates the need for custom connectors for each data source. This not only simplifies the integration process but also fundamentally changes how AI applications interact with live data, making real-time insights more accessible. Importantly, Snow Leopard is designed to work alongside existing ETL processes when batch processing or data warehousing remains appropriate, providing a complementary solution rather than a complete overhaul.
The platform also addresses critical concerns around governance and security. Operating within customer VPCs and incorporating built-in controls, Snow Leopard meets enterprise requirements for data privacy and compliance—issues that can derail AI initiatives if not properly managed. As an early-stage product, note that Snow Leopard has limited production deployments, with most implementations still in the proof-of-concept phase. Additionally, its focus on structured data means it offers limited support for unstructured data types like images or audio.
Despite these constraints, early adoption by fintech and SaaS companies suggests that Snow Leopard is addressing a significant need. By maintaining data fidelity and eliminating transformation steps, it tackles a major pain point in AI development. Perhaps most compelling is its scalability: the platform promises to support multiple use cases across various data sources without extensive customization. For teams burdened by the maintenance of custom data pipelines, this could represent a meaningful shift towards more efficient and responsive AI applications.
The Road Ahead: Connecting AI with Enterprise Reality
In a recent conversation with Snow Leopard's CEO, Deepti Srivastava, she outlined the company's plans to deepen the platform's capabilities. High on the agenda is expanding their connector library to include integrations with a broader array of data sources, driven by customer demand. They plan to introduce advanced query features like cross-source joins and aggregations, enabling more complex data interactions without sacrificing performance. Recognizing the critical importance of governance, Snow Leopard is building robust frameworks for policy enforcement and compliance, ensuring that data flows securely and within regulatory bounds. And in a nod to the growing significance of unstructured data, they will eventually add capabilities to handle formats like text, images, and audio.
While Snow Leopard tackles the real-time operational data challenge, other tools are emerging that could complement its approach. One such tool is BAML, an open-source domain-specific language designed for structured text generation with large language models. By treating prompts as first-class functions with defined inputs and outputs, BAML brings much-needed rigor and efficiency to AI development workflows.
Similarly, Anthropic's Model Context Protocol (MCP) is an open standard for connecting AI assistants to external data sources and systems where data lives, including content repositories, business tools, and development environments. MCP enables AI systems to interact with various data sources while maintaining security through features like per-chat permissions. The protocol provides three main components: the specification and SDKs, local MCP server support in Claude Desktop apps, and an open-source repository of MCP servers. Early implementations include pre-built MCP servers for systems like Google Drive, Slack, GitHub, Git, and Postgres. Companies like Block and Apollo have integrated MCP, while development tools companies including Zed, Replit, Codeium, and Sourcegraph are working with MCP to enhance their platforms. Rather than maintaining separate connectors for each data source, developers can build against this standard protocol, allowing AI systems to maintain context as they move between different tools and datasets.
For teams building AI applications, these emerging tools represent essential infrastructure for creating AI that can understand and respond to business realities in real time. The future of AI agents lies not just in their reasoning capabilities but in their ability to seamlessly integrate with the systems where business happens. Looking ahead to 2025, as tools for accessing operational data like Snow Leopard mature, we can expect to see a dramatic evolution in agent AI capabilities. Teams building RAG applications will increasingly shift toward developing AI systems that can access and leverage the right operational data in real time, leading to more sophisticated and business-aware AI applications that can truly deliver on the promise of autonomous decision-making.
Data Exchange Podcast
The Essential Guide to AI Guardrails. This episode features Shreya Rajpal, CEO and co-founder of Guardrails AI, discussing the critical role of AI guardrails in ensuring safe and reliable AI applications. We explore the technical architecture of the Guardrails framework, its real-world applications in healthcare and content moderation, and practical insights for developers tackling implementation and performance challenges.
Beyond ETL: How Snow Leopard Connects AI, Agents, and Live Data. Deepti Srivastava, CEO of Snow Leopard, shares how their groundbreaking live data access model transforms AI integration by bypassing traditional ETL pipelines, enabling real-time decision-making and scalability in high-stakes, regulated industries.
If you enjoyed this newsletter please support our work by encouraging your friends and colleagues to subscribe:
Ben Lorica edits the Gradient Flow newsletter. He helps organize the AI Conference, the NLP Summit, Ray Summit, and the Data+AI Summit. He is the host of the Data Exchange podcast. You can follow him on Linkedin, Mastodon, Reddit, Bluesky, YouTube, or TikTok. This newsletter is produced by Gradient Flow.
We have seen exactly the same while building Tilores IdentityRAG. First people try to build AI applications based on datawarehouse data, but they struggle with 1) real-time data and 2) connecting data together where there is no unique identifier (identity resolution problem).
Since we already had a highly-scalable, real-time entity resolution engine, it was quite simple for us to extend that to be a GenAI data source (in this case by building a LangChain integration) and hence IdentityRAG was born.
We are finding being connected to the datawarehouse whilst being a mutual datasource for live data streams is working well - i.e. the real-time data is sent to both Tilores and the dwh at the same time but we also stay in sync with the dwh directly.