There’s no doubt that generative AI (genAI) and large language models (LLMs) are disruptive forces that will continue to transform our industry and economy in profound ways. But there’s also something very familiar about the path organizations are taking to tap into gen AI capabilities.It’s the same journey that happens any time there’s a need for data that serves a very specific and narrow purpose. We’ve seen it with search where bolt-on full-text search engines have proliferated, resulting in search-specific domains and expertise required to deploy and maintain. We’ve also seen it with time-series data where the need to deliver real-time experiences while solving for intermittent connectivity has resulted in a proliferation of edge-specific solutions for handling time-stamped data.And now we’re seeing it with gen AI and LLMs, where niche solutions are emerging for handling the volume and velocity of all the new data that organizations are creating. The challenge for IT decision-makers is finding a way to capitalize on innovative new ways of using and working with data while minimizing the extra expertise, storage, and computing resources required for deploying and maintaining purpose-built solutions.Purpose-built cost and complexityThe process of onboarding search databases illustrates the downstream effects that adding a purpose-built database has on developers. In order to leverage advanced search features like fuzzy search and synonyms, organizations will typically onboard a search-specific solution such as Solr, Elasticsearch, Algolia, and OpenSearch. A dedicated search database is yet another system that requires IT resources to deploy, manage, and maintain. Niche or purpose-built solutions like these often require technology veterans who can expertly deploy and optimize them. More often than not, it’s the responsibility of one person or a small team to figure out how to stand up, configure, and optimize the new search environment.Time-series data is another example. The effort it takes to write sync code that resolves conflicts between the mobile device and the back end eats up substantial developer time. On top of that, the work is non-differentiating since users expect to see up-to-date information and not lose data as a result of poorly written conflict-resolution code. So developers are spending precious time on work that is not strategically important to the business, nor does it differentiate their product or service from the competition.The arrival and proliferation of gen AI and LLMs is likely to accelerate new IT investments in order to capitalize on this powerful, game-changing technology. Many of these investments will take the form of dedicated technology resources and developer talent to operationalize. But the last thing tech buyers and developers need is another niche solution that pulls resources away from other strategically important initiatives.Documents to the rescueLeveraging genAI and LLMs to gain new insights, create new user experiences, and drive new sources of revenue can entail something other than additional architectural sprawl and complexity. Drawing on the flexible document data model, developers can store vector embeddings — numerical representations of data that power AI solutions — alongside operational data, which allows them to move swiftly and take advantage of fast-paced breakthroughs in gen AI without having to learn new tools or proprietary services.Documents are the perfect vehicle for genAI feature development because they provide an intuitive and easy-to-understand mapping of data into code objects. Plus, the flexibility they provide enables developers to adapt to ever-changing application requirements, whether it’s the addition of new types of data or the implementation of new features. The huge diversity of your typical application data and even vector embeddings of thousands of dimensions can all be handled with documents.Leveraging a unified platform approach — where text search, vector search, stream processing, and CRUD operations are fully integrated and accessible through a single API — eliminates the hassle of context-switching between different query languages and drivers while keeping your tech stack agile and streamlined.Making the most out of genAIAI-driven innovation is pushing the envelope of what is possible in terms of the user experience — but to find real transformative business value, it must be seamlessly integrated as part of a comprehensive, feature-rich application that moves the needle for companies in meaningful ways.MongoDB Atlas takes the complexity out of AI-driven projects. The Atlas developer data platform streamlines the process of bringing new AI-powered experiences to market quickly and cost-effectively.To find out more about how Atlas helps organizations integrate and operationalize genAI and LLM data, download our white paper, Embedding Generative AI and Advanced Search into your Apps with MongoDB.
Copyright © 2024 IDG Communications, Inc.
Source link