Realtime AI video analysis app Lloyd will offer a development kit

Disclaimer: EndlessAI previously published a Contribution piece on VentureBeat Lloyd’s launch announced in early October.

A four-year-old AI startup EndlessAI Not a household name – yet.

But its founders and leaders believe they have a powerful stroke on their hands: their own freemium Lloyd’s iOS appwhich uses proprietary video streaming and encryption technology to feed the user’s live video feed to underlying AI models including OpenAI’s GPT-4o to help with a wide range of tasks from fixing bikes to telling bedtime stories, has achieved over 50,000 users. Three months after the hidden launch.

41% of those users interact with the app daily, according to data provided to VentureBeat by EndlessAI.

Although there is no ChatGPT – which has become The fastest product in history to cross the 100 million user mark in January 2023just two months after launch — still, it was encouraging enough for EndlessAI CEO Roy Genette and CEO Thomas Pompidou, who told VentureBeat in a recent video call interview that they plan to open their platform to third-party developers in early 2025. and launching its consumer-facing Android app platform in January.

Also, EndlessAI has already begun upgrading Lloyd with what it calls “powers,” or as Pompidou describes it, “large-scale, fundamentally fine-tuned language models (LLMs) that provide deep consumer insight into a specific use case.”

For example, the first Lloyd Power program now live in the app is “Chef,” which gives you a fully AI-powered coach in real-time who watches you while you cook (if you point your smartphone camera at the stovetop or cooking zone) and provides you step-by-step guidance .

Another Lloyd Power company set to launch soon is Tour Guide, which allows users to hold up their phone and see real-time contextual information about their surroundings. By taking a video of a location, it identifies points of interest, provides relevant details, and can also recommend nearby attractions or activities.

Make real-time video analysis widely available

While existing LLMs struggle to process live video efficiently due to high computational costs. EndlessAI overcomes this limitation, reducing the cost of video analysis by more than 99%.

Pompidou emphasized the broader mission of the app: “Our mission is to extend AI to the real world. The real world is visual and live, and existing large language models, as designed, face challenges in analyzing video accurately, at scale, and affordably. This is what we make possible.”

This enables real-time video analysis, allowing users to interact with their environment in new ways, from diagnosing mechanical problems to creating personalized bedtime stories.

Lloyd’s primary differentiation lies in its ability to process video data through LLMs at a fraction of the cost typically associated with such tasks. Traditional LLM architectures are not optimized for video, making real-time video analysis expensive and slow.

“Video analysis using ChatGPT, assuming it can be done, would cost more than $300 per hour,” Pompidou said. “With Lloyd, we deliver the same level of accuracy for just 10 cents per hour.”

This cost-effectiveness is achieved without sacrificing accuracy, setting Lloyd apart from competitors who rely on lower frame rates or lower resolutions to cut costs, often at the expense of reliability.

“Our communication layer is powerful in ways that other solutions are not. It allows developers to integrate real-time AI services such as speech-to-text, text-to-speech, and video analysis with unparalleled reliability and performance.

As he envisioned the future, Pompidou offered a glimpse of the app’s potential: “Imagine an LLM who has been meticulously trained on every IKEA instruction manual, guiding customers step-by-step using video and identifying errors in real time. It’s just one example of how our technology can change experiences.” user.

Another big arena that EndlessAI plans to appeal to with Lloyd and its core video encoding technology: law enforcement, specifically providing analysis of police body camera footage.

“If someone is having a heart attack, they will recognize it and provide the officer with instructions on what to do immediately,” Pompidou said.

Privacy and security

Although Lloyd itself sees exactly what you point your smartphone camera at, EndlessAI prioritizes user privacy.

“The data remains private to (users’) accounts, and we can only access it for support if users explicitly request help,” Genat said.

This approach ensures strong guarantees while enabling seamless interactions.

But as a result, EndlessAI isn’t entirely sure what the most popular uses for Lloyd are among its users. Anecdotally, she says her surveys and feedback forms showed an interest in food preparation, home repairs, fashion and lifestyle coaching, and more.

While Lloyd’s consumer-facing features are gaining traction, EndlessAI is also building tools to enable developers and enterprises to harness its technology.

“Our long-term roadmap includes a software development kit (SDK) for developers, starting early next year,” Pompidou said. “It will enable them to create unique visual AI solutions very simply.”

The SDK will allow developers to integrate AI vision capabilities into their own applications.

“The first offering for developers will be a powerful platform for real-time API calling, connecting to OpenAI and other backends,” Genat told VentureBeat. “Developers can pick and choose which components they want to use, such as voice services or speech-to-text.”

Applications for these tools span industries, from creating AI-powered chat apps to integrating video analysis into production lines and safety monitoring systems.

EndlessAI aims to provide scalable solutions that adapt to different performance and cost requirements.

“Our developer tools will allow for quick adjustments – choosing between back-end services or lightweight solutions available on the device depending on the use case and cost requirements,” added Genat.

By combining powerful APIs and intuitive SDKs, EndlessAI envisions a new wave of AI-driven applications that go beyond traditional text or image processing. “We will offer developers the ability to integrate various services, including video side processing, and enhance their sessions with additional capabilities,” Genat said.

Transforming AI for consumers and enterprises

Lloyd’s ability to take advantage of existing smartphones – without the need for additional hardware – makes it uniquely accessible.

By lowering barriers to entry, EndlessAI is redefining what is possible with AI in everyday life and niche industries alike.

With rapid user adoption, versatile applications, and a robust roadmap, Lloyd is poised to become a defining innovation in AI.

“Our long-term strategy is to remain complementary to LLM holders,” Pompidou said. “Even when models can process video natively, we aim to remain the efficiency layer that makes these applications viable and cost-effective.”