Feature
The Gist
- Apple's AI evolution. The generative AI models like Propensity, Midjourney, Claude, Gemini, and ChatGPT influence customer experience AI, with Apple's Ferret model poised to join the fray.
- Ferret's unique approach. Apple's Ferret model offers spatial referencing, allowing for superior accuracy in identifying elements within images, a novel feature compared to existing AI models.
- Commercial implications. While Ferret is currently for research, its potential integration into Apple products could enhance customer experience AI and influence the competitive landscape in consumer AI.
The current crop of generative AI models, including Propensity, Midjourney, Claude, Gemini and ChatGPT, are familiar to tech enthusiasts and the public alike, influencing the landscape of customer experience and AI.
One model that has yet to garner buzz but soon will is Apple’s Ferret, the company’s open-source multimodal large language model (MLLM) launched quietly last October.
The tool is meant for research, but marketers should still watch how Apple benefits in the consumer space as AI begins to enter devices rather than turning to the cloud.
What Is Ferret?
Apple’s Ferret was created as a joint project between Apple and Cornell University. It is the result of extensive research on how large language models can recognize and contextualize elements within a picture image. This means a user interface with a Ferret model underneath can manage queries like those for ChatGPT or Gemini, yet it can also allow users to create a region by drawing a bounding box and then identify prompt-relevant elements within that region.
How It Works
For example, let’s say we have a Ferret model that is given an image. The image contains an animal somewhere in a section of the image — it’s not in the foreground, just a set of pixels as part of a background. The animal is highlighted by the user within a prompt, and the user asks the model “What is this animal?” Using its trained data as a basis, the model will identify the species of the animal and provide supporting information, showing that it understands how the user is referencing a portion of an image.
Spatial Referencing
This identifying technique is called spatial referencing. It is used for identifying items in a section of media indicated through a drawn box. The benefit of the technique in Apple’s Ferret is superior accuracy and performance using a novel approach — combining discrete coordinates and continuous features of an image — as an identifying methodology. This differs dramatically from the current generative AI models. For example, with Gemini, an image can be uploaded, and items in the image recognized, but the user does not encircle the item as a prompt indicator. Apple’s Ferret is designed to potentially permit that kind of drawing in a prompt.
Apple's Ferret For Research Only
The model code, along with its features such as model weights, has been released on GitHub. The model and material are for research purposes only. While the model is robust, AI engineers and data scientists are studying the spatial referencing techniques further. Therefore, no commercial application is available.
Two Model Sizes
Apple’s Ferret has two model sizes available: a 7-billion parameter model and a 13-billion parameter model. The 7-billion model is of specific interest because it is tailor-made for iOS devices. AI developers have a significant interest in large language models operating from within smart devices. There has been research on having a smaller model because research has discovered that smaller models operate with accuracy equal to or better than a larger model.
LLMs on Small Devices
Apple also released a number of other associated research materials. Two research papers were released, highlighting specific interest in LLMs on small devices. One paper focused on new techniques for 3D avatars, which could potentially be used in portable devices such as Vision Pro, the AR headset. The second paper examines new methods for efficient model inference from its data.
Related Article: Apple's AI Moment Is Coming. It May Not Be Smooth
Learning Opportunities
Conference
Jun
3
Customer Contact Week Las Vegas 2024
Register
Conference
Jun
3
Gartner Marketing Symposium/Xpo Conference Denver 2024
Register
Webinar
Jun
4
More to Images and Video Than Meets the Eye
Examine how IT leaders employ omnichannel strategies to tackle image and video management challenges.
Register
Conference
Jun
10
NICE Interactions Las Vegas 2024
Register
Webinar
Jun
11
How GenAI Is Redefining The Digital Customer Experience
Learn how Coveo's platform leverages AI to understand user intent and behavior to provide more accurate search results.
Register
Webinar
Jun
11
Scaling Up: Using Search to Accelerate Your Content Personalization Journey
Join Jill Roberson from Velir to dive deep into the dynamic synergy between content personalization and search strategies.
Register
Apple’s Ferret Is a Potential Game Changer for Business Models With AI
Despite the limited availability, the Ferret release is a key piece of a commercial puzzle that is slowly taking shape for Apple's business model. Apple has had a very low profile in the AI space compared to Amazon, Google and Microsoft. Despite that, Apple has been launching a few elements in its products that could position it well for where AI is heading.
Tantalizing Possibilities
Companies are looking to integrate more AI models on devices. The interest in smaller LLMs raises the tantalizing possibility of having assistants working in mobile apps and in combination with tablets and smartphones. Given that iPhone represents a 51% share of the global market, along with iPad having a 30% share of the tablet market, Apple's motivation is very high. The end result of that motivation is Apple's most well-known products potentially introducing AI-enhanced features.
M3 Processor Series
For example, Apple released its M3 processor series with the capability to run machine learning and AI models locally. They contain powerful CPU and GPU cores, along with a dedicated AI coprocessor, that significantly increase processor performance compared to their M1 predecessors. The result is faster operation management, crucial for executing machine learning tasks. This increases the value of Apple's core products, including its Mac, MacBook Air and MacBook Pro computers.
Impact on Siri
The one Apple offering that stands to gain the most potential from an AI-feature refresh is Siri. The assistant was the first major voice assistant on the market, released in 2011, giving Siri early exposure and brand recognition. But despite the typical iteration of improvements, Siri has lagged behind Amazon Alexa and Google Assistant in terms of features. Siri has had fewer integrations with third-party apps and services compared to Google. An AI refinement in Siri can renew the value of the product. This is crucial as Siri competitors seek ways to leverage LLMs as a way to renew their convenience. For example, this January, I attended a developer presentation at CodeMash, a developer conference, where the speaker confirmed that Amazon is developing an LLM model application for Alexa.
Tim Cook Coy
All of this is speculation, for sure. But it also complements Apple CEO Tim Cook's teaser comments during a recent earnings report call to Wall Street analysts. Cook mentioned potential new AI-enhanced offerings from Apple several times but remained coy on the details.
Related Article: Transformative Lessons: My Growth in Customer Experience During Steve Jobs' Era at Apple
What Does Apple's Ferret Mean for Customer Experience and AI?
Apple's entry into consumer AI could influence new sales for mobile devices and other products. If Apple promotes Ferret as an iPhone feature, for example, it will reenergize customer interest in the smartphone market. Growth in US sales remains strong, but at a lower rate compared to earlier years. The market is maturing, so a manufacturer like Apple needs another clear benefit for customers other than phone size. AI enhancements like photo search, image enhancement, and animated media can spark customer demand.
Consumer AI Entry
Apple's entry into the consumer AI space will also shape competition for customer experience and AI on mobile devices and other products. An Apple entry would push Google to find opportunities to add Gemini to Android devices, enhancing the customer experience AI landscape.
Attracting AI Developers
Moreover, the AI enhancements would entice developers to build apps to leverage the built-in feature. A three-way competition between Apple, Google, and OpenAI to attract AI developers could emerge, with these big three fighting to retain customers by having the most useful and entertaining apps available in their app stores.
Related Article: ICYMI: AI-Augmented CX, Apple's Potential CX Game-Changer
Until the Day AI Means ‘Apple Innovation’ …
In the meantime, the best marketers can do is await what Apple will do in AI to determine how customer experience AI in portable devices emerges. The big AI reveal Cook referenced is expected toward the end of the year.
Until then, marketers need to stay focused on how Apple invests in the consumer AI space, as well as how its competitors respond.
