In 2024, industry giants like Google and Qualcomm predict a shift towards on-device AI. During MediaTek’s recent Executive Summit, the leading silicon producer for millions of phones and smart devices emphasized the benefits of leveraging generative AI in mobile technology. This approach promises faster, more personalized answers and suggestions compared to cloud-based alternatives, hinting at a significant enhancement in phone capabilities.
David Ku, MediaTek’s Chief Financial Officer, outlined specific use cases on the horizon. For example, Vivo plans to launch a phone next year equipped with a unique feature. This feature allows users to record, transcribe, and summarize meetings directly on the device. The significance lies in the confidentiality it offers, especially in sensitive boardroom discussions. By processing the information on-device, the recording remains private, eliminating the need to upload it to the cloud—a crucial aspect for maintaining confidentiality in private meetings, as pointed out by Ku.
The integration of on-device AI opens up possibilities for enhanced privacy and efficiency in various applications, marking a notable advancement in the realm of mobile technology.
In October, Qualcomm, a competitor in the chipmaking industry, integrated generative AI capabilities directly into the chips powering the upcoming premium Android phones, including the Xiaomi 14. While MediaTek didn’t delve into as many specific use cases for its generative AI, both companies envision the technology providing more personalized suggestions by analyzing individuals’ behavior patterns and engaging in image manipulation, such as expanding photos beyond their original boundaries.
Adding to this technological landscape, MediaTek introduces its new Redcap (abbreviated for “reduced capacity”) 5G modems and chips. These components are designed to support devices with lower data loads, catering to smart home devices, earbuds, and accessories. This strategic move sets the stage for enhanced communication between various devices used by both individuals and companies. MediaTek’s recent announcements signify a transformation of traditional technology into the interconnected future that has been anticipated for years.
Discovering the Vague Demesne of On-Device Generative AI
In mobile technology, on-device generative AI is gaining attention for its potential benefits, promising quicker personalized answers and suggestions compared to cloud-based alternatives. Despite this promise, the transformative impact on phones remains somewhat elusive.
One suggested application of generative AI involves the removal of unwanted objects from photos, akin to Google’s Magic Eraser on Pixel phones and the Google One cloud photo storage service. Another envisioned feature is the automatic generation of accompanying text for photos destined for social media.
Additionally, the potential for enhancing personal assistant features through generative AI is highlighted. For instance, a phone equipped with this technology could offer contextual suggestions, such as sending a message to notify others when you’re running late for a meeting. Even if your phone is on mute, the assistant could alert you to urgent emails.
MediaTek has entered this landscape with two new phone chips: the Dimensity 9300 for premium phones and the Dimensity 8300 for less premium devices. Both chips boast on-device generative AI capabilities. Performance details are available for the former, claiming the ability to generate images using Stable Diffusion in less than a second and run AI models with 7 billion parameters per second at 20 tokens per second. While the Snapdragon 8 Gen 3 achieves a similar tokens-per-second rate, comparisons are limited, especially in the context of on-device AI for Qualcomm’s PC chip (30 tokens per second).
The lower-end Dimensity 8300 can handle language learning models with up to 10 billion parameters, while the Dimensity 9300 takes it a step further, supporting language learning models with up to 33 billion parameters.
To be equitable, there’s currently no universally agreed-upon metric for comparing the capability of generative AI. Presently, ‘tokens per second’ and ‘trillion operations per second’ stand out as the most commonly employed metrics. However, MediaTek is actively collaborating with various ecosystem partners to establish a more definitive benchmark, as mentioned by Ku.
“We’re striving to offer more precise guidance and identify the genuine benchmark that holds greater significance from a user perspective,” stated Ku.
Challenges and Expanding Horizons with On-Device Generative AI
Embracing new technological frontiers brings its own set of challenges. Concerns are already emerging about the potential misuse of generative AI to inundate the internet with fake images and misinformation. Qualcomm, addressing this issue, has proposed a solution by collaborating with the authentication platform Truepic to digitally watermark images altered or created using generative AI. MediaTek is taking a similar approach, partnering with various entities to implement an authentication method resembling an “invisible watermark,” ensuring that any level of modification, be it 50% or 5%, receives this distinct marker.
Ku emphasized the significance of this authentication, stating, “Maybe that’s the most important 5%. Maybe it’s your face.”
Looking beyond smartphones, MediaTek envisions generative AI playing a role in the automotive industry. Affirming its partnership with GPU maker Nvidia, the company speculates on the potential for cars to leverage generative AI alongside external cameras. This could enable the identification of objects ahead and, with gesture recognition, clarification of distant points the driver is indicating. Cars could also detect erratic driving patterns, inquire about driver drowsiness, or alert them when the traffic light turns green.
Generative AI’s utility extends to personal assistants in cars. For instance, on a hot day, an AI could observe when the temperature inside the car has cooled down and prompt the driver to roll up the windows or adjust the air conditioning. In case of traffic delays, the assistant could send notifications to meeting attendees, informing them of the driver’s expected delay.
Anticipating a more interconnected 2024, MediaTek acknowledges the potential benefits of improved connectivity. While devices can utilize generative AI offline, enhancing connectivity between devices and networks remains crucial. MediaTek’s recent chip releases supporting Wi-Fi 7, with further improvements this year, mark a step toward this goal. Looking ahead, the company envisions advancements like Wi-Fi 8, driving higher data rates and lower latency. Wi-Fi 7, with features like multi-link operation, allows devices to simultaneously access the 2.4GHz, 5GHz, and/or 6GHz Wi-Fi bands for optimal connections. These developments are set to empower MediaTek’s chips in next year’s fixed wireless access terminals, enabling users to harness connectivity advances through their home 5G internet service.
RedCap 5G Revolutionizes Connectivity for Cost-Efficient Devices
RedCap 5G, designed by MediaTek, harnesses the low latency and other advantages of 5G mobile networks at a reduced cost, paving the way for more affordable devices to leverage the benefits of next-gen networks. While current 5G modems excel in power, performance, and range, they often surpass the needs of smaller IoT and smart home solutions.
According to Moynihan, the Senior Director of Corporate Sales and Business Development at MediaTek, these smaller devices don’t require the immense 10 gigabits per second provided by traditional 5G modems. Instead, a couple of hundred megabits per second are sufficient. With the advent of 5G standalone networks from carriers and new devices adhering to the 3GPP’s Release 17 standards, simpler gadgets can access 5G with lower latency and power consumption.
Looking ahead, Moynihan envisions the integration of on-device generative AI as part of the burgeoning concept of “ambient computing” in homes. This could involve smart devices utilizing user preferences to autonomously shape the environment. For instance, rather than manually setting lights, heating, or air conditioning preferences upon entering the home, devices could dynamically adjust based on individual preferences and arrival times, creating personalized environments for each household member.
MediaTek’s Foray into Augmented Reality
In pursuit of innovation, MediaTek explores new frontiers, including augmented and virtual reality. Having powered Sony’s PSVR2, the virtual reality headset for the PS5, the company announces a groundbreaking partnership at this year’s summit. MediaTek will collaborate with Meta, the parent company of Facebook, to develop a custom AR/VR chip.
While the specific device utilizing MediaTek’s chip remains undisclosed, speculation suggests applications in Meta Quest 3 or a successor to the Ray-Ban Meta Smart Glasses. Notably, unlike competitors, MediaTek takes a unique approach, customizing chips for each brand’s specific needs. Moynihan emphasizes their collaboration with Sony and Meta, highlighting the tailored nature of their chips and expressing intentions to partner with more brands in the future.
“We bring certain technologies to the table, and in the future, there are connectivity opportunities, whether through Wi-Fi, RedCap, or other technologies,” says Moynihan, hinting at MediaTek’s dynamic role in shaping the future of AR/VR technology.
