NVIDIA Shock | The Full Picture of Omni AI: 9x Speed × 25 Companies Adopting It
機械翻訳 / Machine-translated
A
AI Friends
2026年4月30日・47分で読める
機械翻訳 / Machine-translated
@aifriends
AI Friends（https://aifriends.jp）のクロスポスト公式アカウント。AIツールの紹介・使い方・できることを、中学生でもわかるやさしい日本語で届けます。
"One AI that watches video, listens to audio, and answers in text" — what sounds like a sci-fi movie is now a reality running on just 25 GB of memory.
On April 28, 2026, NVIDIA announced its new model "Nemotron 3 Nano Omni," sending shockwaves through the industry.
With a new 30B-A3B Hybrid MoE design — where only 3 billion of 30 billion parameters are active at a time — it has demonstrated 9x faster video processing than rival Qwen3-Omni, 4x faster than Google's Gemini 3.0 Pro, and more than 2x faster than OpenAI's GPT-5.1.
We break down in plain language exactly what this "AI agent that can see, hear, and speak" is — the one that prompted 25 major companies including Foxconn and Palantir to immediately announce adoption or evaluation.
Let's start by looking at what happened from three angles.
The announcement date was April 28, 2026 (early morning Japan time on April 29, 2026), published simultaneously on NVIDIA's official blog, Hugging Face, and build.nvidia.com.
The official name is "Nemotron 3 Nano Omni." "Nemotron" derives from "Nemo" (Latin for "no one") and is NVIDIA's open model brand. "Omni" is Latin for "all" — meaning "integrating all modalities (types of information) into one."
The design philosophy is: "Pack all the eyes, ears, and a mouth of AI into a single package."
Where traditionally "image-only AI," "audio-only AI," and "language-only AI" ran separately and were connected, this single model handles everything end-to-end.
Think of it as: "Instead of calling in multiple specialists, getting one generalist physician who can handle everything" — a major simplification.
On NVIDIA's official technical blog, it was positioned as "the true contender for agentic AI (AI that acts autonomously)."
Let's look under the hood.
Breaking down the keyword "30B-A3B Hybrid MoE": "30B" is the total of 30 billion parameters (the number of knowledge slots inside the AI); "A3B" means only 3 billion parameters actually activate per query; and "Hybrid MoE" refers to a new type of Mixture-of-Experts structure.
Think of it as: "A 30-person company that calls in only the 3 relevant specialists for each project."
The other 27 are on standby, dramatically reducing memory consumption and energy costs.
It also adopts a new language model design called "Mamba-Transformer Hybrid," combining the best of Mamba (strong at long-text processing) and Transformer (high accuracy).
The visual encoder integrates NVIDIA's proprietary "C-RADIOv4-H," and the audio encoder integrates "Parakeet-TDT-0.6B-v2."
This new design — like "three specialist mechanics sharing one car with a veteran engineer to maximize efficiency" — is the secret behind the 9x speed.
The term "Omni" is spreading rapidly in the AI industry.
Traditional AI existed as separate entities: "text-only (GPT-3)," "image-only (DALL-E)," "audio-only (Whisper)."
It was like "a general hospital where you visit a different building for each specialty" — inconvenient.
In 2024, GPT-4o (OpenAI) appeared as the first full-fledged "omni" model, integrating text, images, and audio into one.
In 2025, Alibaba's Qwen3-Omni, Apple Intelligence, and others followed in rapid succession.
Nemotron 3 Nano Omni in April 2026 surpasses the competition by being both "open (free for anyone to use) and fast."
It's a revolution like: "A 'super-doctor' who combines all specialties into one, offering consultations for free."
Industry publications are describing this as "2026 is Year One of Omni."
From multimodal (multiple modalities) to omni (all modalities unified) — the mainstream of the AI industry is unmistakably shifting right now.
Let's look at the performance figures.
On a single NVIDIA B200 GPU, output throughput reaches up to 5,000 tokens/second at maximum concurrent connections.
That's the speed of "an AI spitting out the equivalent of 12 manuscript pages per second."
While maintaining 50 tokens/second per user (the standard speed at which humans can read), it handles 9x more simultaneous users compared to Qwen3-Omni.
Think of it as: "Where Qwen3 can handle 10 people simultaneously on the same hardware, Nemotron can handle 90 at once."
A single server can cover hundreds of employees at a company.
It's a business revolution where "a call center's phone switchboard is reduced to one unit thanks to AI capacity."
This is what those numbers mean: the cost of electricity and servers for large-scale AI agent operation drops sharply.
Where specifically does it top the performance rankings?
In "document understanding," it ranked #1 on two major benchmarks: "MMlongbench-Doc" and "OCRBenchV2." In "video and audio understanding," it ranked #1 in four areas including "WorldSense" and "VoiceBench" — a total of 6 leaderboard top spots.
Think of it as: "A student who holds a national #1 ranking in 6 different subjects."
OCRBenchV2 in particular measures accuracy in document scanning and reading — one of the most important metrics for business use.
WorldSense tests contextual understanding of video (who did what, where) — a practical test like "summarizing an incident from surveillance camera footage."
"The ability to read textbooks and answer questions (OCR-type)," "the ability to watch a video and describe what happened (World-type)," "the ability to listen and understand (Voice-type)" — all ranked #1.
It's an extraordinary feat, like "a single student winning national competitions in science, social studies, PE, and music."
The cost side is equally stunning.
On the video tagging task "MediaPerf," it processes 9.91 hours of video per hour (meaning 10x+ real-time processing speed), with the same work costing just $14.27 (approximately ¥2,140) — the lowest among all models.
The same job takes GPT-5.1 more than 2x as long, and Gemini 3.0 Pro more than 4x as long.
Economically, it's like "a super employee at ¥2,140/hour who is faster and cheaper than a competitor charging ¥4,000/hour."
The contrast is stark: "An AI summarizing 10 hours of video at ¥2,140/hour vs. a human summarizing it at ¥10,000/hour."
Bulk processing of surveillance footage, corporate meeting minutes, and call center recordings — this is a tipping point where "AI video workflows that were previously ruled out on cost grounds suddenly become practical."
It means an industry shift where "AI agent operating costs can fit within a company's IT budget."
Starting with the biggest rival.
Alibaba's Qwen3-Omni is an open multimodal model with the same "30B-A3B MoE" configuration as Nemotron — "twins that look identical on paper."
The difference is internal design: Qwen3 uses a standard Transformer base, while Nemotron adopts a "Mamba-Transformer Hybrid."
Think of it as "two 30-person companies with different workflows."
Mamba is overwhelmingly more memory-efficient for long-text processing (256K tokens = roughly 400 manuscript pages) — giving Nemotron an advantage for "processing a thick contract in one go."
The result: 9x faster video processing speed than Qwen3 on the MediaPerf benchmark.
"Even companies with the same headcount produce 9x the output if their business system is superior" — that's the difference good design makes.
"The same exterior can yield different results depending on what's inside" — that's the artistry of AI design.
Comparing with Google's latest flagship.
On the same video processing task, Nemotron 3 Nano Omni completes it in 8.30 hours, while Gemini 3.0 Pro requires 33+ hours (more than 4x the time).
It's the difference between "finishing a job by the end of a workday vs. a job that takes 4 straight days and still isn't done."
Gemini 3.0 Pro is a paid model on Google Cloud costing tens of thousands of yen per month; Nemotron 3 Nano Omni is openly published and free for anyone to use — like "a high-end restaurant reversed by a fast food counter that's both faster and cheaper."
Of course, Gemini still excels in reasoning accuracy and breadth of knowledge, but Nemotron has the advantage specifically in cost- and speed-sensitive business processing.
"If you're aiming for 100 points, Gemini; if you need 80 points done 10x faster, Nemotron" — they serve different purposes.
The axis of "premium model vs. high-speed low-cost model" is becoming the new competitive dimension in the 2026 AI industry.
The comparison with OpenAI's flagship is also striking.
On the same video processing task, Nemotron completes it in 8.30 hours; GPT-5.1 takes 16.5+ hours (roughly 2x the time).
GPT-5.1 is the core model behind ChatGPT Plus and Enterprise, with API usage billed per-token at several tens of yen per 1,000 tokens.
Nemotron 3 Nano Omni is freely available and can be deployed on your own servers or in the cloud.
The picture is: "GPT-5.1, supremely capable but time-consuming and expensive, vs. Nemotron, fast and cheap."
The gap is especially pronounced for high-volume batch processing like "large-scale document OCR, surveillance footage analysis, and audio meeting records."
For a company processing 100,000 videos per month, the annual cost savings could run into the tens to hundreds of millions of yen.
"Choosing an AI model is no longer just about 'performance' — it's the age of deciding by three axes: speed × cost × license."
In 2026, AI engineers need the "three-sword style" as an essential skill: switching between premium, mid-tier, and high-speed models depending on the use case.
Let's look at the companies NVIDIA has officially named as having already adopted the model.
①Aible (AI analytics platform), ②Applied Scientific Intelligence/ASI, ③Eka Care (India's largest healthcare AI), ④Foxconn (world's largest EMS: contract electronics manufacturer, makes iPhones too), ⑤H Company (French agentic AI firm), ⑥Palantir (major US AI vendor for defense and intelligence), ⑦Pyler (Korean startup) — 7 companies total.
"Manufacturing (Foxconn), healthcare (Eka), defense (Palantir), analytics (Aible) — the leading players across every industry are lined up together."
Palantir's adoption is particularly noteworthy — as the primary AI vendor for the US Department of Defense and intelligence agencies, its adoption is a certificate that says "this AI is fit for national security."
Foxconn's adoption hints at repurposing for "factory automation = AI agents with physical embodiment."
With "the top companies across every industry planting their flags simultaneously," credibility was established from day one.
The companies in the evaluation stage are equally impressive.
①Dell Technologies (global PC and server giant), ②Docusign (world's largest e-signature company), ③Infosys (one of India's Big Three IT firms, 150,000 employees globally), ④K-Dense, ⑤Lila, ⑥Oracle (database and cloud giant), ⑦Zefr, ⑧several others.
"IT, cloud, and systems integration superpowers that are expected to officially adopt it next" — a formidable lineup.
Oracle, the world leader in databases and ERP, could position Nemotron at the core of its enterprise AI strategy. Infosys, a global SI giant on par with NTT Data and Accenture, is expected to deploy it broadly as an AI implementation partner for multinational corporations.
Docusign's adoption suggests a massive use case: "automated analysis of electronic contracts and generation of minutes from audio meetings."
The future where "one day, Nemotron quietly powers business AI in offices around the world" looks set to become reality within the next six months.
Here are the three use cases NVIDIA envisions.
①Computer Use Agents: Autonomously operate browsers and app UIs by analyzing what's on screen. H Company announced an implementation that reads screens at full HD resolution (1920×1080 pixels).
Think of it as "AI operating accounting software on behalf of humans and issuing invoices" — business automation.
②Document Intelligence: Analyze invoices, contracts, and charts as images; understand a mix of tables, charts, and screenshots. Think of it as "scanning 10,000 paper invoices and automatically sorting their contents" — an accounting revolution.
③Audio/Video Understanding: Integrated analysis of call center recordings and screen operation logs; detecting incidents and anomalies from surveillance camera footage. Think of it as "AI simultaneously handling security guard and receptionist duties" — business consolidation.
"All three go-to use cases can be handled by a single Nemotron" — that's the deciding factor for adoption by 25 companies.
Let's look at the ripple effects on the Japanese market.
Nemotron 3 Nano Omni can run on 25 GB of memory with 4-bit quantization (a compression technique) — "light enough to run on a high-end consumer PC (e.g., one equipped with an RTX 4090)."
This shifts the paradigm: "Multimodal AI that previously required cloud GPUs can now run on a single in-house server."
NVIDIA Japan is scheduled to begin offering Nemotron to Japanese companies via NIM (NVIDIA Inference Microservices: a microservices-based inference platform) from May 2026 onward.
"Usable in a closed, on-premises environment even for tasks with data leakage risks" — a feature that is well received by Japanese companies.
Industry publications report that major SIers such as Fujitsu, NEC, and NTT Data are expected to announce Nemotron-powered solutions one after another in the second half of 2026.
"The era of in-house omni AI — no longer relying on overseas cloud AI" — is beginning in Japan as well.
The impact on Japan's homegrown AI is also worth watching.
The reality is that Japan's domestic large language models — NTT's tsuzumi, SoftBank's "Sakura," CyberAgent's CALM, and others — cannot match Nemotron in raw performance.
However, they differentiate with Japanese language specialization, domestic data storage, and compliance capabilities — "competing on 'reassurance' even where they can't win on performance."
Industry observers predict that in the second half of 2026, domestic AI companies will announce derivative models based on Nemotron with additional Japanese-language fine-tuning.
"Building on an overseas open model and adding Japanese language and industry knowledge" — a pragmatic approach.
The strategy of "not building domestically from scratch, but leveraging the world's best and adapting it for Japan" is becoming mainstream.
The definition of "domestic AI" is evolving — from "entirely homegrown" to "open foundation + Japanese-language specialization."
What are the concrete opportunities for Japanese companies?
①Call center operations: Analyze audio + screen logs + FAQ documents with a single AI, immediately improving response quality. Efficiency gains like "standardizing phone response quality and cutting operator training costs by 70%."
②Factories and construction sites: Integrated analysis of surveillance footage + worker audio + inspection records, improving safety management accuracy. A safety revolution where "anomalies are detected instantly, preventing accidents before they happen."
③Healthcare and nursing care: Process consultation videos + medical records + audio conversations with a single model, automatically generating medical records. A workplace improvement where "the 2 hours a day doctors spend on record entry is freed up."
④Retail: Analyze store cameras + POS audio + product databases, visualizing customer traffic patterns and interests.
From 2026 through 2027, these "omni AI business revolutions" are expected to unfold simultaneously across Japan.
Yoshida works in international procurement at a general trading company with 40,000 employees.
In June 2026, he handles a daily flood of English contract PDFs and English web conference recordings with overseas suppliers.
Previously: three separate steps with three different tools — contract OCR (GPT-4) → translation (DeepL) → meeting summary (Whisper + ChatGPT).
4 hours per case, 300 cases per month = 1,200 hours per month (the equivalent of 7 employees).
After deploying Nemotron 3 Nano Omni on the company's in-house server, feeding the contract PDF + meeting recording into a single model handles everything. 30 minutes per case.
Processing time compressed to 150 hours per month — one-eighth of before — with 5 employees shifted to higher-value work.
Yoshida is commended by his division head for "contribution to AI efficiency," and his salary rises from ¥8.2M to ¥11.5M through rapid promotion.
"The track record of single-handedly leading the department's transition to omni AI" reaches senior management, and he becomes a candidate for head of the AI Promotion Division.
A real-world example of "one decision to switch AI models that completely changed a career."
Saito manages the electronic medical record system at a 500-bed general hospital.
In July 2026, she proposes deploying an omni AI system for "consultation video + doctor audio + automatic medical record generation" to improve physician efficiency.
Previously: "recording → Whisper transcription → manual correction by doctor → record entry" — 15 minutes of post-consultation work per appointment.
With Nemotron 3 Nano Omni, the video and audio from the consultation are analyzed in real time, and a draft medical record is ready the moment the consultation ends. The doctor's review time is reduced to 5 minutes.
30 appointments/day × 100 physicians × 10 minutes saved = 900 hours of resources freed per month.
"Doctors now have more time to focus on patients" and "work satisfaction improved by 42%" — praised by the hospital director. In December 2026, it expands into a joint deployment project with 10 neighboring hospitals.
Saito is promoted to acting head of the information systems department and recognized in the medical AI industry as "a champion of frontline improvement."
A case study in "how discovering a small problem on the ground can transform regional healthcare."
Yamaguchi serves as CTO of a 12-person AI startup.
In May 2026, upon seeing the open release of Nemotron 3 Nano Omni, he completes development of a "one-stop omni AI business support SaaS for SMEs" in just three weeks.
Offering quality control for manufacturers, traffic analysis for retailers, and call center quality improvement in a single package — priced from ¥50,000/month, one-tenth of major competitors.
150 companies rush to adopt it in three months; monthly revenue grows to roughly ¥75M.
In September 2026, the startup raises ¥2.2B in a Series B round, with partnership negotiations underway with major SaaS players.
Yamaguchi is selected for an economics magazine's "30 Young AI Entrepreneurs of 2026." His strategy — "translating overseas open models into practical tools for Japanese businesses" — becomes an industry template.
He embodies the new-era CTO archetype: "A startup's future is decided by whether you can move on the day an open AI is released."
A. The answer is: "Yes, completely free."
The license is "open-weight with commercial use permitted (full release including weights)." It's available for immediate download and use on Hugging Face, OpenRouter (free), build.nvidia.com, Amazon SageMaker JumpStart, and Vultr.
However, running it on your own server requires a high-performance GPU (minimum NVIDIA RTX 4090 class, 25 GB VRAM) — "the model itself is free, but the machine to run it costs money."
For cloud use, there is pay-as-you-go billing via AWS or Vultr (GPU instance charges).
The reality: "Individual engineers can try it on a home PC; enterprise deployment requires a server investment of several million yen."
A. The answer is: "A new design combining MoE architecture × Mamba-Transformer."
MoE (Mixture-of-Experts) works by "activating only the 3 billion specialist parameters relevant to the query, while the other 27 billion rest" — an energy-efficient design like "a 30-person company where only 3 relevant specialists work per project."
This alone makes it 10x more efficient in terms of electricity and processing speed compared to a standard 30B model.
On top of that, the Mamba-Transformer Hybrid is overwhelmingly more memory-efficient for long-text processing (256K = 400 manuscript pages) — faster than competitors for "processing a thick contract in one go."
"Expert activation (MoE) × long-text efficiency (Mamba) × NVIDIA proprietary encoders" — this triple combination is the secret behind 9x speed.
A. "If you have a PC with an RTX 4090 (25 GB VRAM), it runs at home."
Requirements: ①High-performance GPU (RTX 4090, A100, or B200), ②Python 3.10 or higher, ③vLLM and Transformers libraries, ④25 GB free VRAM.
"An open multimodal AI that runs on the latest gaming PC (worth ¥300,000–400,000)" — more accessible than you might think.
Via OpenRouter, no GPU is needed and you can try it for a few thousand yen per month; a free browser demo is also available via Hugging Face Spaces.
Tiered options: "Try it for free, then buy a high-performance PC when you're ready for serious use."
Starting in 2026, the era where individual developers run enterprise-grade AI at home is finally becoming reality.
A. The current situation: "It supports Japanese, but is not a Japanese-specialized model."
Nemotron 3 Nano Omni is a multilingual model centered on English. It can process Japanese text, audio, and images, but accuracy for industry-specific terminology and keigo (polite speech) is somewhat lower than Japanese-specialized LLMs (tsuzumi, Sakura, etc.).
Think of it as "a native English speaker who also speaks Japanese, but without the nuance of a native Japanese speaker."
In the second half of 2026, domestic AI companies are expected to announce derivative models based on Nemotron with additional Japanese-language fine-tuning — "overseas model + Japanese tuning" is expected to become the standard approach for Japanese companies.
For now, the best practice is a "dual-wield" approach: Nemotron in its original form for English-language work, and a domestically specialized model for Japanese-language work.
"See, listen, and respond" — the age has arrived where AI achieves the integrated perception that humans take for granted, with the lightness of just 30B-A3B.
The announcement of Nemotron 3 Nano Omni in April 2026 is not merely the arrival of a new model — it's the moment the mainstream of the AI industry shifted from "multimodal" to "omni integration."
The fact that global giants like Foxconn, Palantir, Oracle, and Infosys announced adoption on day one is a strong signal that "this design is becoming the industry standard."
For Japanese companies, it's a turning point where the door opens to AI-driven business transformation in call centers, healthcare, manufacturing, and retail — areas where cost had previously been a barrier.
Three things you can start doing today: ①Try the demo on Hugging Face, ②Identify projects in your business where video, audio, and documents are mixed, ③Check for announcements of domestically derived models once a month — small steps like these will determine your competitive edge in the era of omni AI.
This article is a cross-post from AI Friends.
