Official SourcesfromTogether AI Blog
Expanding our enterprise inference capacity with IBM Cloud and NVIDIA
Enterprises can now run open models at production scale on a dedicated B300 inference cluster, built by Together AI, IBM Cloud, and NVIDIA
Speed & Cost
50 items tagged with this topic
Recent
Podcasts & NewslettersfromImport AI
Import AI 475: Swarm scaling; Google DeepMind watermarks biology; and the AI science economy
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now When should you use swarms? When you are in a hurry:…How…
Podcasts & NewslettersfromUnsupervised Learning
Ep 94: Applied Compute CEO on the Limits of RL, the New AI Hyperscaler & Why Post-Training Wins Inference
Speaker 1 | 00:00 - 00:02 How do you kind of characterize where we are today? Speaker 2 | 00:02 - 00:25 What we have with RL is we essentially have a hill climbing machine. The hardest part is actually defining the hill to climb, which is…
Older
Official Sourcesfrom Google DeepMind Blog
Introducing SynthID Bio
Proof of concept for watermarking AI-generated proteins while preserving biological function.
Buildersfrom X
Harnesses from labs have an incentive to burn tokens, which means harnesses from startups have…
Harnesses from labs have an incentive to burn tokens, which means harnesses from startups have a real utility, like how Grep can automatically swap token burn into deterministic tested code that is repeatable just by ob…
Podcasts & Newslettersfrom No Priors
Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile
Speaker 1 | 00:00 - 00:30 Right now, we're trying to build a single chip. If you crack open an NVIDIA system, it has anywhere between kind of six and nine custom chips all built by NVIDIA to come together to build something that is really,…
Watchlistfrom Anthropic Engineering
An update on recent Claude Code quality reports
Over the past month, we’ve been looking into reports that Claude’s responses have worsened for some users. We’ve traced these reports to three separate changes that affected Claude Code, the Claude Agent SDK, and Claude Cowork. The API was…
Podcasts & Newslettersfrom Latent Space Newsletter
Why Dwarkesh is Wrong about Computer Use + How OpenAI shipped its Jev competitor in 1 Week
Our DevDay coverage - the first pod on the DevDay lineup - dives in with the leaders of OpenAI’s CUA team and API platform.
Official Sourcesfrom Microsoft Research Blog
Offloaded inference for real-world physical AI robotics
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI work…
Official Sourcesfrom Together AI Blog
Canary rollouts: upgrade models in production without downtime
A hard model swap exposes every user at once, and rolling back means cold-starting the old deployment under pressure. Here's how staged traffic ramps, metric gates, and automatic rollback work on dedicated inference.
Podcasts & Newslettersfrom Latent Space Newsletter
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
GWM Worlds 2 uses persistent context and timed actions to steer a world model generating video and audio in real time.
Podcasts & Newslettersfrom Latent Space Newsletter
Foundries vs Navigators: Lowering the Cost of Science
Guest Post: In science, thinking has gotten cheap but doing has not. This asymmetry is reshaping how research companies operate, largely inconspicuously.
Podcasts & Newslettersfrom The MAD Podcast with Matt Turck
Who Feeds the GPUs? Inside AI's Hidden $30B Layer | Renen Hallak, VAST Data
Speaker 1 | 00:00 - 00:17 Sometimes it scares me. We had a customer, one of these AI clouds, they said we're probably going to need about 500 petabytes over the next three years. Last week, came back to us and said, we're gonna need an ext…
Buildersfrom X
It tells you a lot when someone has moved around a lot in their career. Also tells you a lot wh…
It tells you a lot when someone has moved around a lot in their career. Also tells you a lot when there is low company turnover. A team that has been working together for 3+ years is significantly higher throughput and…
Official Sourcesfrom Together AI Blog
How a global fintech scaled coding agent traffic with Dedicated Model Inference
Inside a global bank's shift to self-serve dedicated inference: how Together's DMI gave engineering teams direct control over scaling, models, and testing.
Official Sourcesfrom OpenAI News
Better prompt caching for GPT-6
Learn how GPT-6 improves prompt caching with higher cache hit rates, new diagnostics, explicit breakpoints, and controls that reduce latency and costs.
Chinese Modelsfrom X
Looks like today may be a record day for token volume % of open models on Vercel AI Gateway: 🟦…
Looks like today may be a record day for token volume % of open models on Vercel AI Gateway: 🟦 Open 78.4% 🟨 Closed 21.6% While spend 💲 usually tells a different story, #3 and #4 today are Moonshot AI & DeepSeek. Addi…
Podcasts & Newslettersfrom No Priors
Why Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon
Speaker 1 | 00:05 - 00:31 Hi, listeners. Welcome back to No Priors. Today, I'm here with Stefano Erman, who is a longtime Stanford professor and now co founder and CEO of Inception. Stefano has a extraordinarily broad body of work around g…
Buildersfrom X
Agents already make up the majority of inference. This will quickly trend toward nearly all inf…
Agents already make up the majority of inference. This will quickly trend toward nearly all inference over the next year or two. The vast majority of tokens used in the world will be agents that are executing unbelievab…
Official Sourcesfrom OpenAI News
Rapidly scaling online storage to serve over 1 billion ChatGPT users
Learn how OpenAI evolved Habitat from a Python library into a globally distributed storage platform serving 1 billion ChatGPT users and 22M requests per second.
Official Sourcesfrom Together AI Blog
The Open Source AI Stack
A deep dive into the open model AI stack — model, inference, gateways and routers, harness, and tools — and how keeping each layer independent lets you swap in a new open model in minutes instead of rebuilding your workflow.
Podcasts & Newslettersfrom No Priors
Coinbase’s Everything Exchange: Agentic Finance, Stablecoins, and Tokenization with CEO Brian Armstrong
Speaker 1 | 00:00 - 00:19 The minimum fee typically on a debit or credit card transaction is about 30¢ fee flat plus the additional percentage. Arguably, it doesn't really work well for anything under a dollar. Certainly, it doesn't make s…
Buildersfrom X
▲ ~/𝚛𝚊𝚞𝚌𝚑𝚐/𝚘𝚜𝚜-𝚐𝚛𝚊𝚗𝚝𝚜/ 𝚕𝚜▐ (v2) I'm announcing v2 of my open source grants of…
▲ ~/𝚛𝚊𝚞𝚌𝚑𝚐/𝚘𝚜𝚜-𝚐𝚛𝚊𝚗𝚝𝚜/ 𝚕𝚜▐ (v2) I'm announcing v2 of my open source grants of $1,000USD to 35 awesome contributors. The themes: ① Agent skills & tools Skills are now a valuable form of software, like Em…
Buildersfrom X
Me: I totally understand how RL, inference time scaling, modern data-verification loops work. A…
Me: I totally understand how RL, inference time scaling, modern data-verification loops work. Also Me: These models are total sorcery. There is no way that they should be able to do what they do.
Buildersfrom X
The hard part with becoming more ambitious is changing the how. We’re living in a time where AI…
The hard part with becoming more ambitious is changing the how. We’re living in a time where AI and market conditions make asymmetric outcomes far more possible. That applies to scale, velocity, breadth of product, pers…
Podcasts & Newslettersfrom Latent Space Newsletter
[AINews] Fal’s H3 Max Live breaks the infinite videogen barrier
You can now create decent video faster than you watch it. This is the start of... something. We’re not sure what.
Official Sourcesfrom OpenAI News
OpenAI supports California’s bill to advance youth AI safety
OpenAI supports California SB 1119, advancing strong, age-appropriate AI safeguards for teens while preserving opportunities to learn, create, and explore.
Podcasts & Newslettersfrom Latent Space Newsletter
[AINews] Hot Chips: OpenAI’s Jalapeño, Cerebras CS-5, Groq 3 LPX, Apple M6
The conference with hot chips and even hotter companies
Official Sourcesfrom OpenAI News
Jalapeño’s first results show industry-leading speed and efficiency in AI inference
Jalapeño is a custom inference chip from OpenAI that delivers faster, more power-efficient AI inference, with higher throughput and lower latency for modern models.
Buildersfrom X
Intelligence is getting cheaper. @OpenAI Sol's price reductions & discounts on Vercel AI Gatewa…
Intelligence is getting cheaper. @OpenAI Sol's price reductions & discounts on Vercel AI Gateway have made Sol our fastest-growing frontier model. This shows ① that the demand for intelligence is highly elastic: as infe…
Buildersfrom X
How to build great evals - part 6 Hill climbing on evals is just a fancy way of saying: pick a…
How to build great evals - part 6 Hill climbing on evals is just a fancy way of saying: pick a dimension that matters and optimize for it. This could be improving the quality of existing features based on your latest pr…
Chinese Modelsfrom Latent Space Newsletter
[AINews] Death of Params: Z.ai CEO Jie Tang on GLM 5.3 and the new Post-training Scaling Law
Every lab CEO is on X now
Buildersfrom X
the models have no moat (OpenAI, Anthropic, XAI) the IDEs have no moat (Cursor, Windsurf) the h…
the models have no moat (OpenAI, Anthropic, XAI) the IDEs have no moat (Cursor, Windsurf) the harnesses have no moat (Cognition, Factory, LangChain) the app builders have no moat (Replit, Lovable, Bolt) the wrappers hav…
Podcasts & Newslettersfrom Import AI
Import AI 468: 23 RSI ideas; PostTrainBench+; and how trust and transparency interplay with AI racing
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now Want to be able to deal with RSI? Here are 23 actionable…
Podcasts & Newslettersfrom Latent Space Newsletter
[AINews] How to steal a Reasoning Trace
Speculative Decoding by any other name would distil as sweet
Buildersfrom X
🤖Android users love Gemini + Gemini can automate actions across 40+ popular apps + Great for b…
🤖Android users love Gemini + Gemini can automate actions across 40+ popular apps + Great for booking rides, reserving tables, and more + More updates coming tomorrow at @madebygoogle!
Podcasts & Newslettersfrom Latent Space Newsletter
[AINews] Megakernels are so dead and so back
A quiet day lets us highlight a Cursor launch and an engineering debate
Podcasts & Newslettersfrom Import AI
Import AI 467: Self-sustaining AI viruses; pacing AI progress; confusion about AI and creativity
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now Self-sustaining and self-replicating AI viruses are here:…
Buildersfrom X
a very primitive form of the near term multiagent agi future is setting up one thread to ping b…
a very primitive form of the near term multiagent agi future is setting up one thread to ping back once its done so you create an implicit kanban/waterfall graph of dependent threads but each preserving their own work a…
Official Sourcesfrom Together AI Blog
Autoscaling endpoints for LLM inference
GPU utilization can read healthy while your queue backs up, and a new replica takes minutes to warm. Here's how to pick autoscaling metrics, tune scale-up/down windows, and budget for cold starts on dedicated inference.
Chinese Modelsfrom Latent Space Newsletter
The Inference Engineering Masterclass — Philip Kiely & Ali Taha, Baseten
Baseten just raised a $13B Series F and is now one of the leading kings of inference engineering. We go into everything you need to know for autoregressive and diffusion engineering.
Buildersfrom X
The best model for early product validation is almost never the best model for production. Talk…
The best model for early product validation is almost never the best model for production. Talked to a bunch of founders this weekend. There’s a standard playbook emerging : 1/ Prototype with the best frontier models. M…
Official Sourcesfrom OpenAI News
How we built a realtime system for responsive voice AI in six months
GPT-Live enables continuous voice interaction with AI, using a turnless speech model and low-latency architecture for faster, more natural conversations.
Official Sourcesfrom Microsoft Research Blog
EvoLib: Turning experience into evolving knowledge
LLMs do not get smarter just by remembering more. EvoLib turns experience into evolving knowledge, taking reusable skills and insights that help models learn and adapt across tasks long after deployment. The post EvoLib: Turning experience…
Buildersfrom X
Another day another near frontier open weights model release. If you had gone back even 3-6 mon…
Another day another near frontier open weights model release. If you had gone back even 3-6 months and given everyone access to what we’re now seeing in open weights even as a closed model, their minds would be complete…
Official Sourcesfrom Together AI Blog
Configuring Dedicated Model Inference
The three-part resource model behind Together AI Dedicated Model Inference—endpoints, deployments, configs—and how capacity-aware routing ties them together.
Official Sourcesfrom Together AI Blog
ThunderAgent: 2x Faster Agentic Inference for Synthetic Data Generation at Scale
ThunderAgent is a program-aware scheduler for agentic inference. By treating each agent workflow as a schedulable program, it eliminates KV cache thrashing to deliver more than 2x single-node throughput and near-linear multi-node scaling.
Podcasts & Newslettersfrom Latent Space Newsletter
[AINews] GPT 5.6 price cut by 20%-80%: Cost of GPT 5.4 Intelligence dropped 13x in 4 months due to GPT 5.6 recursive self-optimization
Distillation is all you need!
Podcasts & Newslettersfrom Import AI
Import AI 466: The bitter lesson for robotics, AIs complete week-long programming tasks; and OpenAI’s accidental AI hacker
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe. Subscribe now Epoch and METR release MirrorCode, a benchmark for seeing…
Podcasts & Newslettersfrom Unsupervised Learning
Ep 92: xAI Co-Founder Unpacks the Future of Model Development
Speaker 1 | 00:00 - 00:16 Welcome back to unsupervised learning. I'm Jacob Efron. We had an awesome episode today with Igor Babushkin. Igor has been at all the right places at all the right times, leading a lot of really interesting AI wor…