Hugging Face launches Storage Buckets for mutable checkpoints, logs, and agent traces

Storage Buckets are the first new repo type on the Hub in four years, built for artifacts that are large, mutable, and operational rather than publish-once deliverables. In the launch post, Hugging Face contrasts them with Models and Datasets repos, saying those are good for “final artifacts,” while Buckets are for the production stream of “checkpoints, optimizer states, processed shards, logs, traces” that “rarely need version control.”

The interface and access model stay close to the rest of the Hub. The launch blog says Buckets can be browsed in the Hub UI, automated from Python, or managed through the hf CLI, and the same post describes them as mutable, non-versioned storage with standard namespace permissions for user or organization scopes. That makes this less of a separate storage product than a new storage primitive inside existing Hub workflows.

The main engineering value is moving high-churn artifacts out of Git-backed repos without leaving the Hub. In Hugging Face's thread, the company frames the problem bluntly: Git breaks down on the “high-throughput side of AI,” and Buckets are meant to handle “fast writes, overwrites, directory sync” for recurring training and inference-adjacent outputs. The blog link adds that this includes intermediate files from many jobs at once, which is the pattern that makes versioned repos noisy and expensive.

The storage backend is also part of the story. Hugging Face says Buckets are powered by Xet dedup, so overlapping files do not re-upload identical chunks; in the product wording, “successive checkpoints skip the bytes that already exist” the thread. The blog post also describes pre-warming, which brings data closer to compute regions to improve throughput for distributed training. Separately, a repost from Hugging Face co-founder Thomas Wolf calls this “our fastest growing recent product” and says the company is making “petabyte storage cheap and fast” Wolf quote, which underscores that this launch is tied to scale-out storage demand rather than a cosmetic Hub update.