SenseNova Loop: SenseTime's Open Looped 9B Model

SenseTime has released SenseNova Loop, a 9-billion-parameter multimodal model that runs part of its network twice, as open weights under the Apache 2.0 licence. The model, published as SenseNova-Loop-9B-Preview on Hugging Face and GitHub on 30 October, takes text and images and produces text. SenseTime says it is built for long documents, coding and work with external tools.

The design is the news. Instead of adding more layers, SenseTime repeats a block of existing ones, so the model does more computation without storing more weights. “Scaling compute depth, not just model size,” the company wrote on X. It is labelled a preview. The technical report is listed as “TBD”, so the design can be described but not yet judged.

How the SenseNova Loop architecture works

The model has 32 decoder layers. According to the model card, input passes through layers 0 to 3. The block of layers 4 to 27 then runs twice in a row, with the second pass taking the first pass’s output as its input. Layers 28 to 31 and the output head finish the job. The card writes this as B(B(h)), where B is the repeated block.

Both passes use the same weights. The result, in the card’s words, is “56 decoder-layer computations per forward pass while storing only 32 sets of decoder-layer weights”. That is why SenseTime gives two sizes: about 9 billion total parameters, and about 14.6 billion “inference parameters”, the amount of computation a forward pass actually uses.

Two further pieces sit around the loop:

Hybrid attention. Each group of four layers has three linear-attention layers followed by one full-attention layer. The loop starts and ends on group boundaries, so the pattern holds on both passes.
Two-stream mHC. Inside the looped section, manifold-constrained hyper-connections carry two residual streams instead of one. Learned mappings decide how each layer reads from and mixes the streams. A Sinkhorn normalisation, run for 20 iterations, constrains the mixing.

The card lists a training context of up to 262,144 tokens, a vocabulary of 248,320, and a hidden size of 4,096. The backbone is stored in BF16, with the mHC routing parameters in FP32. TSN checked the loop range and layer sizes against the model’s published configuration file.

What ships, and what is missing

The release is a first build, not a finished product. The weights load in Hugging Face Transformers 5.18.0 with PyTorch 2.11.0, but only with trust_remote_code=True, which runs SenseTime’s own model code. The card says details of a standard Transformers implementation are “to be announced”. Serving instructions cover LightLLM and an open SGLang pull request. Multi-token prediction is available for faster inference, while two other speed-up methods, Dspark and Dflash2, are still in training.

There is no technical report, so this article makes no claims about how well SenseNova Loop performs. The launch post on X and a chart on the model card compare it with larger models on several benchmarks. Those comparisons cannot be checked until SenseTime publishes its method and test settings. The X post says SenseTime’s next steps are larger models and “adaptive fast/slow thinking”.

The Apache 2.0 licence is among the most permissive in open AI. Recent open releases such as Mistral’s Large 4, Liquid AI’s LFM3 and DiffuSpace’s Clover show how crowded the field is. Others, like Underdog’s Saluki, shrink existing models. SenseTime is trying to get more out of a small one.

Why model risk teams should read the loop before the licence

For a bank or any regulated firm thinking about a model like this, the headline parameter count is the wrong number to start with. A validation file that records “9B” understates the work the model does. Each forward pass runs 56 layer computations, which SenseTime itself counts as about 14.6 billion parameters’ worth. Cost, latency and hardware estimates, and any comparison with other models in an inventory, should use the inference figure and say so.

The code path matters as much as the weights. Running the model today means executing SenseTime’s custom code through trust_remote_code. For a security team, that is third-party software and should be reviewed and pinned to a specific version like any other dependency, not treated as a data file.

The missing report is the bigger gap. Model governance depends on documented training data, methods and test results. Without them, the benchmark comparisons in the launch post are marketing claims, not evidence a validator can use. A permissive licence settles what a firm may do with SenseNova Loop. It does not settle whether the firm can explain the model to a supervisor.

Until the “TBD” on the technical report changes, SenseNova Loop is a design to watch rather than a model to deploy.