Picture a sheep walking behind another sheep. For a moment, the one you were following disappears. Then a white, woolly shape comes back into view. Is that your sheep? A video segmentation model needs to answer this question while also drawing its outline. POSReasoner looks at what happens between those two jobs: deciding whether a new observation should change what the model remembers.
A good outline can still belong to the wrong object
A mask tells us which pixels belong to an object. Across a video, we also need a stable identity: this mask and that earlier mask belong to the same sheep. A neatly drawn outline is not much comfort if it has quietly switched to the sheep next door.
The trouble can outlast the frame where it started. If an ambiguous observation is accepted and written into memory, later observations are compared against that mistake. The model now has the wrong evidence helping it make its next decision. Occlusion makes this particularly awkward: the least reliable moment can become the moment that changes the record.
Keep a record beyond the current frame
POSReasoner gives each object a persistent state. Along with identity information, that state carries confidence and absence history. The current image is therefore one source of evidence, rather than the only evidence used to decide whether to keep its state.
This makes a useful distinction possible. “I cannot see it right now” need not mean “forget what we knew about it.” When the object returns, its new observation can be checked against the surviving state. Keeping that history does not prove the match is correct; it gives the model something better to consult than a single uncertain frame.
Make a suggestion. Then check it.
The method separates this decision into two steps: Propose and Verify. Propose forms provisional associations between persistent states and current observations. These are suggestions, not updates that have already been committed to memory.
Verify revisits those associations using object history, predicted presence, and competition between identities. That last part matters. Two similar objects may both appear to match the same candidate when considered separately. Their claims need to be considered together. A sparse state–observation graph gives the reasoner a way to exchange this information without treating every possible connection as equally relevant.
An update is a decision, too
After verification, the system can retain, update, reactivate, or suppress a state. A learned gate also controls how much evidence is written back. So choosing an association and deciding what to remember are related decisions, but they are not collapsed into one automatic step.
Go back to the sheep. A weak glimpse need not replace a useful history immediately. A later, better-supported observation can provide grounds to reactivate the state. Only the verified transition is carried into the persistent state used for subsequent frames.
The character can leave the picture without erasing the state kept below. When it comes back, a possible match still needs verification before it can change that record.
What the comparison actually tests
To test this, we keep the underlying segmentation model frozen. The paired comparisons use the same host checkpoint, candidate masks, input resolution, and evaluator. The trainable part is POSReasoner, learned from standard video annotations. Keeping the rest fixed helps us see what changes when we change the state decisions.
The reported experiments cover video instance segmentation and long-term video object segmentation, including OVIS, YouTube-VIS, and LVOS. Improvements appear across multiple hosts, particularly around occlusion and reappearance. Those are the moments to look for in the examples: not just a clean outline, but the right object after a gap.
What it can—and cannot—remember
There is still a segmentation model underneath. POSReasoner reasons over the evidence that model supplies; it is not a promise to recover any object from any disappearance. If the available observations remain ambiguous, retaining a state does not magically reveal the correct identity. Verification is learned, too, and can make mistakes.
You can read the paper on arXiv or explore the public repository. If you want to run it, start with the reproduction guide: datasets, host checkpoints, prediction exports, and trained reasoner weights are not bundled. The code makes the steps inspectable, but those external artifacts still need to be prepared.