When to Personalize Household Object Search: A Rigidity-Gated Hybrid Policy

Xianyao Li, Yuhai Wang, Hu Xiao, Kaleb Smith, Gilbert Yang Ye, Eric Jing Du

University of Florida · ICIC Lab

In every home,
small things go missing

A mug gets left in the bedroom. Keys end up somewhere unexpected.

Finding these things again is exactly the kind of job we want to hand to a home service robot. But a robot asked to fetch your mug shouldn't scan every room, one by one — it should look first where the mug is likely to be.

Where things end up depends on
the person — and the object itself

A robot's first guesses come from common sense about rooms — mugs near kitchens, towels near bathrooms. But that only goes so far. Where things actually end up is also shaped by the resident — some people return items to fixed storage, others tolerate clutter and relocate things all day — and by the object itself: some objects keep stable, universally shared placements, while others move fluidly with their owner's habits. That spectrum, from anchored to free-moving, is what we call placement rigidity.

The dilemma follows directly. Personalize where placement is universal, and the robot learns noise. Refuse to personalize where placement depends on the resident, and it wastes a real chance to search faster. A toothbrush is the first kind; a cell phone, the second.

The data to study this
is hard to get

Studying the dilemma needs weeks of object movements, tied to who the resident is, inside private homes — invasive to collect, hard to scale. And existing embodied datasets offer static layouts, not day-to-day movement.

Large language models could synthesize this behavior at scale. But unconstrained, an LLM reproduces its own commonsense prior, not how real residents behave — and biased priors can make search worse. So no one can yet say when personalization pays off.

So the question stands open:
when should a robot personalize?

Answering it takes exactly what's so hard to get: trait-conditioned placement data a robot can learn from, and evidence from real people about where personalization genuinely helps.

This is what our paper proposes: PerSim — the rigidity-gated hybrid policy of our title — a framework that builds both.

Describe residents
with the Big Five

Every part of PerSim conditions on the resident, so we first fix the representation. The Big Five — psychology's standard model: Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism — turns a person into a continuous five-number trait vector, measured by a two-minute questionnaire.

Continuous is the point: PerSim can interpolate across trait space and predict for personalities it has never seen, instead of forcing residents into a few fixed types.

Calibrate the generator
on real people

Here is where the bottleneck gets answered. In round one of the human study, participants report their Big Five profile and their anchors — where they actually keep everyday objects.

We fine-tune the generator LLM on those anchors, pinning it to real behavior rather than its own commonsense. Calibration first, generation second.

Round one also scores
every object's rigidity

The anchors carry one more signal: participants rate how fixed each object's place is. So every object gets a rigidity score — the property we met at the very beginning, now measured object by object.

A trait vector goes in;
weeks of placements come out

Now the generation itself. Give the calibrated generator a home layout and a trait vector — a synthetic resident — and it produces an initial layout: where that resident keeps things by default. Then it updates the home day after day: which room each object moves to, and what it sits next to once there.

We sample trait vectors to cover the full five-dimensional space, repeat across multiple home layouts, and automatically check every output — keeping only valid transitions.

Now we can learn a
trait-conditioned prior

On this data we train a predictor: give it a resident's personality profile and an object, and it ranks where that object is likely to be, for that person.

The same data also yields its competitor. Drop the trait vector and count where objects usually sit across everyone, and you get the population-frequency baseline. Both the data and these two priors now face the real test: round two of the human study, in two layers.

To human eyes, the synthetic
behavior is plausible

Layer one tests the data itself. Judging blind, participants rated our placement transitions 3.85 out of 5 for plausibility — a reliable result under our power analysis.

We keep the claim deliberately modest — plausible, no more. But that's the foundation the two priors stand on.

Personalization is preferred —
mainly for free-moving objects

Layer two compares the two priors head-to-head: a blinded A/B choice between trait-conditioned suggestions and the population-frequency baseline. Participants did prefer personalization — but mostly for objects whose placement varies from person to person. For universally placed items, the population answer held its ground.

This gradient is not a side result — it points directly to the decision rule.

One gate,
two priors, two stages

Here is that rule as a running policy. The gate turns an object's rigidity score into a mixing weight: the lower the rigidity, the more the search trusts the trait-conditioned prior; the higher, the more it trusts the population baseline.

The blended prior then drives a two-stage search: rank the rooms, enter the best one, and check the cues inside — the objects the target usually sits next to.

Personalize selectively,
win overall

Finally, we run the policy end-to-end in a home digital twin. The gated hybrid cuts expected search cost below both the population baseline and full personalization — and the savings concentrate exactly where the study predicted: on low-rigidity objects.

A model you can play with

On top of the paper, we also fine-tuned an open-weights model — Gemma 4 12B — on our human-calibrated data, and released it publicly. We host it as a webchat on our lab server: ask it about a persona, and it reasons about where that person would leave their things.