An assistant that sees what you see

You are in a queue at the pharmacy, wearing glasses with a camera, and you ask your assistant for the name of the medicine the doctor mentioned this morning. It answers at once, because it saw the prescription on the desk. It also saw the prescription of the person in front of you, the screen of the pharmacist’s terminal and the face of a child waiting with her mother. None of them asked to be part of your assistant’s day.

This is no longer a thought experiment. XREAL opened pre-orders for Aura on 7 October: display glasses on Android XR, with Gemini built in and a higher-resolution camera with a privacy LED, there, in XREAL’s words, for real-time perception. Gemini on Android XR, as XREAL describes it, understands what you are looking at and helps with tasks in context. It is a good idea. It also changes what an assistant is.

The real cause: a camera collects context without choosing it

An assistant on a phone sees what you show it. An assistant at eye level sees what you see, and you see a lot that is none of your business: other people’s screens, other people’s papers, other people. The value of a seeing assistant comes from exactly this, from context you did not have to type. So does the risk. Nothing about the camera knows the difference between the diagram you needed and the stranger behind it.

XREAL does not publicly set out which parts of the assistant’s work happen on the glasses’ puck and which in the cloud. That is not an accusation; most makers do not say. It does mean the most intimate data an assistant can hold, a record of what you looked at, goes somewhere you cannot point to.

What goes wrong with an assistant that sees

Seeing turns into remembering. If frames, or descriptions of frames, go into memory by default, a day of glances becomes a searchable diary that nobody decided to write.

Bystanders never agreed. A privacy LED informs the people around you. It does not ask them, and it cannot let them say no.

Recordings live where you do not. Without a clear split between local and remote work, you cannot know how long a frame exists, who can read it, or whether deleting it on your side deletes it anywhere else.

Permission to see is not permission to act. An assistant that can open apps and move through the interface for you, and can also see a form on a screen, is one step away from filling it in. The camera permission screen says nothing about that step.

Context leaks between rooms. The meeting at work and the kitchen at home pass through the same lens into the same memory, unless something keeps them apart.

The light is drawn by software. On many devices the indicator is controlled by the same software that controls the camera. You are asked to trust the light because you trust the code.

How KyberBot is designed for it

KyberBot is pre-launch, and Kyber does not integrate with Android XR. This is the design we are building to, for any body that has a camera.

Looking is not remembering. By default a frame is used for the question you asked and then dropped. Memory keeps only what you mark or what a rule you can read allows, and kyber memory show lists it (the command is illustrative). What it means to be known by an assistant is set out in an assistant that knows you.

Memory on your devices, encrypted. There is no account holding what you saw and no advertising business to feed, as your data is not the product explains. When a task needs more power than a pocket has, where the stream may go is the subject of the cloud behind your eyes on KyberFusion.

Other people first. Faces, screens and documents that are not yours are a design target for being discarded before anything is kept, not after. It is the hardest part of the problem, and we say it as a goal, not a result.

Ask before the sensitive step. Seeing a payment screen does not let KyberBot pay. Anything that spends, sends, signs or shares in your name stops and asks you. When the work goes to a swarm, the glance becomes a brief, which KyberAgent covers in from the glasses to the swarm.

A cut you can see. For the body, we want a physical switch on sensor power rather than a setting; KyberDeck’s view of the glasses-and-puck shape is in glasses with a puck.

The other half of this question is whose assistant it is when it is built into the platform, covered in Android XR: the assistant in the system.

A test you can run today

Open your phone’s photo gallery and scroll back one month. Count the photos that contain people who do not know they are in them, or screens and papers that are not yours. That is roughly what an assistant looking through your eyes would hold after a month, except it would be every minute, not every photo.

Then ask the assistant you use today two things: “what do you remember about me?” and “delete it”. If you cannot see the answer to the first, or cannot check the second, you already know how it would handle what it sees. More on the assistant we are building is on the homepage.