BorgVR Guide

Apple Vision Pro

BorgVR for visionOS

Explore volume data freely in space, annotate it with hands, Logitech Muse, or PlayStation VR2 Sense controllers, and analyze it together through SharePlay.

Mandelbulb rendered by BorgVR

Erase and restore volume

Dataset export is disabled during SharePlay. While an export is running, edits and edit-state changes are blocked for Muse, controllers, and MCP as well. If SharePlay starts during an export, that export is cancelled before the shared edit state is installed.

A small red dot in the top-right corner of the Edit Volume button indicates pending background brick updates. It disappears once calculation and saving have finished.

Export Edited Dataset opens a dialog for the name and description. During calculation it displays progress; Cancel stops the export and removes incomplete export files. Afterwards, choose the save location. The name and description are stored in the new dataset. Export preserves the source compression and copies unchanged bricks without decompressing or recompressing them.

If saved edit states exist, BorgVR asks which one to use before opening a dataset normally. Original stays unchanged. The last personal state is preselected; the trash button deletes one state after confirmation. The Brush panel lets you select, create, duplicate, and rename states.

Pending volume updates in the selected edit state are completed automatically before opening. BorgVR shows progress when needed; no additional choice is required. Export and preparation share the same exact bottom-up rebuilding service with Metal acceleration when available and a CPU fallback.

SharePlay uses a separate shared edit state initialized from the host's selected state. Joining receives its complete mask without overwriting personal states. State selection is locked during a session. Erase/restore cursors and brush operations are shared. Leaving restores the previous personal selection; the shared work remains available as a separate SharePlay state.

Edit states are stored beside the .data file in its .data.edits companion folder; only built-in datasets use private app storage. Remote datasets use the local dataset directory. Discard All Edits restores only the active state to the original after confirmation; other saved states are preserved. During SharePlay, this clears the shared state for all participants.

Choose the red Edit mode in the BorgVR main menu. The separate red Brush button below opens and closes the Brush window; changing modes does not open it automatically. Erase sets voxels inside the spherical brush to zero in every channel; Restore brings back the original values. Pinch and move your hand through the volume. Use two hands to scale the radius, or use the Brush window slider with physical units.

Muse and PSVR have independent Erase and Restore tools. On Muse, cycle to these tools by pressing both side buttons; the large button edits the volume. Double-click the small digital button to switch between Erase and Restore. An eraser with a minus identifies Erase; a plus identifies Restore. Hold the small button and move the pen vertically to change the radius. Assign Erase and Restore separately to each PSVR controller or reach them with Next Tool. The index-finger trigger edits the volume; grip plus vertical stick changes the radius. The red or green sphere shows the affected region. The reset icon above Edit clears the active edit state after confirmation.

By default the brush edits at the actual tool position. Project onto Volume can be enabled separately. The original file remains unchanged and local edits survive reopening. SharePlay synchronizes edits in a separate shared state. Export Edited Dataset creates a new .data file with a new ID containing the changes permanently.

First launch and main menu

Normal opening tries the selected remote server first and known alternatives when needed. Disabling progressive loading downloads the entire dataset before opening. SharePlay joins instead open remote datasets progressively. Keep a Local Copy controls persistence; when notifications are enabled, BorgVR can notify you once the download is complete.

After confirmation, the dataset selector's trash button removes the local .data file and its entire .data.edits directory, not just the last-used variant. Separate object, measurement, TF, and view files remain. Back up or export variants you need first. Data and Edit States explains storage, Original, personal variants, shared SharePlay states, and export in detail.

  1. Choose Open Dataset to load a local file or a dataset from a configured server.
  2. Use Import Dataset to convert DICOM, NRRD/NHDR, PVM, or QVIS data into BorgVR format.
  3. After opening, the Immersive Space starts. The volume appears in front of you while the BorgVR main menu remains available as a visionOS window.

On the first regular launch, BorgVR asks for a SharePlay display name. It identifies you in shared sessions and can be changed later in Settings. When the first launch happens directly through SharePlay, this prompt is deferred to avoid interrupting the join flow.

Hand interaction

The Hand Interaction selector in the main menu determines only what pinch gestures modify. Logitech Muse and PlayStation VR2 Sense controllers maintain independent tool states.

Model
Move, rotate, and scale the volume. Its reset restores the initial transform.
Clipping
Position and rotate the clipping plane to inspect the dataset interior.
Transfer
Horizontal dragging shifts the active transfer function, while vertical dragging changes its slope. In isosurface mode, the same delta interaction changes the isovalue.
Drawing
A pinch gesture draws a spatial stroke. It may optionally project onto the visible volume surface along the viewing ray.
Placement
Place, grab, and move the active sphere or imported polygon object. The preview uses the actual shape and size and can optionally snap to the volume surface.
Measurement
Create and edit points for path lengths, planar areas, or convex volumes. Points may optionally project onto the volume surface.
Screen View
Position the shared screen view of Mac, iPhone, and iPad participants in space. This mode appears only while such a view exists.

Rendering, multi-channel data, and lighting

The main menu switches between transfer function, illuminated transfer function, and isosurface rendering. The editor button opens the transfer-function or isovalue editor for the active mode. The larger transfer-function editor can additionally appear as a HUD or spatial widget attached to the dataset.

Multi-channel datasets have up to four independent transfer functions. Select the data channel in the editor, then edit its red, green, blue, and alpha curves. BorgVR composites the contributions of all channels during rendering.

The Lighting window contains an arcball preview sphere and color controls for ambient, diffuse, and specular light. Rotate the sphere to change light direction. The settings affect TF with Lighting and Iso, not unlit TF. A multiline warning explains the inactive controls in that mode.

Objects and measurements

The Objects window shows the active object asset and all placed spheres, drawings, and mesh instances. Imported .mesh files appear beside the built-in sphere as placeable objects. Multiple selection supports shared edits; .marker files store the complete object collection.

The Measurements window selects length, area, or volume. Lengths may contain any number of segments. Areas use the convex hull of points projected into a common plane; volumes use a spatial convex hull. Points can be selected, moved, and removed. Results use an automatically selected physical unit and are stored as .measurement.

Objects and the complete measurement visualization can be hidden independently. Both are visible when a dataset opens.

Logitech Muse

Hand and accessory modes are independent: for example, rotate the volume with your hands while drawing with Muse. The starting-tool setting applies whenever a dataset opens, not only after an app restart. The rendered tip shows color and size; the tool symbol sits farther back on the pen. Hold the small button to adjust parameters; double-clicks and pressing both buttons have the separate functions described below.

Logitech Muse works as an independent tool and does not follow the current hand mode. Its tip shows color, size, and active mode. Settings choose whether a new dataset starts in marker, length, area, volume, or last-used mode.

  • In drawing mode, the primary button or pressure-sensitive tip creates smoothed spatial strokes. Pressure scales their radius.
  • The small button changes size with up/down motion and hue with left/right motion. A double-click removes the most recently created object while drawing, or the last measurement point while measuring. While editing a volume, it switches between Erase and Restore.
  • Press both buttons together to cycle through drawing, placement, length, area, and volume measurement, Erase, and Restore. A short press in measurement mode starts a new measurement or returns to the last edited one.
  • Settings control whether other SharePlay participants can see the Muse tip. When disabled, no preview positions are transmitted.

PlayStation VR2 Sense

Mappings are saved separately for left and right. Tool actions change only that controller, not the hand mode or the other controller; global actions such as a rendering-mode change affect visualization instead. Settings show the actual button names and symbols reported by visionOS. The index-finger trigger and grip button are different inputs.

Left and right PlayStation VR2 Sense controllers are tracked separately and each maintains its own tool mode. The rendered controller pointer shows the active tool and its current drawing or object preview at the tip.

  • In model mode, the trigger grabs the dataset; controller motion and orientation translate and rotate it.
  • In drawing mode, trigger or grip creates spatial strokes. Tool modes also include clipping, object placement, length, area, and volume measurement, and Screen View.
  • The four face buttons, two on the left and two on the right, can be assigned separately. BorgVR uses the localized names and symbols supplied by visionOS. These are not the system buttons that open visionOS menus.
  • Horizontal and vertical thumbstick axes can independently map to rotation about X, Y, or Z or translation along X, Y, or Z for each controller.
Accessory limitationvisionOS accessory tracking may limit simultaneous use of Logitech Muse and both PS VR2 Sense controllers. If only one controller responds, temporarily disconnect another spatial accessory.

SharePlay

The temporary server offers only the host dataset and is also advertised when a device finishes downloading it during the session. Waiting for Data Source concerns a dataset server's reachability; Synchronizing Edit State instead concerns the shared edit mask. Closing the host dataset or ending its Immersive Space makes clients wait for the next choice. Closing as a client asks to leave the session. See the SharePlay chapter for details and troubleshooting.

Use the SharePlay button to start or join a session. Dataset, rendering parameters, transfer functions for all data channels, lighting, clipping, objects, and measurements are shared. The protocol version is checked before payload data; incompatible app versions receive a clear update message.

BorgVR visualizes the shared screen view as an eye in front of an image plane. If a Mac, iPhone, or iPad participant detaches, a separate color-coded representation appears with that person's display name. Its frustum appears during interaction.

Participants exchange known dataset origins that are allowed to be shared. Devices with a complete local copy can serve the host-selected dataset through a temporary ad-hoc server. While no origin is reachable, BorgVR displays the server currently being tried and retries known origins.

When the host leavesParticipants may take over the host role, leave the session, or continue without a host. Simultaneous takeover attempts are resolved deterministically.

Saving, restoring, and settings

Quick Marker and voice

Simplifying the user interface

Under Settings → User Interface, choose visible rendering and interaction modes. At least one mode of each kind remains enabled; a selector with only one mode disappears. RGB/RGBA rendering is unaffected. Hide the reset row and the panel-button row, or hide individual panel buttons. Diagnostic controls are configured here as well. Preferences stay local and are included in settings export and import. This simplifies the UI rather than restricting SharePlay or automation access. On visionOS, Muse and PSVR retain their independent tool modes.

If SharePlay activates a locally hidden rendering mode, it temporarily appears selected and disabled in the selector, even when only one local mode is allowed. The extra entry disappears when a locally allowed mode becomes active again; preferences remain unchanged. On macOS and iOS, TF/isovalue, object, measurement, and brush panel buttons require their associated interaction modes; the object button remains available for either drawing or placement. Lighting stays independent. On visionOS, panel buttons remain independent of hand modes so Muse and PSVR can still use them.

Quick Marker is off by default. When enabled, a fast double pinch outside Drawing and Placement places a sphere at your hand position. Hold the second pinch to move it, then release to continue in the previous hand mode. Adjust the double-pinch interval beside the information control in Settings. The shortcut is inactive in Drawing, Placement, and Screen View.

Use the microphone control for voice input. Its help window lists the localized commands actually supported, including rendering/hand modes and Quick Marker. Grant the required microphone and speech-recognition permissions. The separate voice-help window is the reference for exact phrases in the selected language.

Save Dataset State and Restore Dataset State combine transfer functions, objects, measurements, transform, rendering mode, isovalue, lighting, clipping, and visibility. Existing files are overwritten only after confirmation.

  • Automatic loading and saving of transfer functions, objects, measurements, and visualization state
  • Transfer-function HUD and spatial widget plus selected data channel
  • Starting tool and SharePlay visibility for Logitech Muse
  • Button and thumbstick mapping for left and right PS VR2 Sense controllers
  • Remote servers, loading timeout, progressive loading, local copies, and origin cache
  • Local dataset and WebGPU server plus independent MCP server
  • LOD, atlas, sampling, and foveation options
  • SharePlay display name and App Store update checks

Foveation sampling strength

Under Settings → Rendering → Sampling, Foveation sampling strength controls how strongly the rate map affects sample spacing. 0 disables this adjustment, not system foveation. 1 preserves the previous default. Values up to 4 increase peripheral step sizes further to trade image quality for performance, without changing sampling in the full-rate center. The setting applies immediately to TF, lit TF, color-volume, and isosurface rendering, stays local, and is included in settings export and import.