BorgVR Guide

Agent control

Model Context Protocol

Trusted AI agents can inspect datasets, control the visualization, and evaluate rendered images without driving the user interface remotely.

Mandelbulb volume controlled through BorgVR

What BorgVR MCP provides

All three native renderers contain the same optional MCP server. It is independent of the dataset and WebGPU servers and uses the same central command layer as the macOS scripting language. An agent can therefore act reproducibly instead of operating buttons through mouse or gaze control.

  • Read the dataset catalog with complete metadata and known origins, then open or close datasets
  • Read and change rendering mode, isovalue, transform, clipping, and lighting
  • Manage objects, sphere and stroke markers, mesh instances, and length, area, and volume measurements
  • Hide objects or the complete measurement visualization without deleting data
  • Wait for a completed rendered frame and retrieve the current view as a PNG image

Enabling MCP

  1. Open BorgVR Settings and select the MCP section.
  2. Enable Enable MCP server and choose another port if required. The default is 8765.
  3. Add one of the displayed endpoints to the configuration of your MCP client.
This device onlyWithout network access enabled, BorgVR listens only at http://127.0.0.1:8765/mcp. Local connections require neither TLS nor an access token.

Access from the network

When Allow connections from the network is enabled, BorgVR binds the MCP server to all available network interfaces and displays an HTTPS endpoint for every reachable address. Depending on router, firewall, and port forwarding, the service may also be reachable from outside the local network.

  • Generate an access token or enter your own. The client sends it as Authorization: Bearer ….
  • The generation button copies the new token directly to the clipboard.
  • Network connections use HTTPS. Without an imported PKCS#12 identity, BorgVR creates a temporary self-signed certificate whenever the server starts.
Trust boundaryA connected MCP client can open datasets, change the visualization, edit annotations and measurements, and read screenshots. Share the endpoint and token only with trusted systems.

Typical agent workflow

  1. The agent uses list_datasets to inspect metadata and origins, then opens the selected dataset with open_dataset.
  2. It inspects get_application_state, changes the view and rendering parameters, and hides objects or measurements when required.
  3. After each change it uses wait_until_rendered to wait for a new completed frame, then calls capture_screenshot.
  4. By comparing multiple views, rendering modes, isovalues, or clipping ranges, the agent can inspect visible structures systematically.

The JSON schemas returned by tools/list are the authoritative reference for argument names and value ranges.

Troubleshooting

Empty response or no connection
Verify that scheme and endpoint match: local access uses http, network access uses https.
Unauthorized
For network access, the Authorization header must contain exactly the currently displayed bearer token. Local access on the same device does not check a token.
Certificate error
Explicitly trust the self-signed certificate or import your own PKCS#12 identity. Some externally hosted agents cannot reach private or local IP addresses at all.