Segmentation Editor: volume rendering in the 3D viewport ("3D Volume" / "Surface" toggles + rendering menu)

1. Overview

Cornerstone3D supports two visualization modes in a 3D viewport: surface rendering and volume rendering. The Sonador 3D Volume Viewer has working, toggleable implementations of both; the Segmentation Editor's 3D viewport currently supports only surface visualization. Bring the volume viewer's capabilities to the Segmentation Editor:

  • Toggle surface and volume rendering on/off in the editor's 3D viewport. Unlike in the volume viewer, these toggles affect only the 3D viewport — they have no effect on the editor's 2D views or the sidebar panel.
  • A state indicator in the menubar, placed in a "3D" submenu, with toggles for "3D Volume" and "Surface" (the equivalent of the volume viewer's "More" menu).
  • When volume rendering is enabled, the rendering menu button appears in the lower-left corner of the 3D viewport (as in the volume viewer), opening the Rendering Presets dialog and Rendering Options menu; changes take immediate effect. When volume rendering is disabled, the button is not visible.

Design elaborated from the discussion on imaging-development-env#95 (note of 2026-07-12).

2. Reference implementation: the 3D Volume Viewer

Every mechanism this feature needs already exists in the volume viewer stack and should be followed closely:

  • Viewport transitions — Cornerstone3DVolumeViewport (extensions/viewer3d-volume/src/components/Cornerstone3DVolumeView.jsx) reacts to two boolean props in componentDidUpdate:
    • imageVolumeRenderingEnabled off: disableVolumeRendering() → c3dUtils.removeVolumeActors(viewportId) + re-render. On: _setImageVolume() (re-set the cached volume on the viewport and apply the modality default preset via viewport.setProperties({ preset })), then — because setVolumes() reinitializes the actor list and clears surface mesh actors — re-fire triggerSegmentationUpdate() so updateSurfaceData re-adds the surfaces.
    • segmentationSurfaceEnabled on: createSurfaceRepresentation() — epoch-guarded (_surfaceEpoch) so a toggle-off mid-computation aborts cleanly, pauses seg-rep updates (segRepUpdatePaused), calls _activateSurfaceRepresentation() (Surface representation + color LUT), then flips surfaceModelInit/surfaceRendering. Off: removeSurfaceRepresentation() (increments the epoch, removes the Surface representation) + c3dUtils.terminateWorkerComputeJobs().
  • Toggle plumbing — the volume viewer's "More" menu is a nested buttons group in extensions/viewer3d-volume/src/toolbarModule.js whose entries use DisplaySetAttributeActiveToolbarButton (extensions/vtk/src/toolbarComponents/DisplaySetAttributeActiveToolbarButton.jsx): the button's active state tracks a displaySet attribute (isActiveDisplaySetAttr, with isActiveDefault) and updates on DISPLAY_SET_CHANGED. The commands (toggleVolumeRendering, toggleSegmentationSurface in extensions/viewer3d-volume/src/commandsModule.js) are built by createViewportToggleFeatureCommand(propertyName), which flips the attribute on the active viewport's displaySet and republishes it through DisplaySetApi.Instance.displaySetService.addDisplaySets([...]). Note the guard: the command only toggles when the attribute is already non-nil, so the attribute must be initialized when the layout opens.
  • Attribute lifecycle — OHIFVtkVolumeViewport (extensions/viewer3d-volume/src/ohifComponents/OHIFVtkVolumeViewport.js) initializes imageVolumeRenderingEnabled / segmentationSurfaceEnabled (+ stableViewport) on the displaySet during load, subscribes to DISPLAY_SET_CHANGED and maps the attributes into component state (_evtDisplaySetUpdate, via _.pick(displaySet, ...)), and clears the attributes on unmount.
  • Rendering menu — VolumeRenderingMenuButton (extensions/vtk/src/components/VolumeRendering/VolumeRenderingMenuButton.jsx, exported from @ohif/extension-vtk) is rendered in the viewport wrapper as <div className="absolute bottom-2 left-2 z-10">, gated on imageVolumeRenderingEnabled, and takes a viewportId. Its presets/options (VolumeRenderingPresets, VolumeRenderingOptions, quality/shift/lighting/shade controls) drive the commands in extensions/vtk/src/commandsModule.js — setViewportPreset, setVolumeRenderingQuality, shiftVolumeOpacityPoints, setVolumeLighting — which resolve the viewport with getCornerstone3dViewport(viewportId) (scans all rendering engines, so it already works for viewports created outside the volume viewer) and call viewport.render() for immediate effect.

3. Current state of the Segmentation Editor 3D viewport

  • The editor's 3D tab (Cornerstone3DSegmentationViewerLayout.js, extensions/seg-editor/src/components/) is a ViewportType.VOLUME_3D viewport — the same viewport type the volume viewer uses — so image-volume rendering is available; it is simply never enabled with a rendering preset today.
  • Surfaces render from the editor's dedicated 3D labelmap (_segVol3d(), the "3D segmentation volume" split out to control the render loop), with progress/state via surfaceModelInit / surfaceRendering / surfaceRenderProgress and the LoadingIndicator.
  • 2D edits propagate to the 3D surface through segmentationService events and c3dUpdateSurfaceData (see the SEGMENT_REMOVED / SEGMENT_ADDED handlers in extensions/seg-editor/src/components/SegmentationEditorLayout.js).
  • The editor's toolbar (extensions/seg-editor/src/toolbarModule.js) currently contains only the Exit button; extensions/seg-editor/src/commandsModule.js contains only segmentationEditor / closeSegEditor.

4. Functional requirements

  • FR-1 — "3D" submenu. The editor menubar gains a "3D" submenu (nested buttons group, same construction as the volume viewer's "More" menu) containing two state-indicating toggles: "3D Volume" and "Surface". Each button's active state reflects the current rendering state.
  • FR-2 — Scope. The toggles affect only the Segmentation Editor's 3D viewport. They have no effect on the editor's 2D (Axial/Coronal/Sagittal) views, and no effect on the sidebar panel (the segmentation table remains fully functional regardless of toggle state).
  • FR-3 — Defaults. Surface: on; 3D Volume: off — matching the editor's current behavior on open.
  • FR-4 — Volume toggle. Enabling "3D Volume" sets the image volume on the 3D viewport and applies the modality default rendering preset (CT/MR, per the volume viewer's defaultVolumeRenderPresetCT / defaultVolumeRenderPresetMR convention); surfaces already in the scene must survive the transition (re-fire triggerSegmentationUpdate() after setVolumes(), per §2). Disabling removes the volume actors from the 3D scene and re-renders.
  • FR-5 — Surface toggle. Disabling "Surface" removes the Surface representation from the 3D viewport and cancels in-flight surface computation. Re-enabling recreates the representation — including any segments edited or added while the surface was off — with the existing Loading/Rendering progress indicator shown during recomputation (the machinery delivered for #94 (closed)).
  • FR-6 — Rendering menu button. When "3D Volume" is enabled, VolumeRenderingMenuButton is visible in the lower-left corner of the 3D viewport and opens the Rendering Presets dialog and Rendering Options menu; preset/quality/opacity/lighting changes apply immediately to the 3D viewport. When "3D Volume" is disabled, the button is not rendered.
  • FR-7 — Editing unaffected. Voxel-level state, the 2D labelmap, undo/redo history, and save behavior are unchanged by either toggle. Toggles are presentation-only.

5. Architectural requirements

  • AR-1 — Dedicated displaySet attributes. Introduce editor-scoped attributes — segEditorVolumeRenderingEnabled and segEditorSurfaceRenderingEnabled — rather than reusing the volume viewer's imageVolumeRenderingEnabled / segmentationSurfaceEnabled. This is required for FR-2: attachCoreSegmentationTableEvents (extensions/viewer3d-volume/src/utils/cornerstone3dSegmentations.js) interprets displaySet.segmentationSurfaceEnabled as panel visibility (setSegmentationsVisible), and OHIFVtkVolumeViewport clears the volume-viewer attributes on unmount. Sharing names would couple the editor toggles to unrelated behavior.
  • AR-2 — Same toggle pattern. Commands follow createViewportToggleFeatureCommand: flip the displaySet attribute, republish via addDisplaySets. The attributes are initialized (per FR-3 defaults) when the editor layout opens and cleared when it closes, mirroring the stableViewport lifecycle in OHIFVtkVolumeViewport.
  • AR-3 — Transitions in the layout, 3D tab only. The editor layout consumes the attributes as props (subscribed via DISPLAY_SET_CHANGED, as OHIFVtkVolumeViewport._evtDisplaySetUpdate does) and applies transitions in componentDidUpdate to the views3d tab viewport(s) only, following the transition logic of Cornerstone3DVolumeViewport.componentDidUpdate (§2). Surface transitions operate on the editor's 3D labelmap (_segVol3d()), not the 2D labelmap.
  • AR-4 — Reuse the rendering pipeline. No new rendering commands: VolumeRenderingMenuButton + the existing setViewportPreset / setVolumeRenderingQuality / shiftVolumeOpacityPoints / setVolumeLighting commands already resolve any registered viewport via getCornerstone3dViewport. Pass the editor's 3D viewport id.
  • AR-5 — Worker discipline. Disabling the surface cancels surface compute jobs (terminateWorkerComputeJobs(), as the volume viewer does). While the surface is disabled, skip c3dUpdateSurfaceData propagation from 2D edits (guard the event handlers in SegmentationEditorLayout.js); on re-enable, recompute the full surface — the single-flight wrapper (platform/core/src/utils/polySegSingleFlight.js) coalesces the recomputation. Use the epoch-guard pattern (_surfaceEpoch) so toggling off mid-computation aborts stale async paths.

6. Implementation plan by file

extensions/seg-editor

  1. src/toolbarModule.js — add the "3D" nested group: 
    {
  id: 'SegEditor3DOptions', label: '3D', icon: 'ellipse-circle',
  buttons: [
    { id: 'SegEditorVolumeRenderingEnabled', label: '3D Volume', icon: 'cube',
      CustomComponent: <wrapper over DisplaySetAttributeActiveToolbarButton, isActiveDisplaySetAttr: 'segEditorVolumeRenderingEnabled', isActiveDefault: false>,
      type: 'command', commandName: 'toggleSegEditorVolumeRendering' },
    { id: 'SegEditorSurfaceRenderingEnabled', label: 'Surface', icon: 'cube-3d-solid',
      CustomComponent: <wrapper, isActiveDisplaySetAttr: 'segEditorSurfaceRenderingEnabled', isActiveDefault: true>,
      type: 'command', commandName: 'toggleSegEditorSurfaceRendering' },
  ]
}
    Thin wrapper components follow ViewerImageRenderingEnabledToolbarButton.jsx / ViewerSegmentationSurfaceEnabledToolbarButton.jsx (extensions/viewer3d-volume/src/toolbarComponents/).
  2. src/commandsModule.js — add toggleSegEditorVolumeRendering and toggleSegEditorSurfaceRendering using the createViewportToggleFeatureCommand pattern from extensions/viewer3d-volume/src/commandsModule.js (duplicate the small helper locally or lift it to a shared utility — either is acceptable; keep the volume viewer untouched).
  3. src/utils/setSegmentationEditorLayout.js — when the editor layout opens, initialize segEditorVolumeRenderingEnabled = false and segEditorSurfaceRenderingEnabled = true on the displaySet and publish via addDisplaySets (satisfies the non-nil guard in the toggle command). In closeSegEditor (commandsModule), clear both attributes, mirroring OHIFVtkVolumeViewport.componentWillUnmount.
  4. src/ohifComponents/OHIFSegmentationEditorViewport.js / src/connectedComponents/ConnectedSegmentationEditorViewport.js — subscribe to DISPLAY_SET_CHANGED, pick the two attributes into state, and pass them as props to the layout (mirror OHIFVtkVolumeViewport._evtDisplaySetUpdate).
  5. src/components/Cornerstone3DSegmentationViewerLayout.js — accept segEditorVolumeRenderingEnabled / segEditorSurfaceRenderingEnabled props and implement the transitions in componentDidUpdate, scoped to the views3d tab viewport:
    • Volume on: viewport.setVolumes([{ volumeId: <image volumeId> }]) + viewport.setProperties({ preset: <modality default> }) (resolve via vtkUtils.volumeColorPresetUtils.getDefaultVolumePresetForModality or the defaultVolumeRenderPresetCT/MR prop convention) + triggerSegmentationUpdate() to restore surface actors. Volume off: c3dUtils.removeVolumeActors(viewportId) + render3d().
    • Surface on: create the Surface representation for _segVol3d() with color LUT (the layout's existing surface-activation path), epoch-guarded, with surfaceRendering/progress state driving the LoadingIndicator. Surface off: remove the Surface representation for _segVol3d() across the 3D viewport, increment the epoch, terminateWorkerComputeJobs().
    • Guard the c3dUpdateSurfaceData calls in the segmentationService event handlers (SegmentationEditorLayout.js) on the surface-enabled prop; on re-enable, trigger a full surface recompute.
  6. 3D tab render (createTab3dView / the 3D tab wrapper in Cornerstone3DSegmentationViewerLayout.js) — render VolumeRenderingMenuButton (from @ohif/extension-vtk) inside the 3D viewport wrapper: <div className="absolute bottom-2 left-2 z-10"><VolumeRenderingMenuButton viewportId={<3D tab viewportId>} /></div>, rendered only when segEditorVolumeRenderingEnabled.

7. Acceptance criteria

  • The editor menubar shows a "3D" submenu with "3D Volume" and "Surface" toggles whose active states reflect the current rendering state; defaults on open are Surface on, 3D Volume off.
  • Enabling 3D Volume renders the image volume in the 3D viewport with the modality default preset; existing surfaces remain visible after the transition.
  • Disabling 3D Volume removes the volume from the 3D scene; surfaces are unaffected.
  • Disabling Surface removes surface rendering and cancels in-flight surface computation; re-enabling recreates surfaces including segments edited while the surface was off, with the Loading/Rendering indicator shown during recomputation.
  • Neither toggle changes the 2D views, the sidebar panel, labelmap data, undo/redo history, or save behavior.
  • With 3D Volume enabled, the rendering menu button appears in the lower-left of the 3D viewport; Rendering Presets and Rendering Options changes (preset, quality, opacity shift, lighting/shade) apply immediately.
  • With 3D Volume disabled, the rendering menu button is not rendered.
  • Toggling rapidly (including mid-surface-computation) does not leave stale actors, duplicated worker jobs, or a stuck progress indicator (epoch guard + single-flight verified).
  • The volume viewer's toggles and attributes are untouched: opening the volume viewer after using the editor (and vice versa) behaves exactly as before.

8. Out of scope

  • Volume or surface rendering changes in the editor's 2D/MPR views.
  • New rendering options beyond what VolumeRenderingMenuButton already provides.
  • Persisting rendering toggle state or preset selections across sessions.
  • Sidebar panel changes.

9. References

Edited by Rob Oakes