klampt.vis package

Commonly used submodules

glinterface	Defines the GLPluginInterface class, which is a unified base class for plugins to the klampt.vis module.
glviewport
colorize	Colorize an object to show heatmaps, false color images, etc.
editors	Functions for visual editing.

Utility / infrequently used submodules

camera	This module defines a basic set of 3D cameras that can be controlled by a GUI.
gldraw	OpenGL drawing functions for geometric primitives.
glcommon	Defines GLWidgetPlugin, GLMultiViewportProgram, and CachedGLObject, which are used by the core visualization module.
glinit
glprogram	Internal classes for building 3D GUI programs.
glrobotprogram	Defines GLWorldPlugin and GLSimulationPlugin, which may be useful as base classes for your own plugins.
ipython	Defines a Jupyter Notebook interface to Klampt.
qtbackend
glutbackend

Module contents

Klamp’t visualization routines. See vistemplate.py in Klampt-examples for an example of how to run this module.

OVERVIEW

The visualization module lets you draw most Klamp’t objects in a 3D world using a simple interface. It also lets you customize the GUI using Qt widgets, OpenGL drawing, and keyboard/mouse intercept routines.

Main features include:

• Simple interface to modify the visualization

• Simple interface to animate and render trajectories

• Simple interface to edit certain Klamp’t objects (configurations, points, transforms)

• Simple interface to drawing text and text labels, and drawing plots

• Multi-window, multi-viewport support

• Automatic camera setup

• Unified interface to PyQt, GLUT, IPython, and HTML backends.

  • PyQT is the backend with the fullest amount of features.

  • GLUT loses resource editing and advanced windowing functionality.

  • IPython loses plugins, resource editing, custom drawing, and advanced windowing functionality.

  • HTML loses plugins, resource editing, custom drawing, and advanced windowing functionality.

The resource editing functionality in the klampt.io.resource module (which uses the widgets in klampt.vis.editors) use this module as well.

INSTRUCTIONS

Basic use of the vis module is fairly straightforward:

0. (optional) Configure the rendering backend.

1. Add things to the visualization scene with vis.add(name,thing). Worlds, geometries, points, transforms, trajectories, contact points, and more can be added in this manner.

2. Modify the appearance of things using modifier calls like vis.setColor(name,r,g,b,a).

3. Launch windows and/or visualization thread (OpenGL or IPython modes)

4. Continue adding, modifying, and removing things as you desire.

More advanced functions allow you to dynamically launch multiple windows, capture user input, and embed the visualization into Qt windows.

The default scene manager lets you set up and modify the visualization scene (Steps 1 and 2). These just mirror the methods in VisualizationScene, which is how the default scene manager is implemented. See Klampt-examples/Python/demos/vistemplate.py for more examples.

The vis module provides some basic methods for capturing user interaction. To add other user interaction functionality, you may create a GLPluginInterface subclass to add functionality on top of the default visualization world. To do so, call vis.pushPlugin(plugin). Note that custom rendering (the display() and display_screen() methods) are only available with the OpenGL rendering backend.

Only one rendering backend can be chosen during the lifetime of your process, and each backend has its own quirks with regards to window launching and configuration. We’ll describe the different options below.

OpenGL (PyQt, GLUT)

OpenGL-based visualizations use either PyQt or GLUT to handle windowing and are used by default. They run in the current process, have the best performance, and offer the richest set of features.

Quick start

• To show the visualization and quit when the user closes the window:

  vis.run()
  

• To show the visualization and return when the user closes the window:

  vis.dialog()
  ... do stuff afterwards ...
  vis.kill()   #cleanup, not strictly needed but good practice
  

• To show the visualization and run a script alongside it until the user closes the window (multithreaded mode):

  vis.show()
  while vis.shown():
      vis.lock()
      ... do stuff ...
      [to exit the loop call vis.show(False)]
      vis.unlock()
      time.sleep(dt)
  ... do stuff afterwards ...
  vis.kill()   #cleanup, not strictly needed but good practice
  

• To show the visualization and run python commands until the user closes the window (single-threaded mode):

  def callback():
      ... do stuff ...
      [to exit the loop manually call vis.show(False)]
  vis.loop(setup=vis.show,callback=callback)
  vis.kill()   #cleanup, not strictly needed but good practice
  

• To run a window with a custom plugin (GLPluginInterface) and terminate on closure:

  vis.run(plugin)
  

• To show a dialog or parallel window:

  vis.setPlugin(plugin)
  ... then call
  vis.dialog()
  ... or
  vis.show()
  ... do stuff afterwards ...
  vis.kill()
  

• To add a GLPluginInterface that just customizes a few things on top of the default visualization:

  vis.pushPlugin(plugin)
  vis.dialog()
  vis.popPlugin()
  

• To run plugins side-by-side in the same window:

  vis.setPlugin(plugin1)
  vis.addPlugin(plugin2)  #this creates a new split-screen
  vis.dialog()
  ... or
  vis.show()
  ... do stuff afterwards ...
  vis.kill()
  

• To run a custom Qt window or dialog containing a visualization window:

  vis.setPlugin([desired plugin or None for visualization])
  def makeMyUI(qtglwidget):
      return MyQtMainWindow(qtglwidget)
  vis.customUI(makeMyUI)
  vis.dialog()   #if you return a QDialog
  ... or
  vis.show()     #if you return a QWidget or QMainWindow
  ... do stuff afterwards ...
  vis.kill()
  

• To launch a second window after the first is closed: just call whatever you want again. Note: if show was previously called with a plugin and you wish to revert to the default visualization, you should call setPlugin(None) first to restore the default.

• To create a separate window with a given plugin:

  w1 = vis.createWindow("Window 1")  #w1=0
  show()
  w2 = vis.createWindow("Window 2")  #w2=1
  vis.setPlugin(plugin)
  vis.dialog()
  #to restore commands to the original window
  vis.setWindow(w1)
  while vis.shown():
      ...
  vis.kill()
  

Implementation Details

There are two primary modes of running OpenGL visualizations: multi-threaded and single-threaded.

• Multi-threaded mode pops up a window using show(), and the caller can then continue to interact with the vis module.

  IMPORTANT: multi-threaded mode is only supported on some systems (Linux, Windows using Qt). Due to weird OpenGL and Qt behavior in multi-threaded programs, if you are using multithreaded mode, you should only interact with OpenGL and the visualization using the methods in this module. Custom OpenGL calls can be implemented inside GLPluginInterface plugins and customDrawFunc.

• Single-threaded mode blocks the calling thread using loop(). To interact with the scene, the caller will provide callbacks that can modify the visualization world, pop up windows etc. Single-threaded mode is the most compatible, and is the only mode that works with GLUT and Mac OS.

There are also some convenience functions that will work in both modes, such as run(), spin(), and func:dialog.

Note

In multithreaded mode, when changing the data shown by the window (e.g., modifying the configurations of robots in a WorldModel) you must call vis.lock() before accessing the data and then call vis.unlock() afterwards.

The biggest drawback of single-threaded operation is that you can only start blocking dialogs at the outer-most level, not inside loop(). So if you have a GUI that’s running in real-time, in a multi-threaded visualization your code can pop up a dialog (like an editor) and the continue running with the returned value. There are some workarounds in single-thread mode (providing a callback to the dialog function) but these are not nearly as convenient.

It is possible to start in single-threaded mode and convert to multi-threaded, but the converse is not possible.

In OpenGL mode, you can also completely override the scene manager and run your own OpenGL calls using a subclass of GLPluginInterface. Here, you will need to perform all the necessary OpenGL drawing / interaction inside its hooks. To use this, you should call vis.setPlugin(plugin) to override the default visualization behavior before creating your window. See Klampt-examples/Python/demos/visplugin.py for an example of this in use.

IPython (Jupyter notebook)

IPython visualizations run in a Jupyter notebook in a web browser, using a WebGL widget to render the content. The Python code communicates with the browser upon certain calls to update the visualization.

This mode will be enabled by default inside a Jupyter notebook, or if you first call vis.init('IPython'). In the cell in which you want to show the WebGL widget, call vis.show(). To show multiple widgets you can create new vis windows, and use setWindow to switch between which widget you’d like subsequent calls to modify.

The WebGL widget is updated automatically upon addX, setColor, clear, and hide calls, but it can’t tell when something changes in the world, like a robot configuration or object transform. When the state of something in the world changes, you must manually make a call to vis.update().

Note

For optimal performance when calling a lot of scene modifiers, you should use vis.lock() / unlock() calls to block off the start and end of scene modification. Doing so means that the WebGL widget is only re-rendered at the very end of your calls.

Note the semantics here are slightly different from the normal sense of locking / unlocking, which is to prevent thread clashes amongst changes to the underlying object data.

This mode does NOT support plugins, dialogs, or custom draw functions. Also, certain types of geometries like VolumeGrids are not supported.

Animations are supported, but you will manually have to advance the animations and call vis.update() or vis.scene().update() for each frame. See Playback for a convenient widget that handles this somewhat automatically.

HTML

This output mode outputs an HTML + Javascript page that uses WebGL to display the scene. Unlike other methods, this is not a live visualization that interacts with your Python script, but rather you create a scene / animation and then retrieve the HTML afterwards. This mode is appropriate for Google Colab, for example.

To show the HTML page in an IPython notebook (e.g., Jupyter or Colab), call vis.show() after the animation is done. This will show the visualizer in an IFrame, whose size is the current viewport size.

To use it with a specified time step, use the following:

vis.add("world",world)
dt = 1.0/30.0   #30fps
while not done:
    ... change the world, add / remove things, etc ...
    vis.stepAnimation(dt)
vis.show()

To have it capture a real-time process, use the vis.update() function instead, as follows:

vis.add("world",world)
while not done:
    ... change the world, add / remove things, etc ...
    vis.update()
    time.sleep(1.0/3.0)
vis.show()

(Don’t mix animations and updates, or else the resulting animation will be strange.)

You may also retrieve raw HTML, either by casting vis.nativeWindow() to a str, calling vis.nativeWindow().iframe(width,height), or vis.nativeWindow().page(). In the first case, the HTML does not contain the <html> or <body> tags, and takes up the whole screen. The .iframe() option places the code into an IFrame of a given size, and the page() option wraps the code into a full HTML page.

For example, to turn a Trajectory traj into a WebGL animation, use this code:

vis.add("world",world)
vis.animate(("world",world.robot(0).getName()),traj)
t = 0
while t < traj.endTime():
    vis.stepAnimation(dt)
    t += dt
vis.show()

To turn a Simulator into a WebGL animation, use this code:

sim = Simulator(world)
vis.add("world",world)
while not done:
    ... add forces, send commands to the robot, etc...
    sim.simulate(dt)
    vis.stepAnimation(dt)
vis.show()

WINDOWING API

• debug(): a super easy way to visualize Klamp’t items.

• init(): initializes the visualization. Can configure here what backend(s) to use.

• createWindow(): creates a new visualization window and returns an integer identifier.

• setWindow(): sets the active window for all subsequent calls. ID 0 is the default visualization window.

• getWindow(): gets the active window ID.

• nativeWindow(): returns the current window object used by the backend.

• setWindowTitle(): sets the title of the visualization window.

• getWindowTitle(): returns the title of the visualization window.

• resizeWindow(): resizes the window. For OpenGL, this can be done after the window is shown. Otherwise, it must take place before showing.

• scene(): returns the current VisualizationScene

• setPlugin(): sets the current plugin (a GLPluginInterface instance). This plugin will now capture input from the visualization and can override any of the default behavior of the visualizer. Set plugin=None if you want to return to the default visualization.

• pushPlugin(): adds a new plugin (e.g., to capture input) on top of the old one.

• splitView(): adds a second scene / viewport to the current window. If a plugin is provided (a GLPluginInterface instance) then the new view is set to use this plugin.

• run(): pops up a dialog and then kills the program afterwards.

• kill(): kills all previously launched visualizations and terminates the visualization thread. Afterwards, you may not be able to start new windows. Call this to cleanly quit.

• multithreaded(): returns true if multithreading is available.

• loop(): Runs the visualization thread inline with the main thread. The setup() function is called at the start, the callback() function is run every time the event thread is idle, and the cleanup() function is called on termination.

  NOTE FOR MAC USERS: having the GUI in a separate thread is not supported on Mac, so the loop function must be used rather than show/spin.

  NOTE FOR GLUT USERS: this may only be run once.

• dialog(): pops up a dialog box (does not return to calling thread until closed).

• show(): shows/hides a visualization window. If not called from the visualization loop, a new visualization thread is run in parallel with the calling script.

• spin(): shows the visualization window for the desired amount of time before returning, or until the user closes the window.

• shown(): returns true if the window is shown.

• lock(): locks the visualization scene for editing. The visualization will be paused until unlock() is called.

• unlock() unlocks the visualization world. Must only be called once after every lock().

• update(): manually triggers a redraw of the current scene.

• threadCall(): Call a user-defined function inside the visualization thread. Advanced users may wish to inject calls that are incompatible with being run outside the Qt or OpenGL thread.

• customUI(): launches a user-defined UI window with the OpenGL window embedded into it (Qt only).

SCENE MODIFICATION API

The following methods operate on the current scene (as returned from scene()).

Objects accepted by add() include:

• text (str)

• Vector3 (3-list)

• Matrix3 (klampt.math.so3 item)

• RigidTransform (klampt.math.se3 item)

• Config (n-list): n must match the number of links of a RobotModel in “world”

• Configs (list of n-lists)

• Trajectory and its subclasses

• polylines (use Trajectory type)

• WorldModel

• RobotModel

• RobotModelLink

• RigidObjectModel

• TerrainModel

• Geometry3D

• GeometricPrimitive

• PointCloud

• TriangleMesh

• Simulator

• ContactPoint

• objects from the coordinates module.

See func:setAttribute for a list of attributes that can be used to customize an object’s appearance. Common attributes include color, hide_label, type, position (for text) and size (for points).

In OpenGL modes and IPython mode, many objects can be edited using the edit() function. In OpenGL, this will provide a visual editing widget, while in IPython this will pop up IPython widgets.

If you are modifying the internal data of an object in an external loop (as opposed to inside a plugin) be sure to call lock()/unlock() before/after doing so to prevent the visualization from accessing the object’s data .

Scene management functions

• add(): adds an item to the visualization. name is a unique identifier. If an item with the same name already exists, it will no longer be shown. Keyword attributes can be given to customize the appearance of the object (see setAttribute().)

• clear(): clears the visualization world.

• listItems(): prints out all names of visualization objects in the scene or under a given object

• getItemName(): retrieves the name / path of a given object in the scene, or returns None if the object doesnt exist.

• dirty(): marks the given item as dirty and recreates the OpenGL display lists. You may need to call this if you modify an item’s geometry, for example.

• remove(): removes an item from the visualization.

• setItemConfig(): sets the configuration of a named item.

• getItemConfig(): returns the configuration of a named item.

• hide(): hides/unhides an item. The item is not removed, it just becomes invisible.

• hideLabel(): hides/unhides an item’s text label.

• setLabel(): changes an item’s text label from its name to a custom string.

• setAppearance(): changes the Appearance of an item.

• revertAppearance(): restores the Appearance of an item

• setAttribute(): sets an attribute to change an item’s appearance.

• getAttribute(): gets an attribute of an item’s appearance.

• getAttributes(): gets all relevant attributes of an item’s appearance.

• setColor(): changes the color of an item.

• setDrawFunc(): sets a custom OpenGL drawing function for an item.

Animation functions

• animate() Starts an animation on an item. The animation be a Trajectory or a list of configurations. Works with points, so3 elements, se3 elements, rigid objects, or robots.

• pauseAnimation(): Turns animation on/off.

• stepAnimation(): Moves forward the animation time by the given amount, in seconds.

• animationTime(): Gets/sets the current animation time

• setTimeCallback(): sets a function that will return the global time

  used by the animation timing, visualization plots, and movie-saving functions. This must be monotonically non-decreasing.

Text and plots

Like other items in the visualization scene, text and plots are referred to by string identifiers.

Text is usually attached to 2D pixel coordinates, but in OpenGL mode can also be attached to 3D points. Use the position attribute to control where the text is located and the size attribute to control its size.

Plots are added to the scene and then items are added to the plot. The configuration of a visualization item is then shown as a live display (OpenGL). You may also log custom numeric data with logPlot() and event data using logPlotEvent().

• addText(): adds text to the visualizer.

• clearText(): clears all previously added text.

• addPlot(): creates a new empty plot.

• addPlotItem(): adds a visualization item to a plot.

• logPlot(): logs a custom visualization item to a plot

• logPlotEvent(): logs an event on the plot.

• hidePlotItem(): hides an item in the plot.

• setPlotDuration(): sets the plot duration.

• setPlotRange(): sets the y range of a plot.

• setPlotPosition(): sets the upper left position of the plot on the screen.

• setPlotSize(): sets the width and height of the plot.

• savePlot(): saves a plot to a CSV (extension .csv) or Trajectory (extension .traj) file.

User Interaction

• edit(): turns on/off visual editing of some item. Points, transforms, coordinates.Point, coordinates.Transform, coordinates.Frame, RobotModel, and RigidObjectModel are currently accepted.

• pick(): asks the user to click on an item in the scene.

• addAction(): adds a keyboard shortcut (GLUT) / menu items (Qt) / buttons (IPython).

• addButton(): adds a button to the GUI.

• addCheckbox(): adds a checkbox to the GUI.

• addSelect(): adds a selection dropdown or listbox to the GUI.

• addInput(): adds a string, integer, or float input box to the GUI.

• addSlider(): adds a slider to the GUI.

• addWidget(): adds a widget(s) to the GUI from a JSON description.

• setGUI(): Sets the GUI from a JSON description.

• bindWidget(): Binds GUI feedback to a callback function.

• removeWidget(): Removes a widget from the GUI.

• getWidgetState(): Gets the state of a widget from the GUI.

Global appearance / camera control functions

• getViewport(): Returns the GLViewport for the currently active view.

• setViewport(): Sets the GLViewport for the currently active scene. (This may also be used to resize windows.)

• setBackgroundColor(): Sets the background color for the active view.

• autoFitCamera(): Automatically fits the camera to all objects in the visualization. A scale > 1 magnifies the zoom.

• followCamera(): Sets the camera to follow a target.

• saveJsonConfig(): Saves the configuration to a JSON object or JSON file.

• loadJsonConfig(): Loads the configuration from a JSON object or JSON file.

• screenshot(): returns a screenshot of the scene. Can retrieve depth, as well.

• screenshotCallback(): sets a callback that will receive a screenshot of the scene after rendering is done.

Utility functions

• objectToVisType() Auto-determines a type of object compatible with the visualizer.

• autoFitViewport(): Automatically fits a GLViewport to see all the given objects.

NAMING CONVENTION

The world, if one exists, should be given the name ‘world’. Configurations and paths are drawn with reference to the first robot in the world.

All items that refer to a name (except add) can either be given a top level item name (a string) or a sub-item (a sequence of strings, given a path from the root to the leaf). For example, if you’ve added a RobotWorld under the name ‘world’ containing a robot called ‘myRobot’, then:

vis.setColor(('world','myRobot'),0,1,0)

will turn the robot green. If ‘link5’ is the robot’s 5th link, then:

vis.setColor(('world','myRobot','link5'),0,0,1)

will turn the 5th link blue.

A shortcut is to use the getItemName() function on the given robot, e.g.:

robot = world.robot(0)
vis.setColor(vis.getItemName(robot),0,0,1)

If there’s a Simulator instance added to the scene under the name ‘sim’ or ‘simulator’, the animation timing for Qt movie saving and HTML animations will follow the simulation time rather than wall clock time.

klampt.vis.visualization.init(backends=None)

Initializes the vis module using some visualization backend. backends can be None, in which case it tries using PyQt, then GLUT, then IPython in that order. It can also be a string or list of strings from the following set:

• ‘PyQt’: uses PyQT + OpenGL

• ‘PyQt4’ / ‘PyQt5’: uses a specific version of PyQT

• ‘GLUT’: uses GLUT + OpenGL

• ‘IPython’: uses an IPython widget

• ‘HTML’: outputs an HTML / Javascript widget

klampt.vis.visualization.debug(*args, **kwargs)

A super easy way to visualize Klamp’t items.

The argument list can be a list of Klamp’t items, and can also include strings or dicts. If a string precedes an item, then it will be labeled by the string. If a dict follows an item, the dict will specify attributes for the item. It can also contain the ‘animation’ key, in which case it should contain a Trajectory animating the item.

Keyword arguments may include:

• title: the window title

• animation: if only one item is given, sets a looping animation

• centerCamera: the name of the item that the camera should look at, or True to center on the whole scene.

• followCamera: the name of the item that the camera will follow if animated, or None (default).

• dialog: True if a dialog should be shown (default), False if a standard show() should be used.

• anything else: Treated as named klamp’t item.

klampt.vis.visualization.nativeWindow()

Returns the active window data used by the backend. The result will be a subclass of GLPluginProgram if OpenGL is used (PyQt or GLUT) or a KlamptWidget

klampt.vis.visualization.scene()

Returns the active scene used by the backend. The result will be a subclass of VisualizationScene. In some backends, there will be exactly one window per scene, but in others (split screen) there may be multiple windows per scene.

Return type:

VisualizationScene

klampt.vis.visualization.createWindow(title=None)

Creates a new window (and sets it active).

Return type:

int

Returns:

An identifier of the window (for use with setWindow()).

klampt.vis.visualization.setWindow(id)

Sets the currently active window.

Return type:

None

Note

ID 0 is the default visualization window.

klampt.vis.visualization.getWindow()

Retrieves ID of currently active window or -1 if no window is active

Return type:

int

klampt.vis.visualization.setPlugin(plugin)

Lets the user capture input via a glinterface.GLPluginInterface class. Set plugin to None to disable plugins and return to the standard visualization.

Return type:

None

Parameters:

plugin (GLPluginInterface) – a plugin that will hereafter capture input from the visualization and can override any of the default behavior of the visualizer. Can be set to None if you want to return to the default visualization.

klampt.vis.visualization.pushPlugin(plugin)

Adds a new plugin on top of the old one.

Return type:

None

Parameters:

plugin (GLPluginInterface) – a plugin that will optionally intercept GUI callbacks. Unhandled callbacks will be forwarded to the next plugin on the stack.

klampt.vis.visualization.popPlugin()

Reverses a prior pushPlugin() call

Return type:

None

klampt.vis.visualization.splitView(plugin=None)

Adds a second OpenGL viewport in the same window, governed by the given plugin.

Return type:

None

Parameters:

plugin (GLPluginInterface) – the plugin used for the second viewport. If None, the new viewport will have the default visualization plugin.

klampt.vis.visualization.addPlugin(plugin=None)

Adds a second OpenGL viewport in the same window, governed by the given plugin. DEPRECATED: use splitView() instead.

Return type:

None

Parameters:

plugin (GLPluginInterface) – the plugin used for the second viewport. If None, the new viewport will have the default visualization plugin.

klampt.vis.visualization.run(plugin=None)

A blocking call to start a single window and then kill the visualization once the user closes the window.

Return type:

None

Parameters:

plugin (GLPluginInterface, optional) – If given, the plugin used to handle all rendering and user input. If plugin is None, the default visualization is used.

Note

Works in both multi-threaded and single-threaded mode.

klampt.vis.visualization.multithreaded()

Returns true if the current GUI system allows multithreading. Useful for apps that will work cross-platform with Macs and systems with only GLUT.

Return type:

bool

klampt.vis.visualization.dialog()

A blocking call to start a single dialog window with the current plugin. It is closed by pressing OK or closing the window.

klampt.vis.visualization.setWindowTitle(title)

Return type:

None

klampt.vis.visualization.getWindowTitle()

Return type:

str

klampt.vis.visualization.resizeWindow(w, h)

Resizes the current window. For OpenGL, this can be done after the window is shown. Otherwise, it must take place before showing.

klampt.vis.visualization.kill()

This should be called at the end of the calling program to cleanly terminate the visualization thread

Return type:

None

klampt.vis.visualization.loop(setup=None, callback=None, cleanup=None)

Runs the visualization thread inline with the main thread. The setup() function is called at the start, the callback() function is run every time the event thread is idle, and the cleanup() function is called on termination.

NOTE FOR MAC USERS: a multithreaded GUI is not supported on Mac, so the loop() function must be used rather than “show and wait”.

NOTE FOR GLUT USERS: this may only be run once.

Return type:

None

klampt.vis.visualization.show(display=True)

Shows or hides the current window.

NOTE FOR MAC USERS: due to a lack of support of multithreading on Mac, this will not work outside of the setup / callback / cleanup functions given in a call to loop().

Return type:

None

klampt.vis.visualization.spin(duration)

Spin-shows a window for a certain duration or until the window is closed.

Return type:

None

klampt.vis.visualization.lock()

Begins a locked section. Needs to be called any time you modify a visualization item outside of the visualization thread. unlock() must be called to let the visualization thread proceed.

Return type:

None

klampt.vis.visualization.unlock()

Ends a locked section acquired by lock().

Return type:

None

klampt.vis.visualization.update()

Manually triggers a redraw of the current window.

Return type:

None

klampt.vis.visualization.shown()

Returns true if a visualization window is currently shown.

Return type:

bool

klampt.vis.visualization.customUI(func)

Tells the next created window/dialog to use a custom UI function. Only available in PyQT mode.

This is used to build custom editors and windows that are compatible with other UI functionality.

Return type:

None

Parameters:

func (function) – a 1-argument function that takes a configured Klamp’t QtWindow as its argument and returns a QDialog, QMainWindow, or QWidget.

(Could also be used with GLUT, but what would you do with a GLUTWindow?)

klampt.vis.visualization.threadCall(func)

Call func inside the visualization thread. This is useful for some odd calls that are incompatible with being run outside the Qt or OpenGL thread.

Possible use cases include performing extra OpenGL draws for camera simulation.

Return type:

None

klampt.vis.visualization.addAction(hook, short_text, key=None, description=None)

Adds a callback to the window that can be triggered by menu choice or keyboard. Alias for nativeWindow().addAction().

Return type:

None

Parameters:

• hook (function) – a python callback function, taking no arguments, called when the action is triggered.

• short_text (str) – the text shown in the menu bar.

• key (str, optional) – a shortcut keyboard command (e.g., can be ‘k’ or ‘Ctrl+k’).

• description (str, optional) – if provided, this is a tooltip that shows up when the user hovers their mouse over the menu item.

klampt.vis.visualization.clear()

Clears the visualization world.

Return type:

None

klampt.vis.visualization.add(name, item, keepAppearance=False, **kwargs)

Adds an item to the visualization.

Return type:

None

Parameters:

• name (str) – a unique identifier. If an item with the same name already exists, it will no longer be shown.

• keepAppearance (bool, optional) – if True, then if there was an item that had the same name, the prior item’s appearance will be kept.

• kwargs – key-value pairs to be added into the attributes dictionary. e.g. vis.add(“geom”,geometry,color=[1,0,0,1]) adds a geometry while setting its color to red.

klampt.vis.visualization.listItems(name=None, indent=0)

Return type:

None

klampt.vis.visualization.getItemName(object)

Retrieves the name / path of a given object in the scene, or returns None if the object doesnt exist.

Return type:

Union[str, Sequence[str]]

klampt.vis.visualization.dirty(item_name='all')

Marks the given item as dirty and recreates the OpenGL display lists. You may need to call this if you modify an item’s geometry, for example.

If things start disappearing from your world when you create a new window, you may need to call this too.

Return type:

None

klampt.vis.visualization.animate(name, animation, speed=1.0, endBehavior='loop')

Sends an animation to the named object. Works with points, so3 elements, se3 elements, rigid objects, or robots, and may work with other objects as well.

Return type:

None

Parameters:

• name (str) – the named object

• animation – may be a Trajectory or a list of configurations.

• speed (float, optional) – a modulator on the animation speed. If the animation is a list of milestones, it is by default run at 1 milestone per second.

• endBehavior (str, optional) – either ‘loop’ (animation repeats forever) or ‘halt’ (plays once).

klampt.vis.visualization.pauseAnimation(paused=True)

Pauses or unpauses the animation.

Return type:

None

klampt.vis.visualization.stepAnimation(amount)

Moves forward the animation time by amount, given in seconds

Return type:

None

klampt.vis.visualization.animationTime(newtime=None)

Gets/sets the current animation time

If newtime is None (default), this gets the animation time.

If newtime is not None, this sets a new animation time.

Return type:

None

klampt.vis.visualization.setTimeCallback(timefunc=None)

Sets a function that will return the window’s global time. This will be used by the animation timing, visualization plots, and movie-saving functions.

Return type:

None

Parameters:

timefunc (callable) – returns a monotonically non-decreasing float. If None, reverts back to using time.time().

klampt.vis.visualization.remove(name)

Removes an item from the visualization

Return type:

None

klampt.vis.visualization.getItemConfig(name)

Returns a configuration of an item from the visualization. Useful for interacting with edited objects.

Return type:

Sequence[float]

Returns:

A list of floats describing the item’s current configuration. Returns None if name doesnt refer to an object.

klampt.vis.visualization.setItemConfig(name, value)

Sets a configuration of an item from the visualization.

Return type:

None

Parameters:

• name (str) – the item to set the configuration.

• value (list of floats) – the item’s configuration. The number of items depends on the object’s type. See the klampt.model.config module for more information.

klampt.vis.visualization.setLabel(name, text)

Changes the label of an item in the visualization

Return type:

None

klampt.vis.visualization.hideLabel(name, hidden=True)

Hides or shows the label of an item in the visualization

Return type:

None

klampt.vis.visualization.hide(name, hidden=True)

Hides an item in the visualization.

Note: the opposite of hide() is not show(), it’s hide(False).

Return type:

None

klampt.vis.visualization.edit(name, doedit=True)

Turns on/off visual editing of some item.

In OpenGL mode, currently accepts items of type: :rtype: None

• Vector3 (3-list)

• Matrix3 (9-list)

• Config (n-list, where n is the # of robot links)

• RigidTransform (se3 object)

• RobotModel

• RigidObjectModel

• Point

• Transform

• Frame

In IPython mode, currently accepts items of type:

• Vector3 (3-lists)

• Config (n-list, where n is the # of robot links)

• RigidTransform (se3 objects)

• RobotModel

• RigidObjectModel

• Point

• Transform

• Frame

Returns:

None if doedit=False, otherwise, an editor object which depends on the backend. In OpenGL mode, returns one of the Widget subclasses. In Jupyter, returns a VBox containing controls.

klampt.vis.visualization.pick(click_callback, hover_callback=None, highlight_color=(1, 1, 0, 0.3), filter=None, tolerance=0.01)

Picks an item from the scene. click_callback is called once the user left-clicks on an item. hover_callback is called when a user hovers over an item.

Once the user clicks, callbacks are disabled. To re-enable callbacks, call pick() again.

Return type:

None

Parameters:

• click_callback (callable) – a function f(name,item,point) that receives the name of the item, the item itself, and the point clicked on, in world coordinates. Set this to None to disable picking.

• hover_callback (callable) – a function f(name,item,point) that receives the name of the item, the item itself, and the point hovered over, in world coordinates. Set this to None to disable picking.

• highlight_color (tuple, optional) – if given, hovered items will be colored with the given color.

• filter (callable, optional) – a function f(name,item) that returns True if the item should be included in the selection.

• tolerance (float, optional) – a collision tolerance for non-volumetric objects.

klampt.vis.visualization.addButton(text, callback=None, id=None, icon=None, iconformat=None, style=None)

Adds a button to the GUI.

Return type:

str

Parameters:

• text (str) – the label

• callback (callable) – a callback that is invoked when the button is clicked.

• id (str, optional) – if given, the id used for removing the widget and retrieving state

• icon (str) – Base64 encoded icon

• iconformat (str) – MIME type of encoded icon

• style (dict) – the HTML style of the button

klampt.vis.visualization.addCheckbox(text, callback=None, checked=False, id=None, icon=None, iconformat=None, style=None)

Adds a checkbox to the GUI.

Return type:

str

Parameters:

• text (str) – the label

• callback (callable) – a callback that is invoked when the button is clicked.

• checked (bool) – whether the checkbox should be checked.

• id (str, optional) – if given, the id used for removing the widget and retrieving state

• icon (str) – Base64 encoded icon

• iconformat (str) – MIME type of encoded icon

• style (dict) – the HTML style of the button

klampt.vis.visualization.addSelect(text, options, callback=None, selection=None, id=None, multiple=False, style=None)

Adds a selection widget to the GUI.

Return type:

str

Parameters:

• text (str) – the label displayed next to the selection

• options (list of str) – the list of options in the box.

• callback (callable) – a callback that is invoked when the selection is changed.

• selection (str or list of str) – the selection

• id (str, optional) – if given, the id used for removing the widget and retrieving state

• multiple (bool) – whether multiple items can be selected.

• style (dict) – the HTML style of the button

klampt.vis.visualization.addInput(text, value, callback=None, id=None, min=None, max=None, style=None)

Adds a text or numeric input to the GUI.

Return type:

str

Parameters:

• text (str) – the label displayed next to the item.

• value (str, int, or float) – the value to edit.

• callback (callable) – a callback that is invoked when the selection is changed.

• selection (str or list of str) – the selection

• id (str, optional) – if given, the id used for removing the widget and retrieving state

• min (int,float) – minimum value allowed

• max (int,float) – maximum value allowed

• style (dict) – the HTML style of the button

klampt.vis.visualization.addWidget(widget)

Adds a widget to the GUI

klampt.vis.visualization.setGUI(menu)

Replaces a whole GUI with the JSONGUI object.

Return type:

None

klampt.vis.visualization.bindWidget(id, callback)

Provides a callback function that is triggered when the named GUI widget is interacted with.

Return type:

None

klampt.vis.visualization.removeWidget(id)

Removes a widget by id.

Return type:

None

klampt.vis.visualization.getWidgetState(id)

Returns the widget state by id.

Return type:

Union[str, int, float, List[str]]

klampt.vis.visualization.setAppearance(name, appearance)

Changes the Appearance of an item, for an item that uses the Appearance item to draw (config, geometry, robots, rigid bodies).

Return type:

None

klampt.vis.visualization.setAttribute(name, attr, value)

Sets an attribute of an item’s appearance.

Return type:

None

Parameters:

• name (str) – the name of the item

• attr (str) – the name of the attribute (see below)

• value – the value (see below)

Accepted attributes are:

• ‘robot’: the index of the robot associated with this (default 0)

• ‘color’: the item’s color (r,g,b) or (r,g,b,a)

• ‘size’: the size of the plot, text, point, ContactPoint, or IKObjective

• ‘length’: the length of axes in RigidTransform, or normal in ContactPoint

• ‘width’: the width of axes and trajectory curves

• ‘duration’: the duration of a plot

• ‘pointSize’: for a trajectory, the size of points (default None, set to 0 to disable drawing points)

• ‘pointColor’: for a trajectory, the size of points (default None)

• ‘endeffectors’: for a RobotTrajectory, the list of end effectors to plot (default the last link).

• ‘maxConfigs’: for a Configs resource, the maximum number of drawn configurations (default 10)

• ‘fancy’: for RigidTransform objects, whether the axes are drawn with boxes or lines (default False)

• ‘type’: for ambiguous items, like a 3-item list when the robot has 3 links, specifies the type to be used. For example, ‘Config’ draws the item as a robot configuration, while ‘Vector3’ or ‘Point’ draws it as a point.

• ‘label’: a replacement label (str)

• ‘hide_label’: if True, the label will be hidden

klampt.vis.visualization.getAttribute(name, attr)

Gets an attribute of an item’s appearance. If not previously set by the user, the default value will be returned.

Parameters:

• name (str) – the name of the item

• attr (str) – the name of the attribute (see setAttribute())

klampt.vis.visualization.getAttributes(name)

Gets a dictionary of all relevant attributes of an item’s appearance. If not previously set by the user, default values will be returned.

Return type:

dict

Parameters:

name (str) – the name of the item

klampt.vis.visualization.revertAppearance(name)

Return type:

None

klampt.vis.visualization.setColor(name, r, g, b, a=1.0)

Return type:

None

klampt.vis.visualization.setDrawFunc(name, func)

Sets a custom OpenGL drawing function for an item.

Return type:

None

Parameters:

• name (str) – the name of the item

• func (function or None) – a one-argument function draw(data) that takes the item data as input. Set func to None to revert to default drawing.

klampt.vis.visualization.autoFitViewport(viewport, objects, zoom=True, rotate=True)

Return type:

None

klampt.vis.visualization.addText(name, text, position=None, **kwargs)

Adds text to the visualizer. You must give an identifier to all pieces of text, which will be used to access the text as any other vis object.

Return type:

None

Parameters:

• name (str) – the text’s unique identifier.

• text (str) – the string to be drawn

• pos (list, optional) – the position of the string. If pos=None, this is added to the on-screen “console” display. If pos has length 2, it is the (x,y) position of the upper left corner of the text on the screen. Negative units anchor the text to the right or bottom of the window. If pos has length 3, the text is drawn in the world coordinates.

• kwargs (optional) – optional keywords to give to setAppearance.

To customize the text appearance, you can set the ‘color’, ‘size’, and ‘position’ attributes, either through the keyword arguments, or using setAttribute(). To refer to this item, use the identifier given in name.

klampt.vis.visualization.clearText()

Clears all text in the visualization.

Return type:

None

klampt.vis.visualization.addPlot(name)

Creates a new empty plot with the identifier name.

Return type:

None

klampt.vis.visualization.addPlotItem(name, itemname)

Adds a scene item named itemname to the plot. All of the item’s configuration variables will be plotted by default, see hidePlotItem() to turn off drawing some of these channels.

Return type:

None

klampt.vis.visualization.logPlot(name, itemname, value)

Logs a custom visualization item to a plot. itemname can be an arbitrary identifier; future logPlot calls with this itemname will add values to the plotted curve.

Return type:

None

klampt.vis.visualization.logPlotEvent(name, eventname, color=None)

Logs an event on the plot.

klampt.vis.visualization.hidePlotItem(name, itemname, hidden=True)

Hides an item in the plot. To hide a particular channel of a given item pass a pair (itemname,channelindex).

For example, to hide configurations 0-5 of ‘robot’, call:

hidePlotItem('plot',('robot',0))
...
hidePlotItem('plot',('robot',5))

klampt.vis.visualization.setPlotDuration(name, time)

Sets the plot duration.

Return type:

None

klampt.vis.visualization.setPlotRange(name, vmin, vmax)

Sets the y range of a plot to [vmin,vmax].

Return type:

None

klampt.vis.visualization.setPlotPosition(name, x, y)

Sets the upper left position of the plot on the screen.

Return type:

None

klampt.vis.visualization.setPlotSize(name, w, h)

sets the width and height of the plot, in pixels.

Return type:

None

klampt.vis.visualization.savePlot(name, fn)

Saves a plot to a CSV (extension .csv) or Trajectory (extension .traj) file.

Return type:

None

klampt.vis.visualization.autoFitCamera(zoom=True, rotate=True, scale=1)

Automatically fits the camera to all items in the visualization.

Return type:

None

Parameters:

• zoom (bool, optional) – zooms the scene to the objects

• rotate (bool, optional) – rotates the scene to face the objects

• scale (float, optional) – a scale > 1 magnifies the camera zoom.

klampt.vis.visualization.followCamera(target, translate=True, rotate=False, center=False)

Sets the camera to follow a target. The camera starts from its current location and keeps the target in the same position on screen.

It can operate in the following modes: :rtype: None

• translation (translate=True, rotate=False): the camera moves with the object. This is default.

• look-at (translate=False, rotate=True): the camera stays in the current location but rotates to aim toward the object.

• follow (translate=True, rotate=True): the camera moves as though it were fixed to the object.

Parameters:

• target (str, SimRobotSensor, Trajectory, or None) – the target that is to be followed. If None, the camera stops following anything.

• translate (bool, optional) – whether the camera should follow using translation.

• rotate (bool, optional) – whether the camera should follow using rotation.

• center (bool, optional) – whether the camera should first aim toward the object before following. Default is False.

klampt.vis.visualization.getViewport()

Returns the GLViewport of the current scene

Return type:

GLViewport

klampt.vis.visualization.setViewport(viewport)

Sets the current scene to use a given GLViewport

Return type:

None

klampt.vis.visualization.setBackgroundColor(r, g, b, a=1)

Sets the background color of the current scene.

Return type:

None

klampt.vis.visualization.saveJsonConfig(fn=None)

Saves the visualization options to a JSON object or file.

If fn is provided, it’s saved to a file. Otherwise, it is returned.

Return type:

Optional[dict]

klampt.vis.visualization.loadJsonConfig(jsonobj_or_fn)

Loads the visualization options from a JSON object or file.

jsonobj_or_fn can either by a dict (previously obtained by saveJsonConfig or a str indicating a filename (previously saved using saveJsonConfig.)

klampt.vis.visualization.screenshot(format='auto', want_depth=False)

Returns a screenshot of the scene. Currently only available in OpenGL modes (PyQt, GLUT).

format describes the desired image format.

• ‘auto’: equivalent to ‘numpy’ if numpy is available, ‘Image’ if PIL is available, or ‘bytes’ otherwise.

• ‘numpy’: the image will be a numpy uint8 array of shape (h,w,3).

• ‘Image’: the image will be a PIL Image if Python Imaging Library is available.

• ‘bytes’: the image will be in the form (w,h,bytes) where bytes is a raw bytes array of length 4*w*h. The buffer follows OpenGL convention with the start in the lower-left corner.

Can also provide a depth image if want_depth=True. The format will a (w,h) numpy float array, a ‘F’ (float) Image, or (w,h,array) an array of floats.

In multithreaded mode, this will block until rendering is complete.

klampt.vis.visualization.screenshotCallback(fn, format='auto', want_depth=False)

Sets a callback fn that will receive a screenshot of the scene when rendering is done.

See screenshot() for a description of the format and want_depth arguments.

Currently only available in OpenGL modes (PyQt, GLUT).

To repeatedly receive screenshots, fn can call vis.screenshotCallback(fn) again.

Return type:

None

klampt.vis.visualization.objectToVisType(item, world)

Returns the default type for the given item in the current world

Return type:

str

class klampt.vis.visualization.VisPlotItem(itemname, linkitem)

Bases: object

customUpdate(item, t, v)

update(t)

discard(tstart)

updateTrace(i, t, v)

class klampt.vis.visualization.VisPlot

Bases: object

update(t, duration, compressThreshold)

discard(tmin)

addEvent(name, t, color=None)

autoRange()

renderGL(window, x, y, w, h, duration, vmin=None, vmax=None)

beginSave(fn)

endSave()

dumpAll()

dumpCurrent()

klampt.vis.visualization.drawTrajectory(traj, width, color, pointSize=None, pointColor=None)

Draws a trajectory of points or transforms.

By default draws points along the trajectory. To turn this off, set pointSize = 0.

klampt.vis.visualization.drawRobotTrajectory(traj, robot, ees, width=2, color=(1, 0.5, 0, 1), pointSize=None, pointColor=None)

Draws trajectories for the robot’s end effectors. Note: no additional discretization is performed, only the end effector points at the trajectory’s milestones are shown. If you want more accurate trajectories, first call traj.discretize(eps).

class klampt.vis.visualization.VisAppearance(item, name=None, type=None)

Bases: object

The core class that governs all of the drawing of an object in the visualization. Can accommodate any drawable Klampt type.

setItem(item)

markChanged(config=True, appearance=True)

destroy()

drawText(text, point)

Draws the given text at the given point

updateAnimation(t)

Updates the configuration, if it’s being animated

updateTime(t)

Updates in real time

swapDrawConfig()

Given self.drawConfig!=None, swaps out the item’s curren configuration with self.drawConfig. Used for animations

clearDisplayLists()

transparent()

Returns true if the item is entirely transparent, None if mixed transparency, and False otherwise

getAttributes()

drawGL(world=None, viewport=None, draw_transparent=None)

Draws the specified item in the specified world, with all the current modifications in attributes.

If a name or label are given, and self.attributes['hide_label'] != False, then the label is shown.

The drawing passes are controlled by draw_transparent – opaque items should be rasterized before transparent ones.

Parameters:

• world (WorldModel) – the world model

• viewport (Viewport) – the C++ viewport of the current view, which is compatible with the Klampt C++ Widget class.

• draw_transparent (bool or None) – If None, then everything is drawn. If True, then only transparent items are drawn. If False, then only opaque items are drawn. (This only affects WorldModels)

getBounds()

Returns a bounding box (bmin,bmax) or None if it can’t be found

getCenter()

getTransform()

rayCast(source, direction, tolerance=0.01)

Returns a point on the item intersected by the ray source + t*direction, or None if it does not intersect.

For non-volumetric items, the item is treated as though it were expanded by tolerance units.

getSubItem(path)

highlight(color)

Turn on/off a temporary highlight. color can be a (r,g,b,strength) tuple or None.

make_editor(world=None)

update_editor(item_to_editor=False)

remove_editor()

class klampt.vis.visualization.VisualizationScene

Bases: object

Holds all of the visualization information for a scene, including labels, edit status, and animations.

items

the top level items in the scene.

Type:

dict of str->VisAppearance

labels

internally used

Type:

list

t

current scene time, internally used

Type:

float

timeCallback

a function time()->float that is used to get the time update. If None, time.time() is used.

Type:

callable

startTime

first time that updateTime() was called, internally used.

Type:

float

doRefresh

whether any item wants to refresh the visualization.

Type:

bool

cameraController

if given, a class to update the camera whenever updateCamera is called.

getItem(item_name)

Returns an VisAppearance according to the given name or path

dirty(item_name='all')

Marks an item or everything as dirty, forcing a deep redraw.

clear()

Clears the visualization world

clearText()

Clears all text in the visualization.

listItems(root=None, indent=0)

Prints out all items in the visualization world.

getItemName(object)

Retrieves a str or tuple of strs giving the path to an object

add(name, item, keepAppearance=False, **kwargs)

Adds a named item to the visualization world. If the item already exists, the appearance information will be reinitialized if keepAppearance=False (default) or be kept if keepAppearance=True.

addText(name, text, **kwargs)

animate(name, animation, speed=1.0, endBehavior='loop')

pauseAnimation(paused=True)

stepAnimation(amount)

animationTime(newtime=None)

remove(name)

getItemConfig(name)

setItemConfig(name, value)

addLabel(text, point, color)

hideLabel(name, hidden=True)

hide(name, hidden=True)

addPlotItem(plotname, itemname)

logPlot(plotname, itemname, value)

logPlotEvent(plotname, eventname, color)

hidePlotItem(plotname, itemname, hidden=True)

savePlot(plotname, fn)

setAppearance(name, appearance)

setAttribute(name, attr, value)

getAttribute(name, attr)

getAttributes(name)

revertAppearance(name)

setColor(name, r, g, b, a=1.0)

setDrawFunc(name, func)

autoFitCamera(zoom=True, rotate=True, scale=1.0)

followCamera(target, translate, rotate, center)

setTimeCallback(cb)

Sets a callback in updateTime() to set the current time

updateTime(t=None)

The backend will call this during an idle loop to update the visualization time. This may also update animations if currently animating.

updateCamera()

Updates the camera, if controlled. The backend should call this whenever the scene is to be drawn.

edit(name, doedit=True)

pick(click_callback, hover_callback, highlight_color, filter, tolerance)

getViewport()

setViewport(viewport)

setBackgroundColor(r, g, b, a=1)

renderGL(view)

Renders the scene in OpenGL

renderScreenGL(view, window)

clearDisplayLists()

rayCast(x, y, filter=None, tolerance=0.01)

Performs ray casting with the scene

Parameters:

• x (float) – the screen x-coordinate of the mouse cursor

• y (float) – the screen y-coordinate of the mouse cursor

• filter (callable, optional) – if given, a function f(name,item) that returns True if the item should be considered.

• tolerance (float, optional) – a tolerance for ray intersection with non-volumetric items.

Returns:

a pair (visapp,pt) with visapp the first intersected VisAppearance and pt the clicked point. If nothing intersects, then (None,None) is returned.

Return type:

tuple

saveJsonConfig(fn=None)

loadJsonConfig(jsonobj_or_file)

All submodules

• klampt.vis.camera module
• klampt.vis.colorize module
• klampt.vis.editors module
• klampt.vis.glcommon module
• klampt.vis.gldraw module
• klampt.vis.glinit module
• klampt.vis.glinterface module
• klampt.vis.glprogram module
• klampt.vis.glrobotprogram module
• klampt.vis.glviewport module
• klampt.vis.ipython package
• klampt.vis.qtbackend module
• klampt.vis.glutbackend module