Only this pageAll pages
Powered by GitBook
Couldn't generate the PDF for 155 pages, generation stopped at 100.
Extend with 50 more pages.
1 of 100

Seahaven Documentation

Seahaven

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Interior Scene

Generates an interior scene using a given room type preset.

Room Type: The room layout to use for the generated scene. Options: (bedroom, livingroom). (See Room Type for more information on room types.)

Room Height: The ceiling height of the generated room (in meters).

Wall type: The material to use for the walls in the room. Options: (Gypsum, Wallpaper).

Floor type: The material to use for the floors in the room. Options: (Oak, Maple, Tile, Ceramic).

Ceiling type: The material to use for the ceilings in the room. Options: (Wood, Gypsum, ACT).

Window type: The window shape to use. The "Tall" preset is approx. 2.3m tall. Options: (Standard, Tall).

Generate recessed lights: If checked, recessed lights will be generated in the ceiling of the room.

Color temp: Color temperature of the lights, in degrees Kelvin. See Wikipedia for more information.

Light strength multiplier: A factor by which to multiply the strength of the recessed lights. Setting both the min and max to 0 will disable the lights.

Usage of the "Generate interior scene" block

Getting Started

Set Up Your Account

Seahaven is a platform that enables you to create synthetic data to train computer vision applications.

Our platform enables data scientists and AI engineers to create simulations of real-world places and events. With Lexset you can configure and run simulations that generate training data for your algorithms.

Once your account has been activated your account you will receive a link to your account page. Click the "login" button in the upper right corner.

Next, you will be directed to a login page. You can set a username and password or use your Gmail account.

After you have accessed your account you can go to your account page where you can access your user ID, bearer token, and other account information that will be necessary for you to use the Lexset APIs.

Note: We have blurred out all personal information for this tutorial. Please use your own account information.

Add a payment method

Seahaven Documentation

Add Camera Curve to a scene

If you want to precisely position a camera in your scene you can load a scene with a curve object to define a specific range of positions for your camera.

  • For example, in this simulation, the user wanted the camera to follow the perimeter of a building and they drew a curve to represent the range of positions for the camera to take.

  • You can name the curve whatever you like. However, be sure to be consistent to assign the name in your camera module later.

Example curve in yellow

Import 3D Assets

The 3D Assets tab is where you will bring 3D assets to use in your simulation into Seahaven. Here, assets are separated into folders called collections.

Collections are an important part of Seahaven. Collections not only contain groupings of 3D assets but the names will be used to manage annotations and other aspects of your simulations. Please name your collections carefully.

  • First, designate a Collection name on the top left-hand side of the upload tool. Each collection is comprised of the files you want to assign to a specific category.

Creating the collection name
  • Drag and drop your .blend files into the upload tool and select upload when they have finished processing.

  • This Collection should now be available for placement in the Relationship Editor

Populating and uploading a collection

Name Objects

You can assign a name to objects in the outliner. These names are typically used to tell manipulators what objects in a scene to transform or manipulate in some defined way.

  • You can assign an object a name in the blender outliner.

  • If you have multiple objects with the same name you can use a "." as a delineator and assign an index number. Seahaven will only look at the prefix if you try to modify these objects later with a manipulator.

    • This is a great way to manipulate large groups of objects.

    • In this example, the user is using a Blender empty to define a possible location to place a person.

Prepare Your Models

Seahaven uses Blender (.blend) files for the 3D asset population, and there are a few steps of preparation to ensure your model will work effectively in your simulation.

Blender is a free download available through the Blender Foundation.

  • Remove everything from the scene except the 3D asset and any materials you will be packing into the file.

Camera and light source have been removed from this blend scene
  • Purge any orphan data from the scene. This unused data can interfere with cameras, lighting, and many other parts of the simulation.

  • To purge orphan data, navigate to display mode, select orphan data. Then select purge.

The orphan data being purged from a blend scene
  • Last, make sure to pack all your data into the blend file.

  • Go to File, then External Data, then Pack All Into .blend

Workflow

The workflow is the container where you will be defining all object relationships.

Child Object

Child objects are arranged in the scene relative to a parent object. It has similar inputs to a parent object with the addition of a series of placement rules which define the specific series of operations to perform in order to place the child object in the scene, typically relative to the parent object.

  • Category Override: An optional override for the category of objects directly associated to this parent object. If no override is provided, the name of the collection is used for the category (only if a collection is used as the source).

  • Tries: The number of attempts to make when placing this object using the provided placement rules before failing and proceeding to the next object in the tree.

  • Chance: The percent chance that the system will attempt to place this object. If a value of less than 100% is provided, there is a chance that this object (and all downstream child objects in the same block) will be skipped.

  • Source: The source of this object, which may be an existing asset in the scene, a loaded collection, or a generated asset.

  • Placement Rules: A sequence of rules to execute when placing this object, typically relative to the direct parent of this object.

  • Children: All child objects associated to this parent object.

Relationship Editor

In the Relationship Editor tab you will designate the relationships of your 3D assets to each other and elements in the scene (e.g. ground_plane). Object relationships are defined using a hierarchical structure of parent and child objects. A Workflow may have multiple parent objects, each of which may have multiple child objects.

Get Object by Properties

Gets an object currently in the scene provided a category and/or object name.

  • Category: Category of the target object.

  • Object Name: Name of the target object.

  • Partial Match: If checked, partial matches to the provided strings will be accepted when retrieving objects from the scene.

Asset Sources

Assets may be sourced from existing objects already in the scene, generated directly, or be loaded from a collection in your account.

Parent Object

A Parent Object is a top-level object in your workflow.

  • Category Override: An optional override for the category of objects directly associated to this parent object. If no override is provided, the name of the collection is used for the category (only if a collection is used as the source).

  • Source: The source of this object, which may be an existing asset in the scene, a loaded collection, or a generated asset.

  • Children: All child objects associated to this parent object.

Collection (Array)

A collection of assets arranged in a grid-like array.

  • Collection: The specific collection to sample for assets.

  • X/Y Count: Number of assets to load from this collection in each direction, provided by an integer value or integer range.

  • X/Y Spacing: The distance between each asset in the array in each direction, provided by a number variable.

  • Spacing Method: Defines how to determine the space between each asset.

    • "Object bounds" sets the spacing of the assets relative to their bounding boxes. If the array consists of multiple assets, the bounds are defined by the maximum value from all sampled assets.

    Array spacing using the "Object bounds" method
    • "Grid lines" places the objects in a grid where the grid lines are spaced using the provided values for X/Y spacing. For example, if the X/Y spacing is set to 1 meter, objects will be placed in a grid where the grid lines are 1 meter apart in the X and Y directions. This method should be used only in cases where assets will have sufficient space between one another and where there is relatively low variation in the size of assets in this collection.

Array spacing using the "Grid lines" method
  • Use Largest Dimension: If checked, the largest dimension of an object is used to determine its bounds. For example, if an object is 1x4 meters, the grid spacing will be 4x4 meters. This is useful if you intend to rotate individual objects in the array.

  • Use Single Asset: If checked, the same asset will be instanced for each array element.

  • Separate Array Elements: If checked, placement operations for all downstream child objects will be executed for each element in the array.

  • Remove Invalid Array Elements: If checked, invalid elements (typically caused by object collisions) will be removed after all placement rules have been executed. This allows for placement of the entire array to be successful even if individual elements would otherwise cause it to fail.


In practice, an array is treated as any other object in your simulation. Many placement rules have an "Execute per Array Element" option. If this is enabled, the corresponding placement rule will be executed on a per-element basis. In the below example, a 3x3 array of a single screw asset is placed on a ground plane using the Put Object rule and is subsequently rotated with Rotate Object. When "Execute per Array Element" is disabled, the entire array of objects is rotated together. When enabled, each screw is rotated in-place.

"Execute per Array Element" disabled
"Execute per Array Element" enabled

Placement Rules

Placement rules are used to transform child objects. Transformations are done in multiple ways:

  • In global (scene) coordinate space (e.g. translation)

  • In local (object) space (e.g. rotation)

  • Relative to a parent object (e.g. alignment, orientation, etc.)

The following workflow constructs a basic scene consisting of a ground plane and screws:

  • A ground plane (0.5m x 0.5m) is generated using a Ground Plane generator

  • Multiple screws are loaded from a collection

  • The screws are organized using a series of placement rules:

    • Put Object: Places the screws on the ground plane between the UV coordinates (0.2, 0.2) and (0.8, 0.8)

    • Rotate Object: Rotates each object about its local Z axis between 0 and 360 degrees

Collection

A collection of assets used as a source for an object.

  • Collection: The specific collection to sample for assets.

  • Samples: Number of assets to load from this collection, provided by an integer value or integer range.

Generator: Ground Plane

Generates a ground plane object.

  • Size: Size of the ground plane in meters

  • Shadow Catcher: If checked, objects will cast a shadow on the ground plane, but the object itself will otherwise not be visible.

Locate Object (Global)

Sets the location of an object in global coordinate space.

  • Position: Sets the position of the object using an XYZ Range.


Basic Transformations

Basic transformations consist of typical affine transformations such as translation, rotation, and scaling.

Locate Object (Local)

Sets the location of an object in local space, relative to a face on the parent object's bounding box.

  • Position: Sets the position of the object using a UVW Range.

  • Face: The face(s) to reference on the parent object when positioning the object in local space.


Translate Object

Translates an object relative to its current position.

  • Translation (XYZ): Determines the translation vector using an XYZ Range.

  • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

Complex Transformations

Complex transformations may consist of several basic transformations and typically reference a parent and/or child bounding box face.

Rotate Object

Rotates an object about its local axes.

  • Rotation (Degrees): Amount by which to rotate the object per axis in degrees. Provided using an XYZ Range.

  • Step Size (Optional): Locks the rotation amount to a given step size. For example, a step size of 90 will limit rotation values to 0, 90, 180, and 270. This value is typically an integer, however float values may also be used.

  • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

Scale Object (Non-Uniform)

Scales an object non-uniformly with a different value per axis.

  • Scale: Factor by which to scale the object along each axis, as provided by an XYZ Range.

Attach Object

Attaches one object to another provided a face for each.

  • Parent Face: Reference face on the parent object.

  • Child Face: Reference face on the child object.

  • Position: Relative position of the child object on the face of the parent object provided by a UV Range.

  • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

  • Use Parent Above: If enabled, this placement rule will instead reference the parent object above this object's direct parent in the tree. For example, if the hierarchy of objects is Table > Plate > Food where "Food" is the child object, enabling "Use Parent Above" in this case would use "Table" as the parent object instead of "Plate".


Scale Object (Uniform)

Scales an object uniformly (equally along all axes).

  • Scale Factor: Factor by which to scale the object, provided by a Number. A value of 1 corresponds to the object's original scale.

Put Object

"Puts" an object on a parent object using the top face of the parent object and the bottom face of the child object.

  • Position: The relative position of the object on the top face of the parent object provided by a UV Range.

  • Use Parent Above: If enabled, this placement rule will instead reference the parent object above this object's direct parent in the tree. For example, if the hierarchy of objects is Table > Plate > Food where "Food" is the child object, enabling "Use Parent Above" in this case would use "Table" as the parent object instead of "Plate".


In this example, an array of screws is placed on the center of a ground plane using the "Put Object" placement rule.

Move Object Nearby

Moves a child object horizontally (within the global XY plane) to be within a specified distance range relative to the parent object.

  • Minimum/Maximum Distance (m): Distance from the bounds of the parent object provided by a Number.

A region representing the provided distances is created by inscribing the bounds of the object with an ellipse. A distance of 0 corresponds to no offset from the original object bounds (shown as "minimum" below). Distances above 0 offset the object bounds by that specified distance. The outer region is defined by inscribing the offset bounds with a new ellipse. By using an ellipse, this prevents the corners of the rectangular region from being sampled, which would otherwise sample distances greater than intended. Furthermore, using a circular region relative to the object center would not account for objects with a footprint of uneven proportions (e.g. a vehicle).


In this example, between 2 and 4 screws are placed on a ground plane using the "Put Object" placement rule. For each object, an additional screw is aligned to it using the "Align Objects" placement rule. The same screw is then moved to be near another screw with a minimum distance of 0.05 meters and a maximum distance of 0.1 meters.

Variables

Variables are used as generic inputs primarily for placement rules in your object relationships.

Orient Object to Surface

Re-orients an object to align to the nearest surface. This is typically used when placing objects on an uneven surface.

  • Child Face: Reference face on the child object.

  • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.


In this example, multiple cars are placed on a ground object currently in the scene. The "Put Object" placement rule performs the initial placement of the car on the ground. It is then rotated about its Z axis by a random amount, and finally oriented to the surface using its bottom face.

Neighbor

Places an object near another, relative to a bounding box face on each.

  • Parent Face: Reference face on the parent object.

  • Child Face: Reference face on the child object.

  • Offset Distance: Distance to offset from the parent face when placing the child object provided by a Number.

  • Horizontal Position: Horizontal position along the parent face to use when placing the child object. A value of 0 corresponds to the left edge, while a value of 1 corresponds to the right edge. Provided by a Number.


Align Objects

Aligns a child object to a parent object, given a bounding box face for each. The objects to not necessarily have to intersect with one another. The parent face is treated as an infinite plane when aligning the child object.

  • Parent Face: Reference face on the parent object.

  • Child Face: Reference face on the child object.

  • Use Nearest Parent/Child Face: If enabled, the most appropriate face will be automatically determined by proximity.


In this example, between 2 and 4 screws are placed on a ground plane using the "Put Object" placement rule. For each object, an additional screw is placed on the ground plane using the same method and referencing the parent object one level above. The objects are then aligned to one another using the right face of the parent object and the left face of the child object.

Drop Object

"Drops" an object onto a parent object using a vertical raycast to determine the point of intersection. If there is no intersection (i.e. the child object is not located directly above a surface), the rule will fail.

  • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

  • Use Parent Above: If enabled, this placement rule will instead reference the parent object above this object's direct parent in the tree. For example, if the hierarchy of objects is Table > Plate > Food where "Food" is the child object, enabling "Use Parent Above" in this case would use "Table" as the parent object instead of "Plate".


The following workflow uses the "drop object" rule to place screws on a ground plane:

  • A ground plane (0.5m x 0.5m) is generated using a Ground Plane generator

  • Multiple screws are loaded from a collection

  • The screws are organized using a series of placement rules:

    • Locate Object (Global): Moves the screws to positions between (-0.5, -0.5) and (0.5, 0.5) at a height of 1 meter.

    • Drop Object: Drops the screws onto the ground plane (parent object).

    • Rotate Object: Rotates the screws about their local Z axis.

Face

Defines a face of a parent object to sample.

If multiple faces are used as a single input, the selected face will be chosen at random.

A placement rule using multiple options for the "Child Face" input

Each option indicates a locally-aligned bounding box face of a given object from the following options:

  • Right, Left, Front, Back, Top, Bottom

Additionally, there is an "Any" option which samples from (Right, Left, Front, Back) at random.

The following sample uses the "Attach" rule to place a child object relative to a parent object where the face on the parent object is the "Top" and the face on the child object is the "Bottom".

Integer Range

Samples a range of integers from a min and max (inclusive) value. For example, if the range is between 4 and 6, the value can be either 4, 5, or 6.

UV Range

Defines a range of coordinates in local 2D space. This is typically associated with the face of a bounding box.

  • U (min/max): Position along the horizontal (U) axis.

  • V (min/max): Position along the vertical (V) axis.


Sample implementation of a UV Range block

Numbers

Numerical inputs are used across many placement rules and associated blocks. Typically, any of these blocks (integer/float) may be used, unless the input is defining a count, in which case an integer block must be used.

Vectors

Various 2D and 3D vectors are used to define positions in the scene, either relative to the scene coordinates (XYZ) or relative to another object/face (UVW).

Float Range

Uniformly samples a floating-point number defined by a minimum and maximum.

UVW Range

Defines a range of coordinates in local 3D space. This is typically associated with the face of a bounding box.

  • U (min/max): Position along the horizontal (U) axis.

  • V (min/max): Position along the vertical (V) axis.

  • W (min/max): Position along the normal (W) axis. Unlike the U and V components, the W component is expressed in meters. For example, the point (0.5, 0.5, 1) in UVW space as implemented in this system corresponds to a point aligned to the center of a bounding box face (0.5, 0.5), offset by 1 meter along the face's normal (away from the face).


Sample implementation of a UVW Range block

Simulation Editor

Integer Value

Provides a single integer value.

Float Value

Provides a single floating point value.

XYZ Range

Defines a range of coordinates/values in object or scene (XYZ) space.


XYZ Ranges are used to sample a range of values per axis (X, Y, Z). The specific implementation of these values varies per placement rule. Different implementations are illustrated below:

  • Locate: Used to determine the position of the object in global space.

  • Translate: Translates the object using a vector defined by an XYZ range.

  • Rotate: Rotates the object about each of its local axes by a specified number of degrees.

  • Scale: Scales the object about each of its local axes by a factor.

Scene

Scene objects are used to define context. Including lighting, and procedurally or parametrically generated objects.

HDRI Lighting

High Dynamic Range Images or HDRI's are large images that project ambient light into your scene. the HDRI Lighting module allows the user to select from a drop-down menu of environment types.

With the HDRI lighting module, you load a variety of pre-loaded HDRI images that you can use as a background in your images. In addition, these images provide physically accurate environmental lighting. Project to ground: You can project your HDRI images to the scenes ground plane. Resulting in a semi-spherical projection. You can learn more about how this feature can be used when creating photorealistic images on our blog. Height: Height refers to the height of the camera when the HDRI was captured. Most HDRI images are taken between 2 and 3 meters off the ground. You can adjust this parameter to reduce distortion around the horizon, which might appear if your camera moves too high above eye level in the z axes.

Intensity: Set a minimum and maximum light intensity. The range will dictate the range of lighting brightness across all the images in your dataset.

Saturation: Sets a range for the color saturation of the HDRI background. This is a normalized value where 0 yields a greyscale background and 1 yields a full-color background.

3D Asset

Import 3D Assets into your simulation

The number of samples determines how many unique items to sample out of the collection.

For example, if we have a collection with four different 3D models of fruit (apple, orange, pear, pineapple), if you set the samples to 2, it will randomly select two from that collection. If "Load Duplicate" is enabled, the same asset may be loaded more than once.

To select a collection, use a "Collection" block from the "Variables" submenu.

Relationships

Import 3D Assets & Relationships: This module is to import relationships between collections that have been defined in the relationships editor.

Ground Plane

This module generated a ground plane at the origin of the scene.

Shadow Plane: If "Shadow Plane" is enabled objects will cast a shadow on the ground plane.

Physical Sun/Sky

Adds a procedural sun and sky to the environment.

Sun Strength: Relative strength of the sun's illumination. It is recommended to use a low value (e.g. 0.1) to avoid over-exposure.

Sun Rotation: Rotation of the sun in the sky relative to the world origin where 0 degrees locates the sun at the positive Y world axis.

Sun Elevation: Elevation of the sun (in degrees) relative to the horizon. Sun/Sky Model: Nishita (Nishita et al. 1993.), Hosek-Wilkie (Hosek-Wilkie et al. 2012.), or Hosek-Wilkie (No-Shadow). If the "no-shadow" option is selected, shadow rays from the sun lamp will be disabled.

Room Type

Defines a room type, which uses accompanying object relationships to populate the space.

Room type: Choose from a list of room layout presets including "bedroom" and "living room" (more coming soon).

Object relationships: Object relationships created from the relationship editor which will be used to populate rooms of this type.

Usage of the "Generate interior scene" block, with accompanying Room Type and Relationships blocks.

Load Relationships

The Load Relationships module allows you to select a relationship file from a drop down menu. If you created a relationship in the Relationship Editor tab, it will appear here.

Curve-Based Camera

Randomly samples a camera pose on a curve.

Frames: Number of frames to sample along the curve.

Curve Name: Name of the curve to sample.

Yaw/Pitch/Roll (Min/Max): Rotation ranges for the camera about its 3 axes. Setting all values to 0 will point the camera straight down. Setting the pitch to 90 degrees will make the camera parallel with the XY/ground plane.

Camera Height (Min/Max): Height of the camera above the curve in meters.

Properties: Perspective camera intrinsic properties. Additional documentation.

Workflow

The workflow is the container you will use to define the parameters of your simulation.

  • Scene: Scene modules are used to configure your scene context. This will include importing/generating 3D environments, backgrounds, and/or lighting.

  • Relationships: Relationships between classes of objects defined in the relationship editor are imported into your workflow here.

  • Camera: Camera type, position, and camera properties are all defined in this section.

  • Manipulators: Manipulators can be used to transform and randomize your scene each time a data point is generated.

  • Render Passes: Individual render passes are defined here (RGB, depth, normal, etc.).

  • Output Files: Non-image output is defined here (COCO annotations, camera matrices, etc.).

  • Raytrace Samples: Number of paths to trace for each pixel in the RGB images. Increasing the number of samples improves render quality, but takes more time.

  • Image Format: Image file format (PNG, JPEG, TIFF).

  • Color Depth: Image bit depth (8- and 16-bit).

  • Color Mode: Image color mode (RGB, Panchromatic).

  • JPEG Quality: JPEG image quality (only applies to JPEG image format).

  • Use experimental raytracer: Enables experimental raytracer features (not applicable in most simulations).

  • Use dynamic BVH tree: If checked, the BVH tree will be built dynamically (per object). This method may be more time-efficient for animated sequences.

Room-Perimeter Camera

Positions a camera on the perimeter of a room generated using the "generate interior scene" block.

Frames: Number of frames to sample in the room.

Position: The location within the room to sample. Options: (Wall, Corner).

Offset Distance: The distance from the camera to the wall (in meters).

Camera Height: Camera height above the floor (in meters).

Yaw/Pitch/Roll: Random rotation along the path (in degrees).

Required Categories: Categories which are required to be visible in each image. Additional documentation.

Properties: Fisheye camera intrinsic properties. Additional documentation.

Minimum Distance to Camera: Restricts objects from being closer than this distance to the camera.

Single Raycast Distance Check: If checked, distance will only be checked from the center of the frame.

Category

Category: Name of the category to check in each image.

Min/Max: The minimum and maximum number of instance allowed of the category in each image.

Camera Configuration

Camera Configuration is a module that is loaded into the Camera bay and allows placement of Position and Properties modules

  • Seahaven currently supports two different camera types (Perspectival and fisheye)

  • Each of these camera types can either be positioned by a curve or by using one of the position modules in Seahaven.

  • Curves can be imported with the 3D Asset module in the scene menu. For more information, please see the prepare your models section of this documentation.

  • Additionally, rotation ranges may be defined for each curve-based camera.

Frames: The number of unique frames to create in the current scene. Each frame represents a unique data point.

Curve name: The name of the curve with which the camera location will be determined.

Yaw (min/max): A range of values, determining the rotation of the camera about its up axis (looking left and right).

Pitch (min/max): A range of values, determining the rotation of the camera about its right axis (looking up and down).

Roll (min/max): A range of values, determining the rotation of the camera about its forward axis (rotating clockwise/counter-clockwise).

Camera Height (min/max): The height of the camera relative to the curve (in meters).

Animated Track Camera

Animates/tracks a camera along a curve.

Frames: Number of frames to sample along the curve.

Curve Name: Name of the curve the camera will follow.

Properties: Perspective camera intrinsic properties. Additional documentation.

Shutter Speed: Shutter speed of the camera in seconds.

Camera Height (Min/Max): Height of the camera above the curve in meters.

Curve Parameter Start (Min/Max): A value between 0 and 1 indicating the start position of the camera where 0 is the start of the curve and 1 is the end of the curve.

Curve Parameter Length (Min/Max): A value between 0 and 1 indicating the distance to travel along the curve where 0 is the start of the curve and 1 is the end of the curve.

Additional Cameras

Allows for additional camera positions to be sampled per frame.

Note: All values are expressed relative to an assumed origin and orientation. In other words, it can be treated as adding cameras to a device (such as a head mounted display), which informs the initial location and orientation.

  • Offset: Distance to offset along each axis (in meters) for this specific camera.

  • Rotation: Additional rotation to apply to the camera (in degrees).

  • Filename Suffix: Suffix to append to each image rendered by this camera. For example, using a suffix of "_L_UP" would change a filename from "rgb_0001.png" to "rgb_0001_L_UP.png"

Room-Based Camera

Positions a camera in a room generated using the "generate interior scene" block.

  • Frames: Number of frames to sample in the room. Frames are generated along a random path within the room.

  • Control Points: Number of points which are used to create the camera path. Higher numbers increase the complexity of the path.

  • Distance Between Points: The min/max distance between consecutive points in the camera path.

  • Camera Height: Camera height above the floor (in meters).

  • Yaw/Pitch/Roll: Random rotation along the path (in degrees).

  • Additional Cameras: Optional list of additional camera positions/orientations to sample per frame. See documentation for additional information.

  • Properties: Fisheye camera intrinsic properties. Additional documentation.

Camera

Define your camera and how it is positioned within your simulated environment.

Cubic

This module allows you to choose points in a 3D bounding box for the camera to sample from

Look at: Define where your camera points using one of the blocks from the orientation menu.

Position Camera

Cameras are positioned within ranges of values. Camera position is defined as a bounding volume or point cloud within which you will sample locations. You can choose from the following options.

Conical

This module moves the camera along a cone

Look at: Define where your camera points using one of the blocks from the orientation menu.

Calculate Area of Interest

This block will automatically point the camera towards the items in the scene.

The bounding box of all the items in the scene is calculated and the camera will point towards the center of that bounding box.

Point

This block will point the camera towards a specific location within the scene.

Define a target point to point your camera at.

Perspectival Camera Properties

Camera intrinsic properties for a perspectival camera.

FOV (Min/Max): Sets the horizontal field of view (in radians).

F-Stop: The F-Stop (aperture) of the lens. Affects the depth-of-field of the image.

Aperture Blades: Number of blades for the aperture. Affects the shape of the bokeh.

Aperture Ratio: Simulates distortion of bokeh. Values below 1 stretch the shape of the bokeh horizontally while values above 1 will stretch the bokeh vertically.

Camera Near/Far Clip: Sets the distance of the clipping plane.

Override Camera Position

This block positions the camera above an object matching the provided name and method (category or object name).

Match By: Property with which to match an object (Category/Object Name).

Match Name: A string to match (case-insensitive) the object category or name.

Use Partial Match: If checked, all objects with a partial match to the provided name will be sampled.

Override Existing Camera Rotation: If checked, the initial rotation of the camera will be replaced with the provided values.

Initial Rotation (Yaw/Pitch/Roll): Ranges of rotation values to use as the camera's initial rotation. If "Override Existing Camera Rotation" is unchecked, the camera's current rotation values will be used.

Additional Rotation (Yaw/Pitch/Roll): Rotation values which are applied in addition to the initial rotation.

Additional Translation (X/Y/Z): Translation applied after the camera's position has already been overridden.

Orientation

Manipulators

Manipulators are modules that can be used to transform or manipulate objects that are already loaded into a scene.

Near IR (Approximation)

Adds a point light coincident with the camera and converts color space to panchromatic to mimic the appearance of near-field IR imagery.

Illuminator Power: Radiant power of the point light in Watts.

Track Camera to Object

Sets the camera to track to an object's position. The position of the tracking point is offset from the actual position of the object by using a randomly-generated spline.

  • Match By: Matches objects by their semantic category or object name.

  • Match Name: String to match (either category or object name, depending on previous option)

  • Use Partial Match: If enabled, the objects will be included if the specified string partially matches.

  • Camera Height: Height of the camera above the ground.

Noise may be added to the camera's tracking target so that the target is not perfectly centered in the frame. This is done by creating a random path which is superimposed over the target path.

  • Curve Height Offset: Height of the offset curve.

  • Control Points: Number of control points to use for the random curve.

  • Curve Min/Max: Min/max bounds of the curve in 2D.

  • Distance Between Points: Distance between control points.

  • Control Points Final: Final number of control points of the resampled curve.

See Camera Path Settings for specific types of camera paths.

Spherical

This module moves the camera along a sphere.

Look at: Define where your camera points using one of the blocks from the orientation menu.

Camera Path Settings

Settings for various camera path types.

"Match Target" will use the target object's path as the camera's path.
  • Offset X/Y: Additional 2D offset of the camera path.

"Elliptical Orbit" uses an ellipse as the camera's path
  • Radius A/B: Radius of the ellipse along each axis. Using equal values for both will yield a circular path. The center of the ellipse is the midpoint of the target's path.

  • Offset X/Y: Additional 2D offset of the camera path.

  • Step Size: Step size (degrees) per frame.

  • Orbit Offset: Initial radial offset along the path of the orbit.

"Linear" uses a line between the start and end points of the target path
  • Offset Start X/Y: 2D offset of the start of the camera's path.

  • Offset End X/Y: 2D offset of the end of the camera's path.

Override Camera Target

This block will point the camera toward a specific object or bone as well as move the camera to a specified distance from said object.

Object Name: Name of the object. If Bone Names is used, the name of this object should be that of a rig.

Distance to Target (min/max): Distance from the target object (or bone). If both values are set to 0, the distance will not be overridden.

Bone Names: Name(s) of the bones within the specified object to point to.

Light Manipulators

Light Object (Polar)

A physical light object using polar coordinates. Used with the Place Lights block.

Light Type: Light object type. "Point", "Area", or "Spot".

Coordinates: Coordinates of the light object relative to the bounding sphere of the target objects.

Radius: Radius of the polar coordinate, as a factor of the bounding sphere. For example, if the bounding sphere has a radius of 2 meters and the radius in this input is also set to 2, the light would be placed 4 meters from the origin.

Theta/Phi: Defines the azimuth angle and polar angle, respectively, expressed in degrees. Theta (azimuth) is limited to a range of [0, 360] and Phi (polar) is limited to a range of [0, 180]. [r=1, θ=0, φ=0]: Point at the upper pole of the sphere. [r=1, θ=0, φ=90]: Point at the equator, aligned to the +X world axis. [r=1, θ=90, φ=45]: Point aligned to the -Y world axis, halfway between the upper pole and equator. Size: "Size" of the light object. For Point and Spot lights, this is the radius of the light. For an Area light, this is the width/height of the light.

Power: Radiant power of the light in watts. Color Temp: Color temperature of the light in degrees Kelvin (more information on blackbody radiation may be found here).

Chance: Chance for this light to be created.

Place Lights

Places physical light objects in the scene relative to a target object's bounding box

Target Object: Optional name of a target object. If none is provided, the overall bounding box of non-background objects in the scene will be used.

Use Partial Match: If checked, objects with a name partially matching the provided "target object" string will be used for the overall bounding box.

Bounding Sphere Mode: Determines the method to fit a bounding sphere for the purposes of placing "polar" light objects. "Bounding Box" samples only the corners of the objects' bounding boxes when fitting spherical bounds. "Geometry" samples all vertices of the target objects' mesh in order to fit the sphere.

Frames: Number of unique frames to sample for each light's position.

Lights: An array of Light objects to place relative to the bounding box.

Camera Manipulators

Camera manipulators modify camera extrinsics by translating and rotating the camera with a variety of methods.

Randomize HDRI Background

Randomizes the HDRI background image per frame

  • Project to ground: If enabled, the lower half of the HDRI is flattened to the ground plane.

  • Height: Height above the ground at which the HDRI was captured. Only applies if "project to ground" is also enabled.

  • Intensity: Relative intensity factor of the HDRI.

  • Saturation: Relative color saturation of the HDRI where 0 is greyscale and 1 is full saturation.

  • Frames: Number of frames in the final sequence. Each frame will have a unique HDRI background.

Randomize Material Properties

Randomizes a numerical property of a material.

Material name: The name of the material to be randomized.

Property name: Name of the material property to randomize.

Value (min/max): A range of values to uniformly sample for the new property value.

Randomize Material Color

Randomizes the color of a material.

Material name: The name of the material to be randomized.

Chance: Chance (as a percentage, between 0 and 100) to randomize the material’s color.

Fisheye Camera Properties

  • FOV: Sets field of view, degrees

  • Sensor Size: The sensor in the camera expressed in millimeters

  • Focal Length: The camera focal length expressed in millimeters

Material Manipulators

Import Material

Load a set of materials from a collection.

Material collections are created on the "Materials" page. You can upload your own shaders and materials to a designated collection.

Modify Light Strength

Adjust the intensity of lights in your scene

This block will change the intensity of all light objects in your scene. Please note that this will impact HDRI light sources. IMPORTANT - please make sure the block is always used before the "Near IR" block. The "Near IR" block will add a light to your scene. Modifiers are executed sequentially and if it is executed first the light used in the IR simulation will not be impacted.

Weighted Material

Input a material name and a weight

Weight defines the likleyhood of this material being used over another one in the list.

Pose Set

A list of pose pairs.

A pose set block is a list of pose pairs, from which one pose pair will be chosen and then applied.

In the image above, two sets of poses will be applied to a single rig, "rig". The first set of poses will apply a pose partially matching the name "_L" (e.g. a left-handed pose).

The second set of poses will apply a pose partially matching the name "_R" (e.g. a right-handed pose).

Replace Material

Define how materials are mapped to other materials in the scene and at what frequency they are replaced.

Replace: Enter the material name in your scene file. In the final port on this row input an object name variable here. With: provide a material name with a weight.

Light Object (Bbox)

A physical light object relative to a bounding box. Used with the Place Lights block.

Light Type: Light object type. "Point", "Area", or "Spot".

Bbox Face: Bounding box face to reference. "Auto" will select the bounding box face closest to the camera.

Coordinates (UVW): Coordinates of the light object relative to the bounding box. "U" references the horizontal axis of the bounding box, "V" references the vertical axis, and "W" is normal to the bounding box face. The point [0, 0, 0] corresponds to the bottom-left corner of the bounding box. Values along the W axis are normalized to the depth of the bounding box. For example, a value of 0 would make the light coincident with the bounding box face and a value of 1 would offset the light along the W axis (normal) by a distance equal to the bounding box depth. [0, 0, 0]: Bottom-left corner of the face, coplanar with the face. [1, 1, 0]: Top-right corner of the face, coplanar with the face. [0.5, 0.5, 1]: Center of the face, offset by 1x the depth of the bounding box. Size: "Size" of the light object. For Point and Spot lights, this is the radius of the light. For an Area light, this is the width/height of the light.

Power: Radiant power of the light in watts. Color Temp: Color temperature of the light in degrees Kelvin (more information on blackbody radiation may be found here).

Chance: Chance for this light to be created.

Delete Objects by Name

This module will remove objects of a specific name from your scene.

Define the object name and what percentage of the items you want to be removed from the scene.

This can be a useful way to randomize a scene by randomly deleting items each time a new data point is generated.

Object Name: Name of the object to delete.

Percentage to delete: Percentage of objects matching the name to delete.

Ignore: Number of objects to skip deleting.

Min/Max Distance: Distance range from the camera in which objects may be ignored.

View Limits X (Min/Max): A pair of values where 0 is the left side of the frame and 1 is the right side of the frame.

View Limits Y (Min/Max): A pair of values where 0 is the bottom of the frame and 1 is the top of the frame.

Swap Materials

Randomly replace a material in your scene.

Import: Load a material collection into your scene.

Replace by: Select materials to be replaced by either "object name" or by category.

Sample assembly

Pose Objects

This module applies a pose to a rigged object

This block can be added to change the pose of an object given the name of the rig and a list of poses. There are two variants of this block--the following will only apply a single pose.

Rig Name: The name of the rig to modify.

Poses: A list of poses. See Pose Pair for more information.

This variant of the block will apply multiple poses (one from each "pose set").

Rig Name: The name of the rig to modify.

Pose Sets: A list of pose sets, each of which contain a list of pose pairs. See Pose Set for more information.

Constrain Object

Constrains (parents) one object to another. All transformations applied to the parent object will be inherited by the child object.

Child Collection: Collection from which to load a child object.

Parent Object Name: Name of the parent object (case-insensitive).

Parent Bone Name: (Optional) Name of the bone in the parent object. Only applies to rigged objects.

Partial Match Name: If checked, a partial match will be used for the parent object name.

Number of Parents (Min/Max): Number of objects to act as parents.

Number of Children (Min/Max): Number of objects to load as children.

Filter Objects/Annotations

Removes objects and/or annotations given the area of their segmentation in pixels

Objects are filtered depending on the provided categories and parameters. The "Category" block is found under the "Variables" section in the sidebar.

  • Category: Category name to which these settings apply.

  • Min segmentation area: Minimum allowable area of the segmentation of the object.

  • Category is optional: If a category is neither optional nor found in the final annotations, the pipeline will be re-run to produce a new data point.

  • Remove objects below min area: If true, objects not meeting the requirements will be removed from the scene and all images will be re-rendered. This process is repeated up to 3 times.

Load Object Along Curve

Loads objects along a curve

Curve Name: Name of the curve(s) along which the objects will be loaded.

Object Collection: Collection of objects to load along the curve.

Curve Parameter (Min/Max): Defines the part of the curve to sample where 0 is the start of the curve and 1 is the end of the curve.

Curve Parameter Tolerance: The tolerance (between 0 and 1) to use when sampling points along the curve. Using a larger value will cause objects to be placed farther apart from one another.

Number of Samples (Min/Max): Number of points to sample for placing objects.

Random Rotation (Min/Max): A range of rotation values (in degrees) to apply to the objects after they have been placed and aligned to the curve.

Pose Pair

Defines a pair of poses for interpolation.

This is used in conjunction with the Pose Objects block. This will interpolate between two poses, the amount of which is determined by min and max bias values (as a percentage). The example below will interpolate between open and closed poses between 0% and 25%. A bias below 50% will label the object using the name of Pose A, whereas a bias above 50% will label the object using the name of Pose B.

Pose A: The first pose.

Pose B: The second pose.

Bias Min: Weight toward the first pose.

Bias Max: Weight toward the second pose.

Pose name partial match: If checked, pose names partially matching the given name will also be sampled. For example, using the pose name "sitting" will match all poses containing the string "sitting".

Add Particle System

Adds a particle system to an object which populates the geometry with instances of another collection of objects. Commonly used to populate a landscape with vegetation.

Target Object Name: Name of the object to populate with particles.

Particle Collection: Collection to instance for unique particles.

Number of Particles (Min/Max): The total number of particles on the target object geometry.

Unique Particle Instance (Min/Max): The number of unique objects to use for the particle system.

Particle Random Scale: A value between 0 and 1 where 0 maintains the original object scale and 1 is the highest level of randomization.