Only this pageAll pages
Powered by GitBook
Couldn't generate the PDF for 155 pages, generation stopped at 100.
Extend with 50 more pages.
1 of 100

Seahaven Documentation

Seahaven

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Delete Objects by Name

This module will remove objects of a specific name from your scene.

Define the object name and what percentage of the items you want to be removed from the scene.

This can be a useful way to randomize a scene by randomly deleting items each time a new data point is generated.

Object Name: Name of the object to delete.

Percentage to delete: Percentage of objects matching the name to delete.

Ignore: Number of objects to skip deleting.

Min/Max Distance: Distance range from the camera in which objects may be ignored.

View Limits X (Min/Max): A pair of values where 0 is the left side of the frame and 1 is the right side of the frame.

View Limits Y (Min/Max): A pair of values where 0 is the bottom of the frame and 1 is the top of the frame.

Getting Started

Name Objects

You can assign a name to objects in the outliner. These names are typically used to tell manipulators what objects in a scene to transform or manipulate in some defined way.

  • You can assign an object a name in the blender outliner.

  • If you have multiple objects with the same name you can use a "." as a delineator and assign an index number. Seahaven will only look at the prefix if you try to modify these objects later with a manipulator.

    • This is a great way to manipulate large groups of objects.

    • In this example, the user is using a Blender empty to define a possible location to place a person.

Add Camera Curve to a scene

If you want to precisely position a camera in your scene you can load a scene with a curve object to define a specific range of positions for your camera.

  • For example, in this simulation, the user wanted the camera to follow the perimeter of a building and they drew a curve to represent the range of positions for the camera to take.

  • You can name the curve whatever you like. However, be sure to be consistent to assign the name in your camera module later.

Example curve in yellow

Prepare Your Models

Seahaven uses Blender (.blend) files for the 3D asset population, and there are a few steps of preparation to ensure your model will work effectively in your simulation.

Blender is a free download available through the Blender Foundation.

  • Remove everything from the scene except the 3D asset and any materials you will be packing into the file.

Camera and light source have been removed from this blend scene
  • Purge any orphan data from the scene. This unused data can interfere with cameras, lighting, and many other parts of the simulation.

  • To purge orphan data, navigate to display mode, select orphan data. Then select purge.

  • Last, make sure to pack all your data into the blend file.

  • Go to File, then External Data, then Pack All Into .blend

The orphan data being purged from a blend scene

Set Up Your Account

Seahaven is a platform that enables you to create synthetic data to train computer vision applications.

Our platform enables data scientists and AI engineers to create simulations of real-world places and events. With Lexset you can configure and run simulations that generate training data for your algorithms.

Once your account has been activated your account you will receive a link to your account page. Click the "login" button in the upper right corner.

Next, you will be directed to a login page. You can set a username and password or use your Gmail account.

After you have accessed your account you can go to your account page where you can access your user ID, bearer token, and other account information that will be necessary for you to use the Lexset APIs.

Note: We have blurred out all personal information for this tutorial. Please use your own account information.

Add a payment method

Child Object

Child objects are arranged in the scene relative to a parent object. It has similar inputs to a parent object with the addition of a series of placement rules which define the specific series of operations to perform in order to place the child object in the scene, typically relative to the parent object.

  • Category Override: An optional override for the category of objects directly associated to this parent object. If no override is provided, the name of the collection is used for the category (only if a collection is used as the source).

  • Tries: The number of attempts to make when placing this object using the provided placement rules before failing and proceeding to the next object in the tree.

  • Chance: The percent chance that the system will attempt to place this object. If a value of less than 100% is provided, there is a chance that this object (and all downstream in the same block) will be skipped.

  • Source: The source of this object, which may be an in the scene, a loaded , or a .

  • Placement Rules: A sequence of to execute when placing this object, typically relative to the direct parent of this object.

  • Children: All associated to this .

Parent Object

A Parent Object is a top-level object in your workflow.

  • Category Override: An optional override for the category of objects directly associated to this . If no override is provided, the name of the collection is used for the category (only if a collection is used as the source).

  • Source: The source of this object, which may be an existing asset in the scene, a loaded collection, or a generated asset.

Import 3D Assets

The 3D Assets tab is where you will bring 3D assets to use in your simulation into Seahaven. Here, assets are separated into folders called collections.

Collections are an important part of Seahaven. Collections not only contain groupings of 3D assets but the names will be used to manage annotations and other aspects of your simulations. Please name your collections carefully.

  • First, designate a Collection name on the top left-hand side of the upload tool. Each collection is comprised of the files you want to assign to a specific category.

  • Drag and drop your .blend files into the upload tool and select upload when they have finished processing.

Collection (Array)

A collection of assets arranged in a grid-like array.

  • Collection: The specific collection to sample for assets.

  • X/Y Count: Number of assets to load from this collection in each direction, provided by an or .

  • X/Y Spacing:

Children: All child objects associated to this parent object.

parent object
child objects
existing asset
collection
generated asset
rules
child objects
parent object

Seahaven Documentation

This Collection should now be available for placement in the Relationship Editor

Populating and uploading a collection
Creating the collection name
The distance between each asset in the array in each direction, provided by a
variable.
  • Spacing Method: Defines how to determine the space between each asset.

    • "Object bounds" sets the spacing of the assets relative to their bounding boxes. If the array consists of multiple assets, the bounds are defined by the maximum value from all sampled assets.

    Array spacing using the "Object bounds" method
    • "Grid lines" places the objects in a grid where the grid lines are spaced using the provided values for X/Y spacing. For example, if the X/Y spacing is set to 1 meter, objects will be placed in a grid where the grid lines are 1 meter apart in the X and Y directions. This method should be used only in cases where assets will have sufficient space between one another and where there is relatively low variation in the size of assets in this collection.

  • Array spacing using the "Grid lines" method
    • Use Largest Dimension: If checked, the largest dimension of an object is used to determine its bounds. For example, if an object is 1x4 meters, the grid spacing will be 4x4 meters. This is useful if you intend to rotate individual objects in the array.

    • Use Single Asset: If checked, the same asset will be instanced for each array element.

    • Separate Array Elements: If checked, placement operations for all downstream child objects will be executed for each element in the array.

    • Remove Invalid Array Elements: If checked, invalid elements (typically caused by object collisions) will be removed after all have been executed. This allows for placement of the entire array to be successful even if individual elements would otherwise cause it to fail.


    In practice, an array is treated as any other object in your simulation. Many placement rules have an "Execute per Array Element" option. If this is enabled, the corresponding placement rule will be executed on a per-element basis. In the below example, a 3x3 array of a single screw asset is placed on a ground plane using the Put Object rule and is subsequently rotated with Rotate Object. When "Execute per Array Element" is disabled, the entire array of objects is rotated together. When enabled, each screw is rotated in-place.

    "Execute per Array Element" disabled
    "Execute per Array Element" enabled
    integer value
    integer range
    number

    Workflow

    The workflow is the container where you will be defining all object relationships.

    Relationship Editor

    In the Relationship Editor tab you will designate the relationships of your 3D assets to each other and elements in the scene (e.g. ground_plane). Object relationships are defined using a hierarchical structure of parent and child objects. A Workflow may have multiple parent objects, each of which may have multiple child objects.

    Asset Sources

    Assets may be sourced from existing objects already in the scene, generated directly, or be loaded from a collection in your account.

    Get Object by Properties

    Gets an object currently in the scene provided a category and/or object name.

    • Category: Category of the target object.

    • Object Name: Name of the target object.

    • Partial Match: If checked, partial matches to the provided strings will be accepted when retrieving objects from the scene.

    Locate Object (Global)

    Sets the location of an object in global coordinate space.

    • Position: Sets the position of the object using an XYZ Range.


    Collection

    A collection of assets used as a source for an object.

    • Collection: The specific collection to sample for assets.

    • Samples: Number of assets to load from this collection, provided by an integer value or integer range.

    Placement Rules

    Placement rules are used to transform child objects. Transformations are done in multiple ways:

    • In global (scene) coordinate space (e.g. translation)

    • In local (object) space (e.g. rotation)

    • Relative to a parent object (e.g. alignment, orientation, etc.)

    The following workflow constructs a basic scene consisting of a ground plane and screws:

    • A ground plane (0.5m x 0.5m) is generated using a generator

    • Multiple screws are loaded from a

    • The screws are organized using a series of :

      • : Places the screws on the ground plane between the UV coordinates (0.2, 0.2) and (0.8, 0.8)

    placement rules

    Rotate Object: Rotates each object about its local Z axis between 0 and 360 degrees

    Ground Plane
    collection
    placement rules
    Put Object

    Basic Transformations

    Basic transformations consist of typical affine transformations such as translation, rotation, and scaling.

    Scale Object (Uniform)

    Scales an object uniformly (equally along all axes).

    • Scale Factor: Factor by which to scale the object, provided by a Number. A value of 1 corresponds to the object's original scale.

    Locate Object (Local)

    Sets the location of an object in local space, relative to a face on the parent object's bounding box.

    • Position: Sets the position of the object using a UVW Range.

    • Face: The face(s) to reference on the parent object when positioning the object in local space.


    Translate Object

    Translates an object relative to its current position.

    • Translation (XYZ): Determines the translation vector using an XYZ Range.

    • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

    Generator: Ground Plane

    Generates a ground plane object.

    • Size: Size of the ground plane in meters

    • Shadow Catcher: If checked, objects will cast a shadow on the ground plane, but the object itself will otherwise not be visible.

    Drop Object

    "Drops" an object onto a parent object using a vertical raycast to determine the point of intersection. If there is no intersection (i.e. the child object is not located directly above a surface), the rule will fail.

    • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

    • Use Parent Above: If enabled, this placement rule will instead reference the parent object above this object's direct parent in the tree. For example, if the hierarchy of objects is Table > Plate > Food where "Food" is the , enabling "Use Parent Above" in this case would use "Table" as the instead of "Plate".


    The following workflow uses the "" rule to place screws on a ground plane:

    • A ground plane (0.5m x 0.5m) is generated using a generator

    • Multiple screws are loaded from a

    • The screws are organized using a series of :

      • : Moves the screws to positions between (-0.5, -0.5) and (0.5, 0.5) at a height of 1 meter.

    Drop Object: Drops the screws onto the ground plane (parent object).

  • Rotate Object: Rotates the screws about their local Z axis.

  • child object
    parent object
    drop object
    Ground Plane
    collection
    placement rules
    Locate Object (Global)

    Complex Transformations

    Complex transformations may consist of several basic transformations and typically reference a parent and/or child bounding box face.

    Align Objects

    Aligns a child object to a parent object, given a bounding box face for each. The objects to not necessarily have to intersect with one another. The parent face is treated as an infinite plane when aligning the child object.

    • Parent Face: Reference face on the parent object.

    • Child Face: Reference face on the child object.

    • Use Nearest Parent/Child Face: If enabled, the most appropriate will be automatically determined by proximity.


    In this example, between 2 and 4 screws are placed on a using the "" . For each object, an additional screw is placed on the using the same method and referencing the one level above. The objects are then aligned to one another using the right of the and the left of the .

    Orient Object to Surface

    Re-orients an object to align to the nearest surface. This is typically used when placing objects on an uneven surface.

    • Child Face: Reference on the .

    • Execute Per Array Element: If enabled, this will be applied to each element within the , provided that the 's source is an .


    Rotate Object

    Rotates an object about its local axes.

    • Rotation (Degrees): Amount by which to rotate the object per axis in degrees. Provided using an .

    • Step Size (Optional): Locks the rotation amount to a given step size. For example, a step size of 90 will limit rotation values to 0, 90, 180, and 270. This value is typically an , however may also be used.

    Scale Object (Non-Uniform)

    Scales an object non-uniformly with a different value per axis.

    • Scale: Factor by which to scale the object along each axis, as provided by an .

    Put Object

    "Puts" an object on a using the top of the and the bottom of the .

    • Position: The relative position of the object on the top of the provided by a .

    • Use Parent Above: If enabled, this will instead reference the above this object's direct parent in the tree. For example, if the hierarchy of objects is Table > Plate > Food where "Food" is the

    Attach Object

    Attaches one object to another provided a for each.

    • Parent Face: Reference on the .

    • Child Face: Reference on the .

    Move Object Nearby

    Moves a horizontally (within the global XY plane) to be within a specified distance range relative to the .

    • Minimum/Maximum Distance (m): Distance from the bounds of the provided by a.

    A region representing the provided distances is created by inscribing the bounds of the object with an ellipse. A distance of 0 corresponds to no offset from the original object bounds (shown as "minimum" below). Distances above 0 offset the object bounds by that specified distance. The outer region is defined by inscribing the offset bounds with a new ellipse. By using an ellipse, this prevents the corners of the rectangular region from being sampled, which would otherwise sample distances greater than intended. Furthermore, using a circular region relative to the object center would not account for objects with a footprint of uneven proportions (e.g. a vehicle).


    UVW Range

    Defines a range of coordinates in local 3D space. This is typically associated with the face of a bounding box.

    • U (min/max): Position along the horizontal (U) axis.

    • V (min/max): Position along the vertical (V) axis.

    Face

    Defines a face of a to sample.

    If multiple faces are used as a single input, the selected face will be chosen at random.

    Each option indicates a locally-aligned bounding box face of a given object from the following options:

    • Right, Left, Front, Back, Top, Bottom

    Additionally, there is an "Any" option which samples from (Right, Left, Front, Back) at random.

    The following sample uses the "" rule to place a relative to a where the face on the is the "Top" and the face on the

    UV Range

    Defines a range of coordinates in local 2D space. This is typically associated with the face of a bounding box.

    • U (min/max): Position along the horizontal (U) axis.

    • V (min/max): Position along the vertical (V) axis.


    Neighbor

    Places an object near another, relative to a bounding box face on each.

    • Parent Face: Reference on the .

    • Child Face: Reference on the .

    Variables

    Variables are used as generic inputs primarily for in your object relationships.

    Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

    XYZ Range
    integer
    float values
    XYZ Range
    placement rules
    face
    ground plane
    Put Object
    placement rule
    ground plane
    parent object
    face
    parent object
    face
    child object

    Scene

    Scene objects are used to define context. Including lighting, and procedurally or parametrically generated objects.

    Simulation Editor

    In this example, multiple cars are placed on a ground object currently in the scene. The "Put Object" placement rule performs the initial placement of the car on the ground. It is then rotated about its Z axis by a random amount, and finally oriented to the surface using its bottom face.

    face
    child object
    placement rule
    array
    child object
    array
    , enabling "Use Parent Above" in this case would use "Table" as the
    instead of "Plate".

    In this example, an array of screws is placed on the center of a ground plane using the "Put Object" placement rule.

    parent object
    face
    parent object
    face
    child object
    face
    parent object
    UV Range
    placement rule
    parent object
    child object
    parent object

    Position: Relative position of the child object on the face of the parent object provided by a UV Range.

  • Execute Per Array Element: If enabled, this placement rule will be applied to each element within the array, provided that the child object's source is an array.

  • Use Parent Above: If enabled, this placement rule will instead reference the parent object above this object's direct parent in the tree. For example, if the hierarchy of objects is Table > Plate > Food where "Food" is the child object, enabling "Use Parent Above" in this case would use "Table" as the parent object instead of "Plate".


  • face
    face
    parent object
    face
    child object

    In this example, between 2 and 4 screws are placed on a ground plane using the "Put Object" placement rule. For each object, an additional screw is aligned to it using the "Align Objects" placement rule. The same screw is then moved to be near another screw with a minimum distance of 0.05 meters and a maximum distance of 0.1 meters.

    child object
    parent object
    parent object
    Number
    W (min/max): Position along the normal (W) axis. Unlike the U and V components, the W component is expressed in meters. For example, the point (0.5, 0.5, 1) in UVW space as implemented in this system corresponds to a point aligned to the center of a bounding box face (0.5, 0.5), offset by 1 meter along the face's normal (away from the face).

    Sample implementation of a UVW Range block
    is the "Bottom".
    parent object
    Attach
    child object
    parent object
    parent object
    child object
    A placement rule using multiple options for the "Child Face" input
    Sample implementation of a UV Range block
    Offset Distance: Distance to offset from the parent face when placing the child object provided by a Number.
  • Horizontal Position: Horizontal position along the parent face to use when placing the child object. A value of 0 corresponds to the left edge, while a value of 1 corresponds to the right edge. Provided by a Number.


  • face
    parent object
    face
    child object

    XYZ Range

    Defines a range of coordinates/values in object or scene (XYZ) space.


    XYZ Ranges are used to sample a range of values per axis (X, Y, Z). The specific implementation of these values varies per . Different implementations are illustrated below:

    • Locate: Used to determine the position of the object in global space.

    • Translate:

    Float Value

    Provides a single floating point value.

    Numbers

    Numerical inputs are used across many placement rules and associated blocks. Typically, any of these blocks (integer/float) may be used, unless the input is defining a count, in which case an integer block must be used.

    3D Asset

    Import 3D Assets into your simulation

    The number of samples determines how many unique items to sample out of the collection.

    For example, if we have a collection with four different 3D models of fruit (apple, orange, pear, pineapple), if you set the samples to 2, it will randomly select two from that collection. If "Load Duplicate" is enabled, the same asset may be loaded more than once.

    To select a collection, use a "Collection" block from the "Variables" submenu.

    Float Range

    Uniformly samples a floating-point number defined by a minimum and maximum.

    Integer Value

    Provides a single integer value.

    Vectors

    Various 2D and 3D vectors are used to define positions in the scene, either relative to the scene coordinates (XYZ) or relative to another / (UVW).

    Workflow

    The workflow is the container you will use to define the parameters of your simulation.

    • Scene: Scene modules are used to configure your scene context. This will include importing/generating 3D environments, backgrounds, and/or lighting.

    • Relationships: Relationships between classes of objects defined in the relationship editor are imported into your workflow here.

    Integer Range

    Samples a range of integers from a min and max (inclusive) value. For example, if the range is between 4 and 6, the value can be either 4, 5, or 6.

    object
    face
    Camera: Camera type, position, and camera properties are all defined in this section.
  • Manipulators: Manipulators can be used to transform and randomize your scene each time a data point is generated.

  • Render Passes: Individual render passes are defined here (RGB, depth, normal, etc.).

  • Output Files: Non-image output is defined here (COCO annotations, camera matrices, etc.).

  • Raytrace Samples: Number of paths to trace for each pixel in the RGB images. Increasing the number of samples improves render quality, but takes more time.

  • Image Format: Image file format (PNG, JPEG, TIFF).

  • Color Depth: Image bit depth (8- and 16-bit).

  • Color Mode: Image color mode (RGB, Panchromatic).

  • JPEG Quality: JPEG image quality (only applies to JPEG image format).

  • Use experimental raytracer: Enables experimental raytracer features (not applicable in most simulations).

  • Use dynamic BVH tree: If checked, the BVH tree will be built dynamically (per object). This method may be more time-efficient for animated sequences.

  • Camera

    Define your camera and how it is positioned within your simulated environment.

    Relationships

    Import 3D Assets & Relationships: This module is to import relationships between collections that have been defined in the relationships editor.

    Translates the object using a vector defined by an XYZ range.
  • Rotate: Rotates the object about each of its local axes by a specified number of degrees.

  • Scale: Scales the object about each of its local axes by a factor.

  • placement rule

    Room Type

    Defines a room type, which uses accompanying object relationships to populate the space.

    Room type: Choose from a list of room layout presets including "bedroom" and "living room" (more coming soon).

    Object relationships: Object relationships created from the relationship editor which will be used to populate rooms of this type.

    Usage of the "Generate interior scene" block, with accompanying Room Type and Relationships blocks.

    Load Relationships

    The Load Relationships module allows you to select a relationship file from a drop down menu. If you created a relationship in the Relationship Editor tab, it will appear here.

    Physical Sun/Sky

    Adds a procedural sun and sky to the environment.

    Sun Strength: Relative strength of the sun's illumination. It is recommended to use a low value (e.g. 0.1) to avoid over-exposure.

    Sun Rotation: Rotation of the sun in the sky relative to the world origin where 0 degrees locates the sun at the positive Y world axis.

    Sun Elevation: Elevation of the sun (in degrees) relative to the horizon. Sun/Sky Model: Nishita (Nishita et al. 1993.), Hosek-Wilkie (Hosek-Wilkie et al. 2012.), or Hosek-Wilkie (No-Shadow). If the "no-shadow" option is selected, shadow rays from the sun lamp will be disabled.

    Ground Plane

    This module generated a ground plane at the origin of the scene.

    Shadow Plane: If "Shadow Plane" is enabled objects will cast a shadow on the ground plane.

    HDRI Lighting

    High Dynamic Range Images or HDRI's are large images that project ambient light into your scene. the HDRI Lighting module allows the user to select from a drop-down menu of environment types.

    With the HDRI lighting module, you load a variety of pre-loaded HDRI images that you can use as a background in your images. In addition, these images provide physically accurate environmental lighting. Project to ground: You can project your HDRI images to the scenes ground plane. Resulting in a semi-spherical projection. You can learn more about how this feature can be used when creating photorealistic images on our blog. Height: Height refers to the height of the camera when the HDRI was captured. Most HDRI images are taken between 2 and 3 meters off the ground. You can adjust this parameter to reduce distortion around the horizon, which might appear if your camera moves too high above eye level in the z axes.

    Intensity: Set a minimum and maximum light intensity. The range will dictate the range of lighting brightness across all the images in your dataset.

    Saturation: Sets a range for the color saturation of the HDRI background. This is a normalized value where 0 yields a greyscale background and 1 yields a full-color background.

    Interior Scene

    Generates an interior scene using a given room type preset.

    Room Type: The room layout to use for the generated scene. Options: (bedroom, livingroom). (See Room Type for more information on room types.)

    Room Height: The ceiling height of the generated room (in meters).

    Wall type: The material to use for the walls in the room. Options: (Gypsum, Wallpaper).

    Floor type: The material to use for the floors in the room. Options: (Oak, Maple, Tile, Ceramic).

    Ceiling type: The material to use for the ceilings in the room. Options: (Wood, Gypsum, ACT).

    Window type: The window shape to use. The "Tall" preset is approx. 2.3m tall. Options: (Standard, Tall).

    Generate recessed lights: If checked, recessed lights will be generated in the ceiling of the room.

    Color temp: Color temperature of the lights, in degrees Kelvin. See for more information.

    Light strength multiplier: A factor by which to multiply the strength of the recessed lights. Setting both the min and max to 0 will disable the lights.

    Curve-Based Camera

    Randomly samples a camera pose on a curve.

    Frames: Number of frames to sample along the curve.

    Curve Name: Name of the curve to sample.

    Yaw/Pitch/Roll (Min/Max): Rotation ranges for the camera about its 3 axes. Setting all values to 0 will point the camera straight down. Setting the pitch to 90 degrees will make the camera parallel with the XY/ground plane.

    Camera Height (Min/Max): Height of the camera above the curve in meters.

    Properties: Perspective camera intrinsic properties.

    Additional documentation.
    Wikipedia
    Usage of the "Generate interior scene" block

    Position Camera

    Cameras are positioned within ranges of values. Camera position is defined as a bounding volume or point cloud within which you will sample locations. You can choose from the following options.

    Room-Based Camera

    Positions a camera in a room generated using the "generate interior scene" block.

    • Frames: Number of frames to sample in the room. Frames are generated along a random path within the room.

    • Control Points: Number of points which are used to create the camera path. Higher numbers increase the complexity of the path.

    • Distance Between Points: The min/max distance between consecutive points in the camera path.

    • Camera Height: Camera height above the floor (in meters).

    • Yaw/Pitch/Roll: Random rotation along the path (in degrees).

    • Additional Cameras: Optional list of additional camera positions/orientations to sample per frame. See documentation for additional information.

    • Properties: Fisheye camera intrinsic properties.

    Category

    Category: Name of the category to check in each image.

    Min/Max: The minimum and maximum number of instance allowed of the category in each image.

    Additional Cameras

    Allows for additional camera positions to be sampled per frame.

    Note: All values are expressed relative to an assumed origin and orientation. In other words, it can be treated as adding cameras to a device (such as a head mounted display), which informs the initial location and orientation.

    • Offset: Distance to offset along each axis (in meters) for this specific camera.

    • Rotation: Additional rotation to apply to the camera (in degrees).

    • Filename Suffix: Suffix to append to each image rendered by this camera. For example, using a suffix of "_L_UP" would change a filename from "rgb_0001.png" to "rgb_0001_L_UP.png"

    Camera Configuration

    Camera Configuration is a module that is loaded into the Camera bay and allows placement of Position and Properties modules

    • Seahaven currently supports two different camera types (Perspectival and fisheye)

    • Each of these camera types can either be positioned by a curve or by using one of the position modules in Seahaven.

    • Curves can be imported with the 3D Asset module in the scene menu. For more information, please see the prepare your models section of this documentation.

    • Additionally, rotation ranges may be defined for each curve-based camera.

    Frames: The number of unique frames to create in the current scene. Each frame represents a unique data point.

    Curve name: The name of the curve with which the camera location will be determined.

    Yaw (min/max): A range of values, determining the rotation of the camera about its up axis (looking left and right).

    Pitch (min/max): A range of values, determining the rotation of the camera about its right axis (looking up and down).

    Roll (min/max): A range of values, determining the rotation of the camera about its forward axis (rotating clockwise/counter-clockwise).

    Camera Height (min/max): The height of the camera relative to the curve (in meters).

    Room-Perimeter Camera

    Positions a camera on the perimeter of a room generated using the "generate interior scene" block.

    Frames: Number of frames to sample in the room.

    Position: The location within the room to sample. Options: (Wall, Corner).

    Offset Distance: The distance from the camera to the wall (in meters).

    Camera Height: Camera height above the floor (in meters).

    Yaw/Pitch/Roll: Random rotation along the path (in degrees).

    Required Categories: Categories which are required to be visible in each image.

    Properties: Fisheye camera intrinsic properties.

    Minimum Distance to Camera: Restricts objects from being closer than this distance to the camera.

    Single Raycast Distance Check: If checked, distance will only be checked from the center of the frame.

    Cubic

    This module allows you to choose points in a 3D bounding box for the camera to sample from

    Look at: Define where your camera points using one of the blocks from the orientation menu.

    Conical

    This module moves the camera along a cone

    Look at: Define where your camera points using one of the blocks from the orientation menu.

    Additional documentation.
    Additional documentation.
    Additional documentation.

    Orientation

    Point

    This block will point the camera towards a specific location within the scene.

    Define a target point to point your camera at.

    Perspectival Camera Properties

    Camera intrinsic properties for a perspectival camera.

    FOV (Min/Max): Sets the horizontal field of view (in radians).

    F-Stop: The F-Stop (aperture) of the lens. Affects the depth-of-field of the image.

    Aperture Blades: Number of blades for the aperture. Affects the shape of the bokeh.

    Aperture Ratio: Simulates distortion of bokeh. Values below 1 stretch the shape of the bokeh horizontally while values above 1 will stretch the bokeh vertically.

    Camera Near/Far Clip: Sets the distance of the clipping plane.

    Fisheye Camera Properties

    • FOV: Sets field of view, degrees

    • Sensor Size: The sensor in the camera expressed in millimeters

    • Focal Length: The camera focal length expressed in millimeters

    Animated Track Camera

    Animates/tracks a camera along a curve.

    Frames: Number of frames to sample along the curve.

    Curve Name: Name of the curve the camera will follow.

    Properties: Perspective camera intrinsic properties. Additional documentation.

    Shutter Speed: Shutter speed of the camera in seconds.

    Camera Height (Min/Max): Height of the camera above the curve in meters.

    Curve Parameter Start (Min/Max): A value between 0 and 1 indicating the start position of the camera where 0 is the start of the curve and 1 is the end of the curve.

    Curve Parameter Length (Min/Max): A value between 0 and 1 indicating the distance to travel along the curve where 0 is the start of the curve and 1 is the end of the curve.

    Spherical

    This module moves the camera along a sphere.

    Look at: Define where your camera points using one of the blocks from the orientation menu.

    Override Camera Target

    This block will point the camera toward a specific object or bone as well as move the camera to a specified distance from said object.

    Object Name: Name of the object. If Bone Names is used, the name of this object should be that of a rig.

    Distance to Target (min/max): Distance from the target object (or bone). If both values are set to 0, the distance will not be overridden.

    Bone Names: Name(s) of the bones within the specified object to point to.

    Calculate Area of Interest

    This block will automatically point the camera towards the items in the scene.

    The bounding box of all the items in the scene is calculated and the camera will point towards the center of that bounding box.

    Override Camera Position

    This block positions the camera above an object matching the provided name and method (category or object name).

    Match By: Property with which to match an object (Category/Object Name).

    Match Name: A string to match (case-insensitive) the object category or name.

    Use Partial Match: If checked, all objects with a partial match to the provided name will be sampled.

    Override Existing Camera Rotation: If checked, the initial rotation of the camera will be replaced with the provided values.

    Initial Rotation (Yaw/Pitch/Roll): Ranges of rotation values to use as the camera's initial rotation. If "Override Existing Camera Rotation" is unchecked, the camera's current rotation values will be used.

    Additional Rotation (Yaw/Pitch/Roll): Rotation values which are applied in addition to the initial rotation.

    Additional Translation (X/Y/Z): Translation applied after the camera's position has already been overridden.

    Manipulators

    Manipulators are modules that can be used to transform or manipulate objects that are already loaded into a scene.

    Light Manipulators

    Camera Manipulators

    Camera manipulators modify camera extrinsics by translating and rotating the camera with a variety of methods.

    Material Manipulators

    Track Camera to Object

    Sets the camera to track to an object's position. The position of the tracking point is offset from the actual position of the object by using a randomly-generated spline.

    • Match By: Matches objects by their semantic category or object name.

    • Match Name: String to match (either category or object name, depending on previous option)

    • Use Partial Match: If enabled, the objects will be included if the specified string partially matches.

    • Camera Height: Height of the camera above the ground.

    Noise may be added to the camera's tracking target so that the target is not perfectly centered in the frame. This is done by creating a random path which is superimposed over the target path.

    • Curve Height Offset: Height of the offset curve.

    • Control Points: Number of control points to use for the random curve.

    • Curve Min/Max: Min/max bounds of the curve in 2D.

    • Distance Between Points: Distance between control points.

    See for specific types of camera paths.

    Near IR (Approximation)

    Adds a point light coincident with the camera and converts color space to panchromatic to mimic the appearance of near-field IR imagery.

    Illuminator Power: Radiant power of the point light in Watts.

    Randomize HDRI Background

    Randomizes the HDRI background image per frame

    • Project to ground: If enabled, the lower half of the HDRI is flattened to the ground plane.

    • Height: Height above the ground at which the HDRI was captured. Only applies if "project to ground" is also enabled.

    • Intensity: Relative intensity factor of the HDRI.

    • Saturation: Relative color saturation of the HDRI where 0 is greyscale and 1 is full saturation.

    • Frames: Number of frames in the final sequence. Each frame will have a unique HDRI background.

    Randomize Material Color

    Randomizes the color of a material.

    Material name: The name of the material to be randomized.

    Chance: Chance (as a percentage, between 0 and 100) to randomize the material’s color.

    Place Lights

    Places physical light objects in the scene relative to a target object's bounding box

    Target Object: Optional name of a target object. If none is provided, the overall bounding box of non-background objects in the scene will be used.

    Use Partial Match: If checked, objects with a name partially matching the provided "target object" string will be used for the overall bounding box.

    Bounding Sphere Mode: Determines the method to fit a bounding sphere for the purposes of placing "polar" light objects. "Bounding Box" samples only the corners of the objects' bounding boxes when fitting spherical bounds. "Geometry" samples all vertices of the target objects' mesh in order to fit the sphere.

    Frames: Number of unique frames to sample for each light's position.

    Lights: An array of Light objects to place relative to the bounding box.

    Camera Path Settings

    Settings for various camera path types.

    "Match Target" will use the target object's path as the camera's path.
    • Offset X/Y: Additional 2D offset of the camera path.

    "Elliptical Orbit" uses an ellipse as the camera's path
    • Radius A/B: Radius of the ellipse along each axis. Using equal values for both will yield a circular path. The center of the ellipse is the midpoint of the target's path.

    • Offset X/Y: Additional 2D offset of the camera path.

    • Step Size: Step size (degrees) per frame.

    • Orbit Offset: Initial radial offset along the path of the orbit.

    • Offset Start X/Y: 2D offset of the start of the camera's path.

    • Offset End X/Y: 2D offset of the end of the camera's path.

    Modify Light Strength

    Adjust the intensity of lights in your scene

    This block will change the intensity of all light objects in your scene. Please note that this will impact HDRI light sources. IMPORTANT - please make sure the block is always used before the "Near IR" block. The "Near IR" block will add a light to your scene. Modifiers are executed sequentially and if it is executed first the light used in the IR simulation will not be impacted.

    Randomize Material Properties

    Randomizes a numerical property of a material.

    Material name: The name of the material to be randomized.

    Property name: Name of the material property to randomize.

    Value (min/max): A range of values to uniformly sample for the new property value.

    Light Object (Polar)

    A physical light object using polar coordinates. Used with the Place Lights block.

    Light Type: Light object type. "Point", "Area", or "Spot".

    Coordinates: Coordinates of the light object relative to the bounding sphere of the target objects.

    Radius: Radius of the polar coordinate, as a factor of the bounding sphere. For example, if the bounding sphere has a radius of 2 meters and the radius in this input is also set to 2, the light would be placed 4 meters from the origin.

    Theta/Phi: Defines the azimuth angle and polar angle, respectively, expressed in degrees. Theta (azimuth) is limited to a range of [0, 360] and Phi (polar) is limited to a range of [0, 180]. [r=1, θ=0, φ=0]: Point at the upper pole of the sphere. [r=1, θ=0, φ=90]: Point at the equator, aligned to the +X world axis. [r=1, θ=90, φ=45]: Point aligned to the -Y world axis, halfway between the upper pole and equator. Size: "Size" of the light object. For Point and Spot lights, this is the radius of the light. For an Area light, this is the width/height of the light.

    Power: Radiant power of the light in watts. Color Temp: Color temperature of the light in degrees Kelvin (more information on blackbody radiation may be found ).

    Chance: Chance for this light to be created.

    Swap Materials

    Randomly replace a material in your scene.

    Import: Load a material collection into your scene.

    Replace by: Select materials to be replaced by either "object name" or by category.

    Sample assembly

    Replace Material

    Define how materials are mapped to other materials in the scene and at what frequency they are replaced.

    Replace: Enter the material name in your scene file. In the final port on this row input an object name variable here. With: provide a material name with a weight.

    Light Object (Bbox)

    A physical light object relative to a bounding box. Used with the Place Lights block.

    Light Type: Light object type. "Point", "Area", or "Spot".

    Bbox Face: Bounding box face to reference. "Auto" will select the bounding box face closest to the camera.

    Coordinates (UVW): Coordinates of the light object relative to the bounding box. "U" references the horizontal axis of the bounding box, "V" references the vertical axis, and "W" is normal to the bounding box face. The point [0, 0, 0] corresponds to the bottom-left corner of the bounding box. Values along the W axis are normalized to the depth of the bounding box. For example, a value of 0 would make the light coincident with the bounding box face and a value of 1 would offset the light along the W axis (normal) by a distance equal to the bounding box depth. [0, 0, 0]: Bottom-left corner of the face, coplanar with the face. [1, 1, 0]: Top-right corner of the face, coplanar with the face. [0.5, 0.5, 1]: Center of the face, offset by 1x the depth of the bounding box. Size: "Size" of the light object. For Point and Spot lights, this is the radius of the light. For an Area light, this is the width/height of the light.

    Power: Radiant power of the light in watts. Color Temp: Color temperature of the light in degrees Kelvin (more information on blackbody radiation may be found ).

    Chance: Chance for this light to be created.

    Import Material

    Load a set of materials from a collection.

    Material collections are created on the "Materials" page. You can upload your own shaders and materials to a designated collection.

    Weighted Material

    Input a material name and a weight

    Weight defines the likleyhood of this material being used over another one in the list.

    Pose Objects

    This module applies a pose to a rigged object

    This block can be added to change the pose of an object given the name of the rig and a list of poses. There are two variants of this block--the following will only apply a single pose.

    Rig Name: The name of the rig to modify.

    Poses: A list of poses. See Pose Pair for more information.

    This variant of the block will apply multiple poses (one from each "pose set").

    Rig Name: The name of the rig to modify.

    Pose Sets: A list of pose sets, each of which contain a list of pose pairs. See for more information.

    Pose Set

    A list of pose pairs.

    A pose set block is a list of pose pairs, from which one pose pair will be chosen and then applied.

    In the image above, two sets of poses will be applied to a single rig, "rig". The first set of poses will apply a pose partially matching the name "_L" (e.g. a left-handed pose).

    The second set of poses will apply a pose partially matching the name "_R" (e.g. a right-handed pose).

    Filter Objects/Annotations

    Removes objects and/or annotations given the area of their segmentation in pixels

    Objects are filtered depending on the provided categories and parameters. The "Category" block is found under the "Variables" section in the sidebar.

    • Category: Category name to which these settings apply.

    • Min segmentation area: Minimum allowable area of the segmentation of the object.

    • Category is optional: If a category is neither optional nor found in the final annotations, the pipeline will be re-run to produce a new data point.

    • Remove objects below min area: If true, objects not meeting the requirements will be removed from the scene and all images will be re-rendered. This process is repeated up to 3 times.

    Constrain Object

    Constrains (parents) one object to another. All transformations applied to the parent object will be inherited by the child object.

    Child Collection: Collection from which to load a child object.

    Parent Object Name: Name of the parent object (case-insensitive).

    Parent Bone Name: (Optional) Name of the bone in the parent object. Only applies to rigged objects.

    Partial Match Name: If checked, a partial match will be used for the parent object name.

    Number of Parents (Min/Max): Number of objects to act as parents.

    Number of Children (Min/Max): Number of objects to load as children.

    Pose Pair

    Defines a pair of poses for interpolation.

    This is used in conjunction with the Pose Objects block. This will interpolate between two poses, the amount of which is determined by min and max bias values (as a percentage). The example below will interpolate between open and closed poses between 0% and 25%. A bias below 50% will label the object using the name of Pose A, whereas a bias above 50% will label the object using the name of Pose B.

    Pose A: The first pose.

    Pose B: The second pose.

    Bias Min: Weight toward the first pose.

    Bias Max: Weight toward the second pose.

    Pose name partial match: If checked, pose names partially matching the given name will also be sampled. For example, using the pose name "sitting" will match all poses containing the string "sitting".

    Add Particle System

    Adds a particle system to an object which populates the geometry with instances of another collection of objects. Commonly used to populate a landscape with vegetation.

    Target Object Name: Name of the object to populate with particles.

    Particle Collection: Collection to instance for unique particles.

    Number of Particles (Min/Max): The total number of particles on the target object geometry.

    Unique Particle Instance (Min/Max): The number of unique objects to use for the particle system.

    Particle Random Scale: A value between 0 and 1 where 0 maintains the original object scale and 1 is the highest level of randomization.

    Load Object Along Curve

    Loads objects along a curve

    Curve Name: Name of the curve(s) along which the objects will be loaded.

    Object Collection: Collection of objects to load along the curve.

    Curve Parameter (Min/Max): Defines the part of the curve to sample where 0 is the start of the curve and 1 is the end of the curve.

    Curve Parameter Tolerance: The tolerance (between 0 and 1) to use when sampling points along the curve. Using a larger value will cause objects to be placed farther apart from one another.

    Number of Samples (Min/Max): Number of points to sample for placing objects.

    Random Rotation (Min/Max): A range of rotation values (in degrees) to apply to the objects after they have been placed and aligned to the curve.

    here
    here
    Pose Set
  • Control Points Final: Final number of control points of the resampled curve.

  • Camera Path Settings
    "Linear" uses a line between the start and end points of the target path