Blender is a free download available through the Blender Foundation.

• Remove everything from the scene except the 3D asset and any materials you will be packing into the file.

• Purge any orphan data from the scene. This unused data can interfere with cameras, lighting, and many other parts of the simulation.

• To purge orphan data navigate to display mode, select orphan data. Then select purge.

• Last, make sure to pack all your data into the blend file.

• Go to File, then External Data, then Pack All Into .blend

• You can assign an object a name in the blender outliner.

• If you have multiple objects with the same name you can use a "." as a delineator and assign an index number. Seahaven will only look at the prefix if you try to modify these objects later with a manipulator.

  • This is a great way to manipulate large groups of objects.

  • In this example, the user is using a Blender empty to define a possible location to place a person.

The workflow is the container where you will be defining all object relationships.

In the Relationship Editor tab you will designate the relationships of your 3D assets to each other and elements in the scene (e.g. ground_plane):

• Workflow: The workflow is the space where you will be defining all object relationships.

• Load Collection: This module is used to place collections of 3D assets into the scene and to define their relationship to other objects and elements using a parent/child system.

• Locate: This module is used for designating the range of placement in the scene via a vector.

• Rotate: This module is used for rotating an object, rotation can be defined as a range or as a static value. Units are in degrees.

• Neighbor: This module will place an object to a specific side of the parent object. You can also declare an offset as a range.

• Drop: This module will place an object directly onto a parent object or element.

• Center: This module will center an object on a particular axis.

Key Concepts

Front, Back, Left, Right

Many of the placement rules will make reference to "front" "back" "right" or "left"

These tags refer to sides of the parent objects bounding box. These tags are designed to be intuitive and will help you quickly define the approximate relationship between objects.

Local Coordinates

If an element is dependent on another element such as placing a window or a door on a wall you will define that object's location in local (U,V) coordinates.

• Place the Neighbour module in the Placements bay

• Use the drop down menu to select the oriention to the parent object to place the child object

• Create an offset range. The distance is in meters.

• Start by dragging the Colormap Workflow from the Start Workflow menu. This is the bay you will be placing all of your colormaps into.

• To create a new category, drag the Category module out of the menu and place it in the Colormap Workflow bay.

• Start by dragging the Colormap Workflow from the Start Workflow menu. This is the bay you will be placing all of your categories into.

• To create a new category, drag the Category module out of the menu and place it in the Colormap Workflow bay.

• Each category should have a corresponding collection, and share the collection name. Category names will be used for annotation as well as placement in the relationship editor.

• Select the color you wish to have for you collection, repeated colors are not recommended.

• It is also possible to define a category and supercategory. In COCO JSON a supercatgory is a group of categories. For example, you might want all the screws to be under a supercategory called "hardware."

• Choose the name of your colormap and click "Add".

Define a scale factor in the input field.

Place a child object on top of a parent object. You can then project the child object to the parent within a range of local (u,v) coordinates.

Select the two sides of the objects you want to join. You can then project the child object to the parent within a range of local (u,v) coordinates.

• Bring your Load Collection module into the Workflow bay

• The top two drop-down menus in this module are used to define the parent/child relationship. In the box on the left select the name of the parent object, and the box on the right will have the name of the child object

• The Placement bay is where you will be placing the Relationship modules, which specify the configuration of the relationship between the parent and child object.

• Use item count to designate the range of model numbers to sample. Use a min/max value.

Align an object to a specific side of the parent object.

• Scene: Scene modules are used to configure your scene context. This will include importing/generating 3D environments, backgrounds, and/or lighting.

• Relationships: Relationships between classes of objects defined in the relationship editor are imported into your workflow here.

• Camera: Camera type, position, and camera properties are all defined in this section.

• Manipulators: Manipulators can be used to transform and randomize your scene each time a data point is generated.

• Colormap: Import your color map and define the labels for your COCO JSON file.

• Additional Output: Additional render passes are defined in this section. For example, a depth map would be defined in this workflow portion.

• Annotation Settings: Any additional limitations or transformations you might want to apply to the data annotations are defined here.

• Post-Processing: Applies post-processing effects to images, such as adding shot noise.

• Raytrace Samples: Number of paths to trace for each pixel in the RGB images. Increasing the number of samples improves render quality, but takes more time.

• Stereoscopic: Renders a stereoscopic image (left and right).

• Exclude Annotations: If checked, RGB images will be generated without any annotations.

• For example, in this simulation, the user wanted the camera to follow the perimeter of a building and they drew a curve to represent the range of positions for the camera to take.

• You can name the curve whatever you like. However, be sure to be consistent to assign the name in your camera module later.

Collections are an important part of Seahaven. Collections not only contain groupings of 3D assets but the names will be used to manage annotations and other aspects of your simulations. Please name your collections carefully.

• First, designate a Collection name on the top left-hand side of the upload tool. Each collection is comprised of the files you want to assign to a specific category.

• Drag and drop your .blend files into the upload tool and select upload when they have finished processing.

• This Collection should now be available for placement in the Relationship Editor

X and Y Axis Count : These fields accept min/max variable blocks and set the number of elements in each direction.

X and Y Distance: These fields accept min/max variable blocks and set the distance between elements in each direction.

The number of samples determines how many unique items to sample out of the collection.

For example, if we have a collection with four different 3D models of fruit (apple, orange, pear, pineapple), if you set the samples to 2, it will randomly select two from that collection. If "Load Duplicate" is enabled, the same asset may be loaded more than once.

To select a collection, use a "Collection" block from the "Variables" submenu.

Shadow Plane: If "Shadow Plane" is enabled objects will cast a shadow on the ground plane.

With the HDRI lighting module, you load a variety of pre-loaded HDRI images that you can use as a background in your images. In addition, these images provide physically accurate environmental lighting. Project to ground: You can project your HDRI images to the scenes ground plane. Resulting in a semi-spherical projection. You can learn more about how this feature can be used when creating photorealistic images on our blog. Height: Height refers to the height of the camera when the HDRI was captured. Most HDRI images are taken between 2 and 3 meters off the ground. You can adjust this parameter to reduce distortion around the horizon, which might appear if your camera moves too high above eye level in the z axes.

Intensity: Set a minimum and maximum light intensity. The range will dictate the range of lighting brightness across all the images in your dataset.

Room Type: The room layout to use for the generated scene. Options: (bedroom, livingroom). (See Room Type for more information on room types.)

Room Height: The ceiling height of the generated room (in meters).

Wall type: The material to use for the walls in the room. Options: (Gypsum, Wallpaper).

Floor type: The material to use for the floors in the room. Options: (Oak, Maple, Tile, Ceramic).

Ceiling type: The material to use for the ceilings in the room. Options: (Wood, Gypsum, ACT).

Window type: The window shape to use. The "Tall" preset is approx. 2.3m tall. Options: (Standard, Tall).

Generate recessed lights: If checked, recessed lights will be generated in the ceiling of the room.

Color temp: Color temperature of the lights, in degrees Kelvin. See Wikipedia for more information.

Light strength multiplier: A factor by which to multiply the strength of the recessed lights. Setting both the min and max to 0 will disable the lights.

Room type: Choose from a list of room layout presets including "bedroom" and "living room" (more coming soon).

Object relationships: Object relationships created from the relationship editor which will be used to populate rooms of this type.

• Seahaven currently supports two different camera types (Perspectival and fisheye)

• Each of these camera types can either be positioned by a curve or by using one of the position modules in Seahaven.

• Curves can be imported with the 3D Asset module in the scene menu. For more information, please see the prepare your models section of this documentation.

• Additionally, rotation ranges may be defined for each curve-based camera.

Frames: The number of unique frames to create in the current scene. Each frame represents a unique data point.

Curve name: The name of the curve with which the camera location will be determined.

Yaw (min/max): A range of values, determining the rotation of the camera about its up axis (looking left and right).

Pitch (min/max): A range of values, determining the rotation of the camera about its right axis (looking up and down).

Roll (min/max): A range of values, determining the rotation of the camera about its forward axis (rotating clockwise/counter-clockwise).

Camera Height (min/max): The height of the camera relative to the curve (in meters).

Frames: Number of frames to sample along the curve.

Curve Name: Name of the curve the camera will follow.

Properties: Perspective camera intrinsic properties. Additional documentation.

Shutter Speed: Shutter speed of the camera in seconds.

Camera Height (Min/Max): Height of the camera above the curve in meters.

Curve Parameter Start (Min/Max): A value between 0 and 1 indicating the start position of the camera where 0 is the start of the curve and 1 is the end of the curve.

Curve Parameter Length (Min/Max): A value between 0 and 1 indicating the distance to travel along the curve where 0 is the start of the curve and 1 is the end of the curve.

Frames: Number of frames to sample along the curve.

Curve Name: Name of the curve to sample.

Yaw/Pitch/Roll (Min/Max): Rotation ranges for the camera about its 3 axes. Setting all values to 0 will point the camera straight down. Setting the pitch to 90 degrees will make the camera parallel with the XY/ground plane.

Camera Height (Min/Max): Height of the camera above the curve in meters.

Properties: Perspective camera intrinsic properties. Additional documentation.

Frames: Number of frames to sample in the room.

Position: The location within the room to sample. Options: (Wall, Corner).

Offset Distance: The distance from the camera to the wall (in meters).

Camera Height: Camera height above the floor (in meters).

Yaw/Pitch/Roll: Random rotation along the path (in degrees).

Required Categories: Categories which are required to be visible in each image. Additional documentation.

Properties: Fisheye camera intrinsic properties. Additional documentation.

Minimum Distance to Camera: Restricts objects from being closer than this distance to the camera.

Single Raycast Distance Check: If checked, distance will only be checked from the center of the frame.

Look at: Define where your camera points using one of the blocks from the orientation menu.

Category: Name of the category to check in each image.

Min/Max: The minimum and maximum number of instance allowed of the category in each image.

• Frames: Number of frames to sample in the room. Frames are generated along a random path within the room.

• Control Points: Number of points which are used to create the camera path. Higher numbers increase the complexity of the path.

• Distance Between Points: The min/max distance between consecutive points in the camera path.

• Camera Height: Camera height above the floor (in meters).

• Yaw/Pitch/Roll: Random rotation along the path (in degrees).

• Additional Cameras: Optional list of additional camera positions/orientations to sample per frame. See documentation for additional information.

• Properties: Fisheye camera intrinsic properties. Additional documentation.

Look at: Define where your camera points using one of the blocks from the orientation menu.

The bounding box of all the items in the scene is calculated and the camera will point towards the center of that bounding box.

Look at: Define where your camera points using one of the blocks from the orientation menu.

Import 3D Assets & Relationships: This module is to import relationships between collections that have been defined in the relationships editor.

Manipulators are modules that can be used to transform or manipulate objects that are already loaded into a scene.

• FOV: Sets field of view, degrees

• Sensor Size: The sensor in the camera expressed in millimeters

• Focal Length: The camera focal length expressed in millimeters

Define a target point to point your camera at.

Camera manipulators modify camera extrinsics by translating and rotating the camera with a variety of methods.

FOV (Min/Max): Sets the horizontal field of view (in radians).

F-Stop: The F-Stop (aperture) of the lens. Affects the depth-of-field of the image.

Aperture Blades: Number of blades for the aperture. Affects the shape of the bokeh.

Aperture Ratio: Simulates distortion of bokeh. Values below 1 stretch the shape of the bokeh horizontally while values above 1 will stretch the bokeh vertically.

Camera Near/Far Clip: Sets the distance of the clipping plane.

Match By: Property with which to match an object (Category/Object Name).

Match Name: A string to match (case-insensitive) the object category or name.

Use Partial Match: If checked, all objects with a partial match to the provided name will be sampled.

Override Existing Camera Rotation: If checked, the initial rotation of the camera will be replaced with the provided values.

Initial Rotation (Yaw/Pitch/Roll): Ranges of rotation values to use as the camera's initial rotation. If "Override Existing Camera Rotation" is unchecked, the camera's current rotation values will be used.

Additional Rotation (Yaw/Pitch/Roll): Rotation values which are applied in addition to the initial rotation.

Additional Translation (X/Y/Z): Translation applied after the camera's position has already been overridden.

Object Name: Name of the object. If Bone Names is used, the name of this object should be that of a rig.

Distance to Target (min/max): Distance from the target object (or bone). If both values are set to 0, the distance will not be overridden.

Bone Names: Name(s) of the bones within the specified object to point to.

This block will change the intensity of all light objects in your scene. Please note that this will impact HDRI light sources. IMPORTANT - please make sure the block is always used before the "Near IR" block. The "Near IR" block will add a light to your scene. Modifiers are executed sequentially and if it is executed first the light used in the IR simulation will not be impacted.

Illuminator Power: Radiant power of the point light in Watts.

Material name: The name of the material to be randomized.

Chance: Chance (as a percentage, between 0 and 100) to randomize the material’s color.

Import: Load a material collection into your scene.

Replace by: Select materials to be replaced by either "object name" or by category.

Material name: The name of the material to be randomized.

Property name: Name of the material property to randomize.

Value (min/max): A range of values to uniformly sample for the new property value.

Replace: Enter the material name in your scene file. In the final port on this row input an object name variable here. With: provide a material name with a weight.

Weight defines the likleyhood of this material being used over another one in the list.

This block can be added to change the pose of an object given the name of the rig and a list of poses. There are two variants of this block--the following will only apply a single pose.

Rig Name: The name of the rig to modify.

Poses: A list of poses. See Pose Pair for more information.

This variant of the block will apply multiple poses (one from each "pose set").

Rig Name: The name of the rig to modify.

Pose Sets: A list of pose sets, each of which contain a list of pose pairs. See Pose Set for more information.

This is used in conjunction with the Pose Objects block. This will interpolate between two poses, the amount of which is determined by min and max bias values (as a percentage). The example below will interpolate between open and closed poses between 0% and 25%. A bias below 50% will label the object using the name of Pose A, whereas a bias above 50% will label the object using the name of Pose B.

Pose A: The first pose.

Pose B: The second pose.

Bias Min: Weight toward the first pose.

Bias Max: Weight toward the second pose.

Pose name partial match: If checked, pose names partially matching the given name will also be sampled. For example, using the pose name "sitting" will match all poses containing the string "sitting".

A pose set block is a list of pose pairs, from which one pose pair will be chosen and then applied.

In the image above, two sets of poses will be applied to a single rig, "rig". The first set of poses will apply a pose partially matching the name "_L" (e.g. a left-handed pose).

The second set of poses will apply a pose partially matching the name "_R" (e.g. a right-handed pose).

Child Collection: Collection from which to load a child object.

Parent Object Name: Name of the parent object (case-insensitive).

Parent Bone Name: (Optional) Name of the bone in the parent object. Only applies to rigged objects.

Partial Match Name: If checked, a partial match will be used for the parent object name.

Number of Parents (Min/Max): Number of objects to act as parents.

Number of Children (Min/Max): Number of objects to load as children.

Material collections are created on the "Materials" page. You can upload your own shaders and materials to a designated collection.

You can also transform the items in your scene by rotating them, aligning them to the camera or setting them a minimum distance from your camera.

Object Name: Name of the object to delete.

Items: Number of objects to replace in the scene.

Collection: Collection of 3d assets to sample (objects which will be used to replace). If the Load Child Object block is used, this will serve as the parent object.

New Object Name: (Optional) new name for the replaced object.

Rotation (Min/Max): Rotation range, in degrees, about the object's vertical axis.

View settings (optional): A block defining view-related settings. See more.

Load child object (optional): A block which loads and parents a child object. See more.

Target Object Name: Name of the object to populate with particles.

Particle Collection: Collection to instance for unique particles.

Number of Particles (Min/Max): The total number of particles on the target object geometry.

Unique Particle Instance (Min/Max): The number of unique objects to use for the particle system.

Particle Random Scale: A value between 0 and 1 where 0 maintains the original object scale and 1 is the highest level of randomization.

This block loads a child object, using two bones to parent the object.

Load child object from: Collection of 3d assets to sample (objects which will be used to replace).

Parent bone name: Name of the bone in the parent rig.

Child bone name: Name of the bone in the child rig.

Define the object name and what percentage of the items you want to be removed from the scene.

This can be a useful way to randomize a scene by randomly deleting items each time a new data point is generated.

Object Name: Name of the object to delete.

Percentage to delete: Percentage of objects matching the name to delete.

Ignore: Number of objects to skip deleting.

Min/Max Distance: Distance range from the camera in which objects may be ignored.

View Limits X (Min/Max): A pair of values where 0 is the left side of the frame and 1 is the right side of the frame.

View Limits Y (Min/Max): A pair of values where 0 is the bottom of the frame and 1 is the top of the frame.

Align to Camera: If checked, replaced objects will face the camera (affected by rotation values).

Distance to Replace (Min/Max): Distance range from the camera within which objects will be replaced. If both values are set to 0, there will be no restriction.

Replace in View: If checked, only objects within the view of the camera will be replaced. The extents of the view may be restricted using X and Y view limits.

View Limits X (Min/Max): A pair of values where 0 is the left side of the frame and 1 is the right side of the frame.

View Limits Y (Min/Max): A pair of values where 0 is the bottom of the frame and 1 is the top of the frame.

Loads objects along a curve

Curve Name: Name of the curve(s) along which the objects will be loaded.

Object Collection: Collection of objects to load along the curve.

Curve Parameter (Min/Max): Defines the part of the curve to sample where 0 is the start of the curve and 1 is the end of the curve.

Curve Parameter Tolerance: The tolerance (between 0 and 1) to use when sampling points along the curve. Using a larger value will cause objects to be placed farther apart from one another.

Number of Samples (Min/Max): Number of points to sample for placing objects.

Random Rotation (Min/Max): A range of rotation values (in degrees) to apply to the objects after they have been placed and aligned to the curve.

This module will check each data point to see if the annotations meet specific criteria.

You can check if a particular category is visible and/or filter out annotations of that category under a specific size.

Exclude Background: If checked, all annotations with the category "background" will be excluded from the final set of annotations.

Categories for Area Check: (Optional) Filters out annotations based on a defined minimum area of the segmentation (in pixels).

Categories for Unoccluded Bbox: (Optional) Adds information for the size of the unoccluded bounding box of the provided categories.

Note: Intrinsic matrices are currently only supported for perspectival cameras.

Sample output:

Show Bounding Box: Toggle to display 2D bounding boxes around each of the annotations.

Show Label: Toggle to display the category name inside of the annotation's bounding box.

Show Segmentation: Toggle to display segmentation contours.

Show Keypoints: Toggle to display keypoint annotations, if applicable.

Categories to Visualize: The names of the categories to visualize. If left empty, annotations for all categories will be visualized. Category names are provided using a Category Name block under Variables/Text Inputs.

Note: All values are expressed relative to an assumed origin and orientation. In other words, it can be treated as adding cameras to a device (such as a head mounted display), which informs the initial location and orientation.

• Offset: Distance to offset along each axis (in meters) for this specific camera.

• Rotation: Additional rotation to apply to the camera (in degrees).

• Filename Suffix: Suffix to append to each image rendered by this camera. For example, using a suffix of "_L_UP" would change a filename from "rgb_0001.png" to "rgb_0001_L_UP.png"

This data may be used in conjunction with ex/intrinsic matrices.

Check for self-occlusion: If checked, keypoint visibility will also be determined by self-occlusion, e.g. a hand behind one's back, not visible by the camera.

Use helper mesh for self-occlusion: If checked, a simplified version of the object will be used when checking for self-occlusion. In general, this will resolve most false negatives, i.e. a keypoint determined to be occluded when it, in fact, is not.

Self occlusion tolerance: Tolerance (in meters) of the self-occlusion test. Distances below this value will be ignored. For example, setting this to an extremely high value such as 1 meter may cause no keypoints to be labeled as "occluded".

Keypoints are provided in a separate keypoints.json file in global XYZ coordinates (right-handed, Z-up) as well as in image coordinates in the coco_annotations.json file, in standard COCO format.

[
    [
        -0.1620437204837799,
        -0.5179221630096436,
        -0.12565040588378906
    ],
    [
        -0.12105777859687805,
        -0.4973163604736328,
        -0.07849514484405518
    ],
    [
        -0.09404271841049194,
        -0.5151250958442688,
        -0.05913042277097702
    ],
    [
        -0.06482458114624023,
        -0.5248016715049744,
        -0.03984642028808594
    ],
    [
        -0.03244906663894653,
        -0.535769522190094,
        -0.014489471912384033
    ],
    [
        -0.0765962302684784,
        -0.5679489970207214,
        -0.06707155704498291
    ],
    [
        -0.027885496616363525,
        -0.5921242237091064,
        -0.05503487586975098
    ],
    [
        0.0020690560340881348,
        -0.605049192905426,
        -0.04992568492889404
    ],
    [
        0.02948659658432007,
        -0.6149576306343079,
        -0.04585710167884827
    ],
    [
        -0.07202258706092834,
        -0.5843294262886047,
        -0.09811919927597046
    ],
    [
        -0.023500025272369385,
        -0.6105403304100037,
        -0.09183114767074585
    ],
    [
        0.01646476984024048,
        -0.6219707131385803,
        -0.09130734205245972
    ],
    [
        0.04623621702194214,
        -0.631866455078125,
        -0.08960628509521484
    ],
    [
        -0.07942312955856323,
        -0.5902279615402222,
        -0.12521019577980042
    ],
    [
        -0.02996969223022461,
        -0.6118033528327942,
        -0.12961751222610474
    ],
    [
        0.010281682014465332,
        -0.6226822733879089,
        -0.13220316171646118
    ],
    [
        0.03999173641204834,
        -0.6305294632911682,
        -0.13364771008491516
    ],
    [
        -0.08599165081977844,
        -0.5793569684028625,
        -0.15705984830856323
    ],
    [
        -0.0427803099155426,
        -0.5936245322227478,
        -0.16432231664657593
    ],
    [
        -0.015383332967758179,
        -0.5988448262214661,
        -0.1668248176574707
    ],
    [
        0.014523148536682129,
        -0.6049606800079346,
        -0.16560997068881989
    ],
    [
        0.10252949595451355,
        -0.5383086204528809,
        -0.12470000982284546
    ],
    [
        0.06510964035987854,
        -0.5131881833076477,
        -0.07174670696258545
    ],
    [
        0.043320655822753906,
        -0.5231153964996338,
        -0.04907417297363281
    ],
    [
        0.007606238126754761,
        -0.535979151725769,
        -0.03419727087020874
    ],
    [
        -0.027878761291503906,
        -0.5478533506393433,
        -0.01286536455154419
    ],
    [
        0.006563246250152588,
        -0.5702809691429138,
        -0.059491097927093506
    ],
    [
        -0.044106364250183105,
        -0.5814489126205444,
        -0.041900694370269775
    ],
    [
        -0.07735973596572876,
        -0.5804719924926758,
        -0.03367578983306885
    ],
    [
        -0.10841655731201172,
        -0.5795875787734985,
        -0.02636873722076416
    ],
    [
        -0.002912282943725586,
        -0.581595242023468,
        -0.09197115898132324
    ],
    [
        -0.052548348903656006,
        -0.6064146757125854,
        -0.0745585560798645
    ],
    [
        -0.09170496463775635,
        -0.6095761060714722,
        -0.06768995523452759
    ],
    [
        -0.12528395652770996,
        -0.6122599840164185,
        -0.06267893314361572
    ],
    [
        0.00454673171043396,
        -0.5865257382392883,
        -0.11947542428970337
    ],
    [
        -0.04839956760406494,
        -0.60919588804245,
        -0.11547577381134033
    ],
    [
        -0.08624148368835449,
        -0.6130762696266174,
        -0.11081117391586304
    ],
    [
        -0.11841791868209839,
        -0.6166180968284607,
        -0.10573410987854004
    ],
    [
        0.008174806833267212,
        -0.5839972496032715,
        -0.1527620553970337
    ],
    [
        -0.029848814010620117,
        -0.6036266684532166,
        -0.1509932279586792
    ],
    [
        -0.05647456645965576,
        -0.6115958094596863,
        -0.15266847610473633
    ],
    [
        -0.08532905578613281,
        -0.6195536255836487,
        -0.15325206518173218
    ]
]

A normal pass may be exported as additional output. Normals are represented in camera space and are provided as RGB values in an EXR image file.

Glare Exposure (Stops): Exposure of the glare effect. Setting a higher value will increase the intensity of the glare in the image.

Noise Amount (min/max): The amount of noise to add, defined as 1/n where n is the average number of photons per fully-illuminated pixel. A value of 0 adds no noise.

Chance to add noise: Chance that noise will be added to a given image.

• Name your simulation, give it a description and designate the number of Datapoints you wish to produce. Select Run Simulation

• Number of Compute Nodes tells Seahaven how many GPUs to use for this simulation.

• Navigate to the Simulation Manager tab. Here you will see your simulation in the que

• Click the play button to start your simulation

Depth Maps are included within your simulation as an additional output. Annotations that fall outside of standard COCO format will be included as additional output.