Core

Class Probablity Filter Algorithm

class class_probablity_filter : public kwiver::vital::algo::detected_object_filter

Filters detections based on class probability.

This algorithm filters out items that are less than the threshold. The following steps are applied to each input detected object set.

1) Select all class names with scores greater than threshold.

2) Create a new detected_object_type object with all selected class names from step 1. The class name can be selected individually or with the keep_all_classes option.

3) The input detection_set is cloned and the detected_object_type from step 2 is attached.

Public Functions

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block

This method returns the required configuration for the algorithm. The implementation of this method should be light-weight and only create and fill in the config block.

This base virtual function implementation returns an empty configuration.

Get this alg’s configuration block

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::detected_object_set_sptr filter(const vital::detected_object_set_sptr input_set) const

Filter set of detected objects.

This method applies a filter to the input set to create an output set. The input set of detections is unmodified.

Parameters

input_set – Set of detections to be filtered.

Returns

Filtered set of detections.

Close Loops Bad Frames Only Algorithm

class close_loops_bad_frames_only : public kwiver::vital::algo::close_loops

Attempts to stitch over incomplete or bad input frames.

This class attempts to only make short term loop closures due to bad or incomplete feature point tracking. It operates on the principle that when a bad frame occurs, there is generally a lower percentage of feature tracks.

Public Functions

PLUGIN_INFO ("bad_frames_only", "Attempts short-term loop closure based on percentage " "of feature points tracked.") close_loops_bad_frames_only()

Default Constructor.

virtual ~close_loops_bad_frames_only() = default

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block

This base virtual function implementation returns an empty configuration block whose name is set to this->type_name.

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algo’s properties via a config block.

Set this algorithm’s properties via a config block

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is’ otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::feature_track_set_sptr stitch(vital::frame_id_t frame_number, vital::feature_track_set_sptr input, vital::image_container_sptr image, vital::image_container_sptr mask = vital::image_container_sptr()) const

Handle track bad frame detection if enabled.

Perform basic shot stitching for bad frame detection

Parameters
  • frame_number[in] the frame number of the current frame

  • input[in] the input feature track set to stitch

  • image[in] image data for the current frame

  • mask[in] Optional mask image where positive values indicate regions to consider in the input image.

Returns

an updated set a feature tracks after the stitching operation

Close Loops Exhaustive Algorithm

class close_loops_exhaustive : public kwiver::vital::algo::close_loops

Attempts to stitch over previous frames.

This class attempts close loops with all previous (or as specified) frames

Public Functions

PLUGIN_INFO ("exhaustive", "Exhaustive matching of all frame pairs, " "or all frames within a moving window.") close_loops_exhaustive()

Default Constructor.

virtual ~close_loops_exhaustive() noexcept

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this alg’s configuration block .

Get this algorithm’s configuration block

This base virtual function implementation returns an empty configuration block whose name is set to this->type_name.

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algo’s properties via a config block.

Set this algorithm’s properties via a config block

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is’ otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::feature_track_set_sptr stitch(vital::frame_id_t frame_number, vital::feature_track_set_sptr input, vital::image_container_sptr image, vital::image_container_sptr mask = vital::image_container_sptr()) const

Exaustive loop closure.

Perform exhaustive stitching

Parameters
  • frame_number[in] the frame number of the current frame

  • input[in] the input feature track set to stitch

  • image[in] image data for the current frame

  • mask[in] Optional mask image where positive values indicate regions to consider in the input image.

Returns

an updated set of feature tracks after the stitching operation

class priv

Private implementation class.

Public Functions

inline priv()

Constructor.

Public Members

size_t match_req

number of feature matches required for acceptance

int num_look_back

Max frames to close loops back to (-1 to beginning of sequence)

vital::algo::match_features_sptr matcher

The feature matching algorithm to use.

Close Loops Keyframe Algorithm

class close_loops_keyframe : public kwiver::vital::algo::close_loops

Attempts to stitch over previous frames.

This class attempts close loops with all previous (or as specified) frames

Public Functions

PLUGIN_INFO ("keyframe", "Establishes keyframes matches to all keyframes.") close_loops_keyframe()

Default Constructor.

virtual ~close_loops_keyframe() noexcept

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this alg’s configuration block .

Get this algorithm’s configuration block

This base virtual function implementation returns an empty configuration block whose name is set to this->type_name.

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algo’s properties via a config block.

Set this algorithm’s properties via a config block

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is’ otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::feature_track_set_sptr stitch(vital::frame_id_t frame_number, vital::feature_track_set_sptr input, vital::image_container_sptr image, vital::image_container_sptr mask = vital::image_container_sptr()) const

Frame stitching using keyframe-base matching.

Perform keyframe guided stitching

Parameters
  • frame_number[in] the frame number of the current frame

  • input[in] the input feature track set to stitch

  • image[in] image data for the current frame

  • mask[in] Optional mask image where positive values indicate regions to consider in the input image.

Returns

an updated set of feature tracks after the stitching operation

class priv

Private implementation class.

Public Functions

inline priv()

Constructor.

Public Members

int match_req

number of feature matches required for acceptance

int search_bandwidth

number of adjacent frames to match

unsigned int min_keyframe_misses

minimum number of keyframe misses before creating a new keyframe

bool stop_after_match

stop matching against additional keyframes if at least one succeeds

std::map<frame_id_t, unsigned int> frame_matches

histogram of matches associated with each frame

std::vector<frame_id_t> keyframe_misses

a collection of recent frame that didn’t match any keyframe

vital::algo::match_features_sptr matcher

The feature matching algorithm to use.

Close Loops Multi Method Algorithm

class close_loops_multi_method : public kwiver::vital::algo::close_loops

Attempts to stitch over incomplete or bad input frames.

This class can run multiple other close_loops algorithm implementations in attempt to accomplish this.

Public Functions

PLUGIN_INFO ("multi_method", "Iteratively run multiple loop closure algorithms.") close_loops_multi_method()

Default Constructor.

virtual ~close_loops_multi_method() = default

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block

This base virtual function implementation returns an empty configuration block whose name is set to this->type_name.

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is’ otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::feature_track_set_sptr stitch(vital::frame_id_t frame_number, vital::feature_track_set_sptr input, vital::image_container_sptr image, vital::image_container_sptr mask = vital::image_container_sptr()) const

Run all internal loop closure algorithms.

Parameters
  • frame_number – the frame number of the current frame

  • image – image data for the current frame

  • input – the input feature track set to stitch

  • mask – Optional mask image where positive values indicate regions to consider in the input image.

Returns

an updated set of feature tracks after the stitching operation

Compute Ref Homography Core Algorithm

class compute_ref_homography_core : public kwiver::vital::algo::compute_ref_homography

Default impl class for mapping each image to some reference image.

This class differs from estimate_homographies in that estimate_homographies simply performs a homography regression from matching feature points. This class is designed to generate different types of homographies from input feature tracks, which can transform each image back to the same coordinate space derived from some initial refrerence image.

This implementation is state-based and is meant to be run in an online fashion, i.e. run against a track set that has been iteratively updated on successive non-regressing frames. This is ideal for when it is desired to compute reference frames on all frames in a sequence.

Public Functions

PLUGIN_INFO ("core", "Default online sequential-frame reference homography estimator.") compute_ref_homography_core()

Default Constructor.

virtual ~compute_ref_homography_core()

Default Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block

This base virtual function implementation returns an empty configuration block whose name is set to this->type_name.

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is’ otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::f2f_homography_sptr estimate(vital::frame_id_t frame_number, vital::feature_track_set_sptr tracks) const

Estimate the transformation which maps some frame to a reference frame

Similarly to track_features, this class was designed to be called in an online fashion for each sequential frame.

Parameters
  • frame_number – frame identifier for the current frame

  • tracks – the set of all tracked features from the image

Returns

estimated homography

class priv

Public Functions

inline bool compute_homography(std::vector<vector_2d> const &pts_src, std::vector<vector_2d> const &pts_dst, homography_sptr &out_h) const

Estimate the homography between two corresponding points sets

Check for homography validity.

Output homography describes transformation from pts_src to pts_dst.

If estimate homography is deemed bad, true is returned and the homography passed to out_h is not modified. If false is returned, the computed homography is valid and out_h is set to the estimated homography.

Public Members

bool use_backproject_error

Should we remove extra points if the backproject error is high?

double backproject_threshold_sqr

Backprojection threshold in terms of L2 distance (number of pixels)

unsigned forget_track_threshold

After how many frames should we forget all info about a track?

unsigned min_track_length

Minimum track length to use for homography regression.

double inlier_scale

The scale of inlier points used for homography calculation.

unsigned minimum_inliers

Minimum points number of matching points between source and reference images when computing homography

track_info_buffer_sptr buffer

Buffer storing track extensions.

algo::estimate_homography_sptr h_estimator

Pointer to homography estimator.

unsigned frames_since_reset

Number of frames since last new reference frame declared.

bool allow_ref_frame_regression

If we should allow reference frame regression or not when determining the earliest reference frame of active tracks.

frame_id_t min_ref_frame

Minimum allowable reference frame. This is updated when homography estimation fails.

Convert Image Bypass Algorithm

class convert_image_bypass : public kwiver::vital::algo::convert_image

A class for bypassing image conversion.

Public Functions

PLUGIN_INFO ("bypass", "Performs no conversion and returns the given image container.") convert_image_bypass()

Default Constructor.

virtual vital::image_container_sptr convert(vital::image_container_sptr img) const

Default image converter ( does nothing )

Default image converter ( does nothing )

Parameters

img[in] image to be converted

Returns

the input image

Detected Object Set Input csv Algorithm

class detected_object_set_input_csv : public kwiver::vital::algo::detected_object_set_input

Public Functions

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual bool read_set(kwiver::vital::detected_object_set_sptr &set, std::string &image_name)

Read next detected object set

This method reads the next set of detected objects from the file. False is returned when the end of file is reached.

Parameters
  • set[out] Pointer to the new set of detections. Set may be empty if there are no detections on an image.

  • image_name[out] Name of the image that goes with the detections. This string may be empty depending on the source format.

Returns

true if detections are returned, false if end of file.

class priv

Expected format:

  • 1: frame number

  • 2: file name

  • 3: TL-x

  • 4: TL-y

  • 5: BR-x

  • 6: BR-y

  • 7: confidence

  • 8,9 : class-name score (this pair may be omitted or may repeat any number of times)

Detected Object Set Input kw18 Algorithm

class detected_object_set_input_kw18 : public kwiver::vital::algo::detected_object_set_input

Public Functions

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual bool read_set(kwiver::vital::detected_object_set_sptr &set, std::string &image_name)

Read next detected object set

This method reads the next set of detected objects from the file. False is returned when the end of file is reached.

Parameters
  • set[out] Pointer to the new set of detections. Set may be empty if there are no detections on an image.

  • image_name[out] Name of the image that goes with the detections. This string may be empty depending on the source format.

Returns

true if detections are returned, false if end of file.

class priv

Detected Object Set Output csv Algorithm

class detected_object_set_output_csv : public kwiver::vital::algo::detected_object_set_output

Public Functions

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual void write_set(const kwiver::vital::detected_object_set_sptr set, std::string const &image_name)

Write detected object set.

This method writes the specified detected object set and image name to the currently open file.

Parameters
  • set – Detected object set

  • image_path – File path to image associated with the detections.

class priv

Detected Object Set Output kw18 Algorithm

class detected_object_set_output_kw18 : public kwiver::vital::algo::detected_object_set_output

Public Functions

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block

This method returns the required configuration for the algorithm. The implementation of this method should be light-weight and only create and fill in the config block.

This base virtual function implementation returns an empty configuration.

Get this alg’s configuration block

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual void write_set(const kwiver::vital::detected_object_set_sptr set, std::string const &image_name)

Write detected object set.

This method writes the specified detected object set and image name to the currently open file.

Parameters
  • set – Detected object set

  • image_path – File path to image associated with the detections.

class priv

This format should only be used for tracks.

  • Column(s) 1: Track-id

  • Column(s) 2: Track-length (# of detections)

  • Column(s) 3: Frame-number (-1 if not available)

  • Column(s) 4-5: Tracking-plane-loc(x,y) (Could be same as World-loc)

  • Column(s) 6-7: Velocity(x,y)

  • Column(s) 8-9: Image-loc(x,y)

  • Column(s) 10-13: Img-bbox(TL_x,TL_y,BR_x,BR_y) (location of top-left & bottom-right vertices)

  • Column(s) 14: Area (0 - when not available)

  • Column(s) 15-17: World-loc(x,y,z) (longitude, latitude, 0 - when not available)

  • Column(s) 18: Timesetamp(-1 if not available)

  • Column(s) 19: Track-confidence(-1_when_not_available)

Dynamic Config None Algorithm

class dynamic_config_none : public kwiver::vital::algo::dynamic_configuration

A class for bypassing image conversion.

Public Functions

PLUGIN_INFO ("none", "Null implementation of dynamic_configuration.\" "This algorithm always returns an empty configuration block.") dynamic_config_none()

Default constructor.

virtual void set_configuration(kwiver::vital::config_block_sptr config)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(kwiver::vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual kwiver::vital::config_block_sptr get_dynamic_configuration()

Return dynamic configuration values

This method returns dynamic configuration values. A valid config block is returned even if there are not values being returned.

Estimate Canonical Transform Algorithm

class estimate_canonical_transform : public kwiver::vital::algo::estimate_canonical_transform

Algorithm for estimating a canonical transform for cameras and landmarks

A canonical transform is a repeatable transformation that can be recovered from data. In this case we assume at most a similarity transformation. If data sets P1 and P2 are equivalent up to a similarity transformation, then applying a canonical transform to P1 and separately a canonical transform to P2 should bring the data into the same coordinates.

This implementation centers the data at the mean of the landmarks. It orients the data using PCA on the landmarks such that the X-axis aligns with the largest principal direction and the Z-axis aligns with the smallest. The data is oriented such that the positive Z axis points toward the mean of the camera centers. The scale is set to normalized the landmarks to unit standard deviation.

Public Functions

PLUGIN_INFO ("core_pca", "Uses PCA to estimate a canonical similarity transform" " that aligns the best fit plane to Z=0") estimate_canonical_transform()

Constructor.

virtual ~estimate_canonical_transform()

Destructor.

estimate_canonical_transform(const estimate_canonical_transform &other)

Copy Constructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid.

virtual kwiver::vital::similarity_d estimate_transform(kwiver::vital::camera_map_sptr const cameras, kwiver::vital::landmark_map_sptr const landmarks) const

Estimate a canonical similarity transform for cameras and points

Note

This algorithm does not apply the transformation, it only estimates it.

Parameters
  • cameras – The camera map containing all the cameras

  • landmarks – The landmark map containing all the 3D landmarks

Throws

algorithm_exception – When the data is insufficient or degenerate.

Returns

An estimated similarity transform mapping the data to the canonical space.

class priv

Private implementation class.

Public Functions

inline priv()

Constructor.

Example Detector Algorithm

class example_detector : public kwiver::vital::algo::image_object_detector

Public Functions

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block

This method returns the required configuration for the algorithm. The implementation of this method should be light-weight and only create and fill in the config block.

This base virtual function implementation returns an empty configuration.

Get this alg’s configuration block

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config_in)

Set this algorithm’s properties via a config block

This method is called to pass a configuration to the algorithm. The implementation of this method should be light-weight and only save the necessary config values. Defer any substantial processing in another method.

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::detected_object_set_sptr detect(vital::image_container_sptr image_data) const

Find all objects on the provided image

This method analyzes the supplied image and along with any saved context, returns a vector of detected image objects.

Parameters

image_data – the image pixels

Returns

vector of image objects found

class priv

Feature Descriptor I/O Algorithm

class feature_descriptor_io : public kwiver::vital::algo::feature_descriptor_io

A class for reading and writing feature and desriptor sets.

Public Functions

PLUGIN_INFO ("core", "Read and write features and descriptor" " to binary files using Cereal serialization.") feature_descriptor_io()

Constructor.

virtual ~feature_descriptor_io()

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

class priv

Public Functions

inline priv()

Constructor.

Filter Features Magnitude Algorithm

class filter_features_magnitude : public kwiver::vital::algo::filter_features

Algorithm that filters features based on feature magnitude.

Public Functions

PLUGIN_INFO ("magnitude", "Filter features using a threshold" " on the magnitude of the detector response function.") filter_features_magnitude()

Constructor.

virtual ~filter_features_magnitude()

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid.

class priv

Private implementation class.

Public Functions

inline priv()

Constructor.

Filter Fatures Scale Algorithm

class filter_features_scale : public kwiver::vital::algo::filter_features

Algorithm that filters features based on feature scale.

Public Functions

PLUGIN_INFO ("scale", "Filter features using a threshold on the scale of the detected features.") filter_features_scale()

Constructor.

virtual ~filter_features_scale()

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid.

class priv

Filter Tracks Algorithm

class filter_tracks : public kwiver::vital::algo::filter_tracks

Algorithm that filters tracks on various attributes.

Public Functions

PLUGIN_INFO ("core", "Filter tracks by track length or matrix matrix importance.") filter_tracks()

Constructor.

virtual ~filter_tracks()

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s configuration config_block is valid.

virtual vital::track_set_sptr filter(vital::track_set_sptr input) const

filter a track set

Parameters

track – set to filter

Returns

a filtered version of the track set

class priv

Private implementation class.

Public Functions

inline priv()

Constructor.

Formulate Query Core Algorithm

Warning

doxygenclass: Cannot find class “kwiver::arrows::core::formulate_query_core” in doxygen xml output for project “kwiver” from directory: ./_build/xml

Hierarchical Bundle Adjust Algorithm

Warning

doxygenclass: Cannot find class “kwiver::arrows::core::hierarchical_bundle_adjust” in doxygen xml output for project “kwiver” from directory: ./_build/xml

Initialize Cameras Landmarks Algorithm

Warning

doxygenclass: Cannot find class “kwiver::arrows::core::initialize_cameras_landmarks” in doxygen xml output for project “kwiver” from directory: ./_build/xml

Match Features Fundamental Matrix Algorithm

class match_features_fundamental_matrix : public kwiver::vital::algo::match_features

Combines a feature matcher, fundamental matrix estimation, and filtering

This is a meta-algorithm for feature matching that combines one other feature matcher with fundamental matrix estimation and feature filtering. The algorithm applies another configurable feature matcher algorithm and then applies a fundamental matrix estimation algorithm to the resulting matches. Outliers to the fit fundamental matrix are discarded from the set of matches.

If a filter_features algorithm is provided, this will be run on the input features before running the matcher.

Public Functions

PLUGIN_INFO ("fundamental_matrix_guided", "Use an estimated fundamental matrix as a geometric filter" " to remove outlier matches.") match_features_fundamental_matrix()

Default Constructor.

virtual ~match_features_fundamental_matrix()

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this alg’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algo’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

virtual vital::match_set_sptr match(vital::feature_set_sptr feat1, vital::descriptor_set_sptr desc1, vital::feature_set_sptr feat2, vital::descriptor_set_sptr desc2) const

Match one set of features and corresponding descriptors to another

Parameters
  • feat1[in] the first set of features to match

  • desc1[in] the descriptors corresponding to feat1

  • feat2[in] the second set of features to match

  • desc2[in] the descriptors corresponding to feat2

Returns

a set of matching indices from feat1 to feat2

class priv

Match Features Homography Algorithm

class match_features_homography : public kwiver::vital::algo::match_features

Combines a feature matchers, homography estimation, and filtering

This is a meta-algorithm for feature matching that combines one or more other feature matchers with homography estimation and feature filtering. The algorithm applies another configurable feature matcher algorithm and then applies a homography estimation algorithm to the resulting matches. Outliers to the fit homography are discarded from the set of matches.

If a second matcher algorithm is provided, this algorithm will warp the feature locations by the estimated homography before applying the second matching algorithm to the aligned points. This approach is useful for finding weak matches that were missed by the first matcher but are easier to detect once approximate location is known. A good choice for the second matcher is vxl::match_features_constrained.

If a filter_features algorithm is provided, this will be run on the input features before running the first matcher. The second matcher will then run on the original unfilter features. This allows, for example, a slower but more robust feature matcher to run on a subset of the strongest feature points in order to quickly establish an and estimated homography. Then a second, fast matcher can pick up the additional weak matches using the constraint that the location in the image is now known approximately.

Public Functions

PLUGIN_INFO ("homography_guided", "Use an estimated homography as a geometric filter" " to remove outlier matches.") match_features_homography()

Default Constructor.

virtual ~match_features_homography()

Destructor.

virtual vital::config_block_sptr get_configuration() const

Get this alg’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algo’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

virtual vital::match_set_sptr match(vital::feature_set_sptr feat1, vital::descriptor_set_sptr desc1, vital::feature_set_sptr feat2, vital::descriptor_set_sptr desc2) const

Match one set of features and corresponding descriptors to another

Parameters
  • feat1[in] the first set of features to match

  • desc1[in] the descriptors corresponding to feat1

  • feat2[in] the second set of features to match

  • desc2[in] the descriptors corresponding to feat2

Returns

a set of matching indices from feat1 to feat2

inline void set_first_feature_matcher(vital::algo::match_features_sptr alg)

Set the feature matching algorithms to use.

inline void set_second_feature_matcher(vital::algo::match_features_sptr alg)

Set the optional second pass feature matching algorithm to use.

inline void set_feature_filter(vital::algo::filter_features_sptr alg)

Set the optional feature filter to use.

inline void set_homography_estimator(vital::algo::estimate_homography_sptr alg)

Set the homography estimation algorithm to use.

class priv

Track Descriptor Set Output csv Algorithm

Warning

doxygenclass: Cannot find class “kwiver::arrows::core::track_descriptor_set_output_csv” in doxygen xml output for project “kwiver” from directory: ./_build/xml

Track Features Core Algorithm

class track_features_core : public kwiver::vital::algo::track_features

A basic feature tracker.

Public Functions

PLUGIN_INFO ("core", "Track features from frame to frame" " using feature detection, matching, and loop closure.") track_features_core()

Default Constructor.

virtual ~track_features_core() noexcept

Destructor.

virtual vital::config_block_sptr get_configuration() const override

Get this alg’s configuration block .

Get this algorithm’s configuration block

This base virtual function implementation returns an empty configuration block whose name is set to this->type_name.

Returns

config_block containing the configuration for this algorithm and any nested components.

virtual void set_configuration(vital::config_block_sptr config) override

Set this algo’s properties via a config block.

Set this algorithm’s properties via a config block

Throws
  • no_such_configuration_value_exception – Thrown if an expected configuration value is not present.

  • algorithm_configuration_exception – Thrown when the algorithm is given an invalid config_block or is’ otherwise unable to configure itself.

Parameters

config – The config_block instance containing the configuration parameters for this algorithm

virtual bool check_configuration(vital::config_block_sptr config) const override

Check that the algorithm’s currently configuration is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters

config – The config block to check configuration of.

Returns

true if the configuration check passed and false if it didn’t.

virtual vital::feature_track_set_sptr track(vital::feature_track_set_sptr prev_tracks, vital::frame_id_t frame_number, vital::image_container_sptr image_data, vital::image_container_sptr mask = {}) const override

Extend a previous set of tracks using the current frame.

Extend a previous set of feature tracks using the current frame

Throws

image_size_mismatch_exception – When the given non-zero mask image does not match the size of the dimensions of the given image data.

Parameters
  • prev_tracks[in] the feature tracks from previous tracking steps

  • frame_number[in] the frame number of the current frame

  • image_data[in] the image pixels for the current frame

  • mask[in] Optional mask image that uses positive values to denote regions of the input image to consider for feature tracking. An empty sptr indicates no mask (default value).

Returns

an updated set of feature tracks including the current frame

class priv

Private implementation class.

Public Functions

inline priv()

Constructor.

Public Members

vital::algo::detect_features_sptr detector

The feature detector algorithm to use.

vital::algo::extract_descriptors_sptr extractor

The descriptor extractor algorithm to use.

vital::algo::feature_descriptor_io_sptr feature_io

The file I/O for feature descriptor caching.

vital::algo::match_features_sptr matcher

The feature matching algorithm to use.

vital::algo::close_loops_sptr closer

The loop closure algorithm to use.

Frame Index Track Set Class

class frame_index_track_set_impl : public kwiver::vital::track_set_implementation

A custom track set implementation that provides fast indexing by frame id

This track_set_implementation is designed to make querying tracks by frame id more efficient. The simple track set must scan every track state of every track to find tracks on a given frame for each request. This implementation caches the mapping from frames to track states for faster retrieval.

Unnamed Group

virtual void insert(vital::track_sptr const &t)

Insert a track shared pointer into this container.

virtual void insert(vital::track_sptr &&t)

Insert a track shared pointer into this container.

Public Functions

frame_index_track_set_impl()

Default Constructor.

explicit frame_index_track_set_impl(const std::vector<vital::track_sptr> &tracks)

Constructor from a vector of tracks.

virtual ~frame_index_track_set_impl() = default

Destructor.

virtual size_t size() const

Return the number of tracks in the set.

virtual bool empty() const

Return whether or not there are any tracks in the set.

virtual bool contains(vital::track_sptr t) const

Return true if the set contains a specific track.

virtual void set_tracks(std::vector<vital::track_sptr> const &tracks)

Assign a vector of track shared pointers to this container.

virtual void notify_new_state(vital::track_state_sptr ts)

Notify the container that a new state has been added to an existing track.

virtual void notify_removed_state(vital::track_state_sptr ts)

Notify the container that a state has been removed from an existing track.

virtual bool remove(vital::track_sptr t)

Remove a track from the set and return true if successful.

virtual std::vector<vital::track_sptr> tracks() const

Return a vector of track shared pointers.

virtual std::set<vital::frame_id_t> all_frame_ids() const

Return the set of all frame IDs covered by these tracks.

virtual std::set<vital::track_id_t> all_track_ids() const

Return the set of all track IDs in this track set.

virtual vital::frame_id_t first_frame() const

Return the first (smallest) frame number containing tracks.

virtual vital::frame_id_t last_frame() const

Return the last (largest) frame number containing tracks.

virtual vital::track_sptr const get_track(vital::track_id_t tid) const

Return the track in this set with the specified id.

Return the track in the set with the specified id.

virtual std::vector<vital::track_sptr> active_tracks(vital::frame_id_t offset = -1) const

Return all tracks active on a frame.

virtual std::vector<vital::track_sptr> inactive_tracks(vital::frame_id_t offset = -1) const

Return all tracks inactive on a frame.

Return all tracks not active on a frame.

virtual std::vector<vital::track_sptr> new_tracks(vital::frame_id_t offset = -1) const

Return all tracks newly initialized on the given frame.

Return all new tracks on a given frame.

virtual std::vector<vital::track_sptr> terminated_tracks(vital::frame_id_t offset = -1) const

Return all tracks terminated on the given frame.

Return all terminated tracks on a given frame.

virtual double percentage_tracked(vital::frame_id_t offset1 = -2, vital::frame_id_t offset2 = -1) const

Return the percentage of tracks successfully tracked between two frames.

Return the percentage of tracks successfully tracked to the next frame.

virtual std::vector<vital::track_state_sptr> frame_states(vital::frame_id_t offset = -1) const

Return a vector of state data corresponding to the tracks on the given frame.

inline virtual vital::track_set_frame_data_map_t all_frame_data() const

Returns all frame data as map of frame index to track_set_frame_data.

virtual vital::track_set_frame_data_sptr frame_data(vital::frame_id_t offset = -1) const

Return the additional data associated with all tracks on the given frame.

virtual bool remove_frame_data(vital::frame_id_t offset)

Removes the frame data for the frame offset.

inline virtual bool set_frame_data(vital::track_set_frame_data_map_t const &fmap)

Set additional frame data associated with all tracks for all frames.

virtual bool set_frame_data(vital::track_set_frame_data_sptr data, vital::frame_id_t offset = -1)

Set additional data associated with all tracks on the given frame.

virtual vital::track_set_implementation_uptr clone(vital::clone_type = vital::clone_type::DEEP) const override

Clone this track set implementation.

Triangulate Landmarks Algorithm

Warning

doxygenclass: Cannot find class “kwiver::arrows::core::triangulate_landmarks” in doxygen xml output for project “kwiver” from directory: ./_build/xml

Video Input Filter Algorithm

class video_input_filter : public kwiver::vital::algo::video_input

A video reader that filters the frames and metadata.

This class implements a video input that down selects frames ready by another video reader. It may down sample the framerate, remove frames before or after indicated frames, etc.

Public Functions

PLUGIN_INFO ("filter", "A video input that calls another video input" " and filters the output on frame range and other parameters.") video_input_filter()

Constructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

virtual void open(std::string name)

Open a video stream.

This method opens the specified video stream for reading. The format of the name depends on the concrete implementation. It could be a file name or it could be a URI.

Capabilities are set in this call, so they are available after.

Note

Once a video is opened, it starts in an invalid state (i.e. before the first frame of video). You must call next_frame() to step to the first frame of video before calling frame_image().

Parameters

video_name – Identifier of the video stream.

Throws

exception – if open failed

virtual void close()

Close video stream.

Close the currently opened stream and release resources. Closing a stream that is already closed does not cause a problem.

virtual bool end_of_video() const

Return end of video status.

This method returns the end-of-video status of the input video. true is returned if the last frame has been returned.

This method will always return false for video streams that have no ability to detect end of video, such as network streams.

Returns

true if at end of video, false otherwise.

virtual bool good() const

Check whether state of video stream is good.

This method checks the current state of the video stream to see if it is good. A stream is good if it refers to a valid frame such that calls to frame_image() and frame_metadata() are expected to return meaningful data. After calling open() the initial video state is not good until the first call to next_frame().

Returns

true if video stream is good, false if not good.

virtual bool seekable() const

Return whether video stream is seekable.

This method returns whether the video stream is seekable.

Returns

true if video stream is seekable, false otherwise.

virtual size_t num_frames() const

Get the number of frames in the video stream.

Get the number of frames available in the video stream.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

the number of frames in the video stream, or 0 if the video stream is not seekable.

virtual bool next_frame(kwiver::vital::timestamp &ts, uint32_t timeout = 0)

Advance to next frame in video stream.

This method advances the video stream to the next frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing either frame number or time or both, depending on the actual implementation.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the video input is already an end, then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual bool seek_frame(kwiver::vital::timestamp &ts, kwiver::vital::timestamp::frame_t frame_number, uint32_t timeout = 0)

Seek to the given frame number in video stream.

This method seeks the video stream to the requested frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing the time.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the frame requested does not exist, then calling this method will return false.

If the video input is not seekable then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • frame_number[in] The frame to seek to.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual kwiver::vital::timestamp frame_timestamp() const

Obtain the time stamp of the current frame.

This method returns the time stamp of the current frame, if any, or an invalid time stamp. The returned time stamp shall have the same value as was set by the most recent call to next_frame().

This method is idempotent. Calling it multiple times without calling next_frame() will return the same time stamp.

Returns

The time stamp of the current frame.

virtual kwiver::vital::image_container_sptr frame_image()

Get current frame from video stream.

This method returns the image from the current frame. If the video input is already an end, then calling this method will return a null pointer.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same image.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Pointer to image container.

virtual kwiver::vital::metadata_vector frame_metadata()

Get metadata collection for current frame.

This method returns the metadata collection for the current frame. It is best to call this after calling next_frame() to make sure the metadata and video are synchronized and that no metadata collections are lost.

Metadata typically occurs less frequently than video frames, so if you call next_frame() and frame_metadata() together while processing a video, there may be times where no metadata is returned. In this case an empty metadata vector will be returned.

Also note that the metadata collection contains a timestamp that can be used to determine where the metadata fits in the video stream.

In video streams without metadata (as determined by the stream capability), this method may return and empty vector, indicating no new metadata has been found.

Calling this method at end of video will return an empty metadata vector.

Metadata is returned as a vector, instead of a single object, to handle cases where there is multiple metadata packets between frames. This can happen in video streams with a fast metadata rate and slow frame rate. Multiple metadata objects can be also returned from video streams that contain metadata in multiple standards, such as MISB-601 and MISB-104.

In cases where there are multiple metadata packets between frames, it is inappropriate for the reader to try to select the best metadata packet. That is why they are all returned.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same metadata.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Vector of metadata pointers.

virtual kwiver::vital::video_raw_metadata_sptr raw_frame_metadata() override

Return implementation-defined data for efficiently copying this frame’s metadata.

Using this method can help avoid the loss of efficiency and fidelity that comes with re-encoding metadata, if no changes to the metadata are to be performed before writing it back out. May return nullptr, indicating the reader does not support this operation.

Returns

Pointer to raw metadata.

virtual kwiver::vital::metadata_map_sptr metadata_map()

Get metadata map for video.

This method returns a metadata map for the video assuming the video is seekable. If the video is not seekable it will return an empty map. Depending on the implementation if the metamap has not been previously requested then the video will have to loop over to create and store the metadata map.

In video streams without metadata (as determined by the stream capability), this method will return an empty map, indicating no metadata has been found.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Map of vectors of metadata pointers.

virtual kwiver::vital::video_settings_uptr implementation_settings() const override

Extract implementation-specific video encoding settings.

The returned structure is intended to be passed to a video encoder of similar implementation so that the output video can be encoded using the settings of the input video.

Returns

Implementation video settings, or nullptr if none are needed.

class priv

Video Input Image_list Algorithm

class video_input_image_list : public kwiver::vital::algo::video_input

Video input using list of images.

This class implements a video input algorithm using a list of images to simulate a video. Only the images are returned. This algorithm produces no metadata.

Example config: select reader type

image_reader:type = vxl

Public Functions

virtual vital::config_block_sptr get_configuration() const override

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config) override

Set this algorithm’s properties via a configuration block .

virtual bool check_configuration(vital::config_block_sptr config) const override

Check that the algorithm’s currently configuration is valid.

virtual void open(std::string list_name) override

Open a list of images.

This method opens the file that contains the list of images. Each image verified to exist at this time.

Parameters

list_name – Name of file that contains list of images.

virtual void close() override

Close video stream.

Close the currently opened stream and release resources. Closing a stream that is already closed does not cause a problem.

virtual bool end_of_video() const override

Return end of video status.

This method returns the end-of-video status of the input video. true is returned if the last frame has been returned.

This method will always return false for video streams that have no ability to detect end of video, such as network streams.

Returns

true if at end of video, false otherwise.

virtual bool good() const override

Check whether state of video stream is good.

This method checks the current state of the video stream to see if it is good. A stream is good if it refers to a valid frame such that calls to frame_image() and frame_metadata() are expected to return meaningful data. After calling open() the initial video state is not good until the first call to next_frame().

Returns

true if video stream is good, false if not good.

virtual bool seekable() const override

Return whether video stream is seekable.

This method returns whether the video stream is seekable.

Returns

true if video stream is seekable, false otherwise.

virtual size_t num_frames() const override

Get the number of frames in the video stream.

Get the number of frames available in the video stream.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

the number of frames in the video stream, or 0 if the video stream is not seekable.

virtual bool next_frame(kwiver::vital::timestamp &ts, uint32_t timeout = 0) override

Advance to next frame in video stream.

This method advances the video stream to the next frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing either frame number or time or both, depending on the actual implementation.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the video input is already an end, then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual bool seek_frame(kwiver::vital::timestamp &ts, kwiver::vital::timestamp::frame_t frame_number, uint32_t timeout = 0) override

Seek to the given frame number in video stream.

This method seeks the video stream to the requested frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing the time.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the frame requested does not exist, then calling this method will return false.

If the video input is not seekable then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • frame_number[in] The frame to seek to.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual kwiver::vital::timestamp frame_timestamp() const override

Obtain the time stamp of the current frame.

This method returns the time stamp of the current frame, if any, or an invalid time stamp. The returned time stamp shall have the same value as was set by the most recent call to next_frame().

This method is idempotent. Calling it multiple times without calling next_frame() will return the same time stamp.

Returns

The time stamp of the current frame.

virtual kwiver::vital::image_container_sptr frame_image() override

Get current frame from video stream.

This method returns the image from the current frame. If the video input is already an end, then calling this method will return a null pointer.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same image.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Pointer to image container.

virtual kwiver::vital::metadata_vector frame_metadata() override

Get metadata collection for current frame.

This method returns the metadata collection for the current frame. It is best to call this after calling next_frame() to make sure the metadata and video are synchronized and that no metadata collections are lost.

Metadata typically occurs less frequently than video frames, so if you call next_frame() and frame_metadata() together while processing a video, there may be times where no metadata is returned. In this case an empty metadata vector will be returned.

Also note that the metadata collection contains a timestamp that can be used to determine where the metadata fits in the video stream.

In video streams without metadata (as determined by the stream capability), this method may return and empty vector, indicating no new metadata has been found.

Calling this method at end of video will return an empty metadata vector.

Metadata is returned as a vector, instead of a single object, to handle cases where there is multiple metadata packets between frames. This can happen in video streams with a fast metadata rate and slow frame rate. Multiple metadata objects can be also returned from video streams that contain metadata in multiple standards, such as MISB-601 and MISB-104.

In cases where there are multiple metadata packets between frames, it is inappropriate for the reader to try to select the best metadata packet. That is why they are all returned.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same metadata.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Vector of metadata pointers.

virtual kwiver::vital::metadata_map_sptr metadata_map() override

Get metadata map for video.

This method returns a metadata map for the video assuming the video is seekable. If the video is not seekable it will return an empty map. Depending on the implementation if the metamap has not been previously requested then the video will have to loop over to create and store the metadata map.

In video streams without metadata (as determined by the stream capability), this method will return an empty map, indicating no metadata has been found.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Map of vectors of metadata pointers.

class priv

Video Input Pos Algorithm

class video_input_pos : public kwiver::vital::algo::video_input

Metadata reader using the AFRL POS file format.

This class implements a video input algorithm that returns only metadata.

The algorithm takes configuration for a directory full of images and an associated directory name for the metadata files. These metadata files have the same base name as the image files.

Public Functions

PLUGIN_INFO ("pos", "Read video metadata in AFRL POS format." " The algorithm takes configuration for a directory full of images" " and an associated directory name for the metadata files." " These metadata files have the same base name as the image files." " Each metadata file is associated with the image file" " of the same base name.") video_input_pos()

Constructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

virtual void open(std::string list_name)

Open a list of images.

This method opens the file that contains the list of images. The individual image names are used to find the associated metadata file in the directory supplied via the configuration.

Parameters

list_name – Name of file that contains list of images.

virtual void close()

Close video stream.

Close the currently opened stream and release resources. Closing a stream that is already closed does not cause a problem.

virtual bool end_of_video() const

Return end of video status.

This method returns the end-of-video status of the input video. true is returned if the last frame has been returned.

This method will always return false for video streams that have no ability to detect end of video, such as network streams.

Returns

true if at end of video, false otherwise.

virtual bool good() const

Check whether state of video stream is good.

This method checks the current state of the video stream to see if it is good. A stream is good if it refers to a valid frame such that calls to frame_image() and frame_metadata() are expected to return meaningful data. After calling open() the initial video state is not good until the first call to next_frame().

Returns

true if video stream is good, false if not good.

virtual bool seekable() const

Return whether video stream is seekable.

This method returns whether the video stream is seekable.

Returns

true if video stream is seekable, false otherwise.

virtual size_t num_frames() const

Get the number of frames in the video stream.

Get the number of frames available in the video stream.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

the number of frames in the video stream, or 0 if the video stream is not seekable.

virtual bool next_frame(kwiver::vital::timestamp &ts, uint32_t timeout = 0)

Advance to next frame in video stream.

This method advances the video stream to the next frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing either frame number or time or both, depending on the actual implementation.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the video input is already an end, then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual bool seek_frame(kwiver::vital::timestamp &ts, kwiver::vital::timestamp::frame_t frame_number, uint32_t timeout = 0)

Seek to the given frame number in video stream.

This method seeks the video stream to the requested frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing the time.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the frame requested does not exist, then calling this method will return false.

If the video input is not seekable then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • frame_number[in] The frame to seek to.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual kwiver::vital::timestamp frame_timestamp() const

Obtain the time stamp of the current frame.

This method returns the time stamp of the current frame, if any, or an invalid time stamp. The returned time stamp shall have the same value as was set by the most recent call to next_frame().

This method is idempotent. Calling it multiple times without calling next_frame() will return the same time stamp.

Returns

The time stamp of the current frame.

virtual kwiver::vital::image_container_sptr frame_image()

Get current frame from video stream.

This method returns the image from the current frame. If the video input is already an end, then calling this method will return a null pointer.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same image.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Pointer to image container.

virtual kwiver::vital::metadata_vector frame_metadata()

Get metadata collection for current frame.

This method returns the metadata collection for the current frame. It is best to call this after calling next_frame() to make sure the metadata and video are synchronized and that no metadata collections are lost.

Metadata typically occurs less frequently than video frames, so if you call next_frame() and frame_metadata() together while processing a video, there may be times where no metadata is returned. In this case an empty metadata vector will be returned.

Also note that the metadata collection contains a timestamp that can be used to determine where the metadata fits in the video stream.

In video streams without metadata (as determined by the stream capability), this method may return and empty vector, indicating no new metadata has been found.

Calling this method at end of video will return an empty metadata vector.

Metadata is returned as a vector, instead of a single object, to handle cases where there is multiple metadata packets between frames. This can happen in video streams with a fast metadata rate and slow frame rate. Multiple metadata objects can be also returned from video streams that contain metadata in multiple standards, such as MISB-601 and MISB-104.

In cases where there are multiple metadata packets between frames, it is inappropriate for the reader to try to select the best metadata packet. That is why they are all returned.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same metadata.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Vector of metadata pointers.

virtual kwiver::vital::metadata_map_sptr metadata_map()

Get metadata map for video.

This method returns a metadata map for the video assuming the video is seekable. If the video is not seekable it will return an empty map. Depending on the implementation if the metamap has not been previously requested then the video will have to loop over to create and store the metadata map.

In video streams without metadata (as determined by the stream capability), this method will return an empty map, indicating no metadata has been found.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Map of vectors of metadata pointers.

class priv

Video Input Split Algorithm

class video_input_split : public kwiver::vital::algo::video_input

Video input that pulls image and metadata inputs from different sources.

This class implements a video input algorithm that holds two other video input algorithms and pulls imagery from one and metadata from the other.

Public Functions

PLUGIN_INFO ("split", "Coordinate two video readers." " One reader supplies the image/data stream." " The other reader supplies the metadata stream.") video_input_split()

Constructor.

virtual vital::config_block_sptr get_configuration() const

Get this algorithm’s configuration block .

virtual void set_configuration(vital::config_block_sptr config)

Set this algorithm’s properties via a config block.

virtual bool check_configuration(vital::config_block_sptr config) const

Check that the algorithm’s currently configuration is valid.

virtual void open(std::string name)

Open a video stream.

This method opens the specified video stream for reading. The format of the name depends on the concrete implementation. It could be a file name or it could be a URI.

Capabilities are set in this call, so they are available after.

Note

Once a video is opened, it starts in an invalid state (i.e. before the first frame of video). You must call next_frame() to step to the first frame of video before calling frame_image().

Parameters

video_name – Identifier of the video stream.

Throws

exception – if open failed

virtual void close()

Close video stream.

Close the currently opened stream and release resources. Closing a stream that is already closed does not cause a problem.

virtual bool end_of_video() const

Return end of video status.

This method returns the end-of-video status of the input video. true is returned if the last frame has been returned.

This method will always return false for video streams that have no ability to detect end of video, such as network streams.

Returns

true if at end of video, false otherwise.

virtual bool good() const

Check whether state of video stream is good.

This method checks the current state of the video stream to see if it is good. A stream is good if it refers to a valid frame such that calls to frame_image() and frame_metadata() are expected to return meaningful data. After calling open() the initial video state is not good until the first call to next_frame().

Returns

true if video stream is good, false if not good.

virtual bool seekable() const

Return whether video stream is seekable.

This method returns whether the video stream is seekable.

Returns

true if video stream is seekable, false otherwise.

virtual size_t num_frames() const

Get the number of frames in the video stream.

Get the number of frames available in the video stream.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

the number of frames in the video stream, or 0 if the video stream is not seekable.

virtual bool next_frame(kwiver::vital::timestamp &ts, uint32_t timeout = 0)

Advance to next frame in video stream.

This method advances the video stream to the next frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing either frame number or time or both, depending on the actual implementation.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the video input is already an end, then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual bool seek_frame(kwiver::vital::timestamp &ts, kwiver::vital::timestamp::frame_t frame_number, uint32_t timeout = 0)

Seek to the given frame number in video stream.

This method seeks the video stream to the requested frame, making the image and metadata available. The returned timestamp is for new current frame.

The timestamp returned may be missing the time.

Calling this method will make a new image and metadata packets available. They can be retrieved by calling frame_image() and frame_metadata().

Check the HAS_TIMEOUT capability from the concrete implementation to see if the timeout feature is supported.

If the frame requested does not exist, then calling this method will return false.

If the video input is not seekable then calling this method will return false.

Parameters
  • ts[out] Time stamp of new frame.

  • frame_number[in] The frame to seek to.

  • timeout[in] Number of seconds to wait. 0 = no timeout.

Throws
  • video_input_timeout_exception – when the timeout expires.

  • video_stream_exception – when there is an error in the video stream.

Returns

true if frame returned, false if end of video.

virtual kwiver::vital::timestamp frame_timestamp() const

Obtain the time stamp of the current frame.

This method returns the time stamp of the current frame, if any, or an invalid time stamp. The returned time stamp shall have the same value as was set by the most recent call to next_frame().

This method is idempotent. Calling it multiple times without calling next_frame() will return the same time stamp.

Returns

The time stamp of the current frame.

virtual kwiver::vital::image_container_sptr frame_image()

Get current frame from video stream.

This method returns the image from the current frame. If the video input is already an end, then calling this method will return a null pointer.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same image.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Pointer to image container.

virtual kwiver::vital::metadata_vector frame_metadata()

Get metadata collection for current frame.

This method returns the metadata collection for the current frame. It is best to call this after calling next_frame() to make sure the metadata and video are synchronized and that no metadata collections are lost.

Metadata typically occurs less frequently than video frames, so if you call next_frame() and frame_metadata() together while processing a video, there may be times where no metadata is returned. In this case an empty metadata vector will be returned.

Also note that the metadata collection contains a timestamp that can be used to determine where the metadata fits in the video stream.

In video streams without metadata (as determined by the stream capability), this method may return and empty vector, indicating no new metadata has been found.

Calling this method at end of video will return an empty metadata vector.

Metadata is returned as a vector, instead of a single object, to handle cases where there is multiple metadata packets between frames. This can happen in video streams with a fast metadata rate and slow frame rate. Multiple metadata objects can be also returned from video streams that contain metadata in multiple standards, such as MISB-601 and MISB-104.

In cases where there are multiple metadata packets between frames, it is inappropriate for the reader to try to select the best metadata packet. That is why they are all returned.

This method is idempotent. Calling it multiple times without calling next_frame() will return the same metadata.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Vector of metadata pointers.

virtual kwiver::vital::metadata_map_sptr metadata_map()

Get metadata map for video.

This method returns a metadata map for the video assuming the video is seekable. If the video is not seekable it will return an empty map. Depending on the implementation if the metamap has not been previously requested then the video will have to loop over to create and store the metadata map.

In video streams without metadata (as determined by the stream capability), this method will return an empty map, indicating no metadata has been found.

Throws

video_stream_exception – when there is an error in the video stream.

Returns

Map of vectors of metadata pointers.

virtual kwiver::vital::video_settings_uptr implementation_settings() const override

Extract implementation-specific video encoding settings.

The returned structure is intended to be passed to a video encoder of similar implementation so that the output video can be encoded using the settings of the input video.

Returns

Implementation video settings, or nullptr if none are needed.

class priv