VXL

This arrow is a collection of vital algorithms implemented with the VXL API

This arrow can be built by enabling the KWIVER_ENABLE_VXL CMake flag

This arrow contains the following functionality:

Aligned Edge Detection Algorithm

class aligned_edge_detection : public kwiver::vital::algo::image_filter 

Extract axis-aligned edges.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const: Check that the algorithm’s current configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Convert to the right type and optionally transform.

class priv: Private implementation class.

Average Frames Algorithm

class average_frames : public kwiver::vital::algo::image_filter 

VXL Frame Averaging Process.

This method contains basic methods for image filtering on top of input images via performing assorted averaging operations.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Average frames temporally.

class priv

Bundle Adjust Algorithm

class bundle_adjust : public kwiver::vital::algo::bundle_adjust

A class for bundle adjustment of feature tracks using VXL.

Public Functions

PLUGGABLE_IMPL (bundle_adjust, "Use VXL (vpgl) to bundle adjust cameras and landmarks.", PARAM_DEFAULT(verbose, bool, "If true, write status messages to the terminal showing " "optimization progress at each iteration", false), PARAM_DEFAULT(use_m_estimator, bool, "If true, use a M-estimator for a robust loss function. " "Currently only the Beaton-Tukey loss function is supported.", false), PARAM_DEFAULT(m_estimator_scale, double, "The scale of the M-estimator, if enabled, in pixels. " "Inlier landmarks should project to within this distance " "from the feature point.", 1.0), PARAM_DEFAULT(estimate_focal_length, bool, "If true, estimate a shared intrinsic focal length for all " "cameras. Warning: there is often a depth/focal length " "ambiguity which can lead to long optimizations.", false), PARAM_DEFAULT(normalize_data, bool, "Normalize the data for numerical stability. " "There is no reason not enable this option, except " "for testing purposes.", true), PARAM_DEFAULT(max_iterations, unsigned, "Termination condition: maximum number of LM iterations", 1000), PARAM_DEFAULT(x_tolerance, double, "Termination condition: Relative change is parameters. " "Exit when (mag(delta_params) / mag(params) < x_tol).", 1e-8), PARAM_DEFAULT(g_tolerance, double, "Termination condition: Maximum gradient magnitude. " "Exit when (max(grad_params) < g_tol)", 1e-8)) virtual ~bundle_adjust()=default: Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual void optimize(vital::camera_map_sptr &cameras, vital::landmark_map_sptr &landmarks, vital::feature_track_set_sptr tracks, vital::sfm_constraints_sptr constraints = nullptr) const

Optimize the camera and landmark parameters given a set of feature tracks.

Optimize the camera and landmark parameters given a set of feature tracks

Parameters:

cameras – [inout] the cameras to optimize
landmarks – [inout] the landmarks to optimize
tracks – [in] the feature tracks to use as constraints
metadata – [in] the frame metadata to use as constraints

Close Loops Homography Guided Algorithm

class close_loops_homography_guided : public kwiver::vital::algo::close_loops

Attempts to stitch feature tracks over a long period of time.

This class attempts to make longer-term loop closures by utilizing a variety of techniques, one of which involves using homographies to estimate potential match locations in the past, followed up by additional filtering.

Public Functions

virtual ~close_loops_homography_guided() = default: Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const override

Check that the algorithm’s currently configuration is valid

This checks solely within the provided vital::config_block and not against the current state of the instance. This isn’t static for inheritence reasons.

Parameters:: config – The config block to check configuration of.
Returns:: true if the configuration check passed and false if it didn’t.

virtual vital::feature_track_set_sptr stitch(vital::frame_id_t frame_number, vital::feature_track_set_sptr input, vital::image_container_sptr image, vital::image_container_sptr mask = vital::image_container_sptr()) const

Perform loop closure operation.

Parameters:

frame_number – the frame number of the current frame
input – the input feature track set to stitch
image – image data for the current frame
mask – Optional mask image where positive values indicate regions to consider in the input image.

Returns:

an updated set of feature tracks after the stitching operation

class priv

Private implementation class.

Public Functions

inline bool c_enabled() const: Is long term loop closure enabled?

inline unsigned c_max_checkpoint_frames() const: Maximum past search distance in terms of number of checkpoints.

inline double c_checkpoint_percent_overlap() const: Term which controls when we make new loop closure checkpoints.

inline std::string c_homography_filename() const: Output filename for homographies.

Public Members

checkpoint_buffert buffer: Buffer storing past homographies for checkpoint frames.

vital::algo::compute_ref_homography_sptr ref_computer: Reference frame homography computer.

vital::algo::match_features_sptr matcher: The feature matching algorithm to use.

Color Commonality Filter Algorithm

class color_commonality_filter : public kwiver::vital::algo::image_filter 

VXL Color Commonality Filter.

This method produces an output image where each pixel corresponds to how frequent the pixel’s color is in the entire image.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Perform pixel frequency computation for one frame.

class priv

Convert Image Algorithm

class convert_image : public kwiver::vital::algo::image_filter 

Convert between VXL image formats.

This can be used, for example, to turn a floating point image into a byte image and vice versa.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Set this algorithm’s properties via a config block.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Convert to the right type and optionally transform.

class priv: Private implementation class.

Estimate Canonical Transform Algorithm

class estimate_canonical_transform : public kwiver::vital::algo::estimate_canonical_transform

Algorithm for estimating a canonical transform for cameras and landmarks

A canonical transform is a repeatable transformation that can be recovered from data. In this case we assume at most a similarity transformation. If data sets P1 and P2 are equivalent up to a similarity transformation, then applying a canonical transform to P1 and separately a canonical transform to P2 should bring the data into the same coordinates.

This implementation first fits a “ground” plane to the landmark points using robust estimation methods provided by the rrel library in VXL. It then estimates the remaining degrees of freedom using PCA much like the implementation in the core plugin. The scale is set to normalize the landmarks to unit standard deviation.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Set this algorithm’s properties via a config block.

virtual kwiver::vital::similarity_d estimate_transform(kwiver::vital::camera_map_sptr const cameras, kwiver::vital::landmark_map_sptr const landmarks) const

Estimate a canonical similarity transform for cameras and points

Note

This algorithm does not apply the transformation, it only estimates it.

Parameters:

cameras – The camera map containing all the cameras
landmarks – The landmark map containing all the 3D landmarks

Throws:

algorithm_exception – When the data is insufficient or degenerate.

Returns:

An estimated similarity transform mapping the data to the canonical space.

class priv

Private implementation class.

Public Functions

inline priv(estimate_canonical_transform &parent): Constructor.

Estimate Essential Matrix Algorithm

class estimate_essential_matrix : public kwiver::vital::algo::estimate_essential_matrix

A class that uses 5 pt algorithm to estimate an initial xform between 2 pt sets

Public Functions

PLUGGABLE_IMPL (estimate_essential_matrix, "Use VXL (vpgl) to estimate an essential matrix.", PARAM_DEFAULT(verbose, bool, "If true, write status messages to the terminal showing " "debugging information", false), PARAM_DEFAULT(num_ransac_samples, unsigned, "The number of samples to use in RANSAC", 512)) virtual ~estimate_essential_matrix()=default: Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const: Check that the algorithm’s currently configuration is valid.

virtual vital::essential_matrix_sptr estimate(const std::vector<vital::vector_2d> &pts1, const std::vector<vital::vector_2d> &pts2, const vital::camera_intrinsics_sptr cal1, const vital::camera_intrinsics_sptr cal2, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate an essential matrix from corresponding points

Parameters:

pts1 – [in] the vector or corresponding points from the first image
pts2 – [in] the vector of corresponding points from the second image
cal1 – [in] the intrinsic parameters of the first camera
cal2 – [in] the intrinsic parameters of the second camera
inliers – [out] for each point pa:wir, the value is true if this pair is an inlier to the estimate
inlier_scale – [in] error distance tolerated for matches to be inliers

class priv

Private implementation class.

Public Functions

inline priv(estimate_essential_matrix &parent): Constructor.

Estimate Fundamental Matrix Algorithm

class estimate_fundamental_matrix : public kwiver::vital::algo::estimate_fundamental_matrix

A class that uses 5 pt algorithm to estimate an initial xform between 2 pt sets

Public Functions

PLUGGABLE_IMPL (estimate_fundamental_matrix, "Use VXL (vpgl) to estimate a fundamental matrix.", PARAM_DEFAULT(precondition, bool, "If true, precondition the data before estimating the " "fundamental matrix", true), PARAM_DEFAULT(method, std::string, "Fundamental matrix estimation method to use. " "(Note: does not include RANSAC). Choices are: "+method_converter().element_name_string(), method_converter().to_string(EST_8_POINT))) virtual ~estimate_fundamental_matrix()=default: Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual vital::fundamental_matrix_sptr estimate(const std::vector<vital::vector_2d> &pts1, const std::vector<vital::vector_2d> &pts2, std::vector<bool> &inliers, double inlier_scale = 1.0) const

Estimate an fundamental matrix from corresponding points

Parameters:

pts1 – [in] the vector or corresponding points from the first image
pts2 – [in] the vector of corresponding points from the second image
inliers – [out] for each point pair, the value is true if this pair is an inlier to the estimate
inlier_scale – [in] error distance tolerated for matches to be inliers

Public Static Functions

static void mark_inliers(vital::fundamental_matrix_sptr const &fm, std::vector<vital::vector_2d> const &pts1, std::vector<vital::vector_2d> const &pts2, std::vector<bool> &inliers, double inlier_scale = 1.0)

Test corresponding points against a fundamental matrix and mark inliers

Parameters:

fm – [in] the fundamental matrix
pts1 – [in] the vector or corresponding points from the first image
pts2 – [in] the vector of corresponding points from the second image
inliers – [out] for each point pair, the value is true if this pair is an inlier to the estimate
inlier_scale – [in] error distance tolerated for matches to be inliers

Estimate Homography Algorithm

class estimate_homography : public kwiver::vital::algo::estimate_homography: A class that uses RREL in VXL to estimate a homography from matching 2D points

Estimate Similarity Transform Algorithm

class estimate_similarity_transform : public kwiver::vital::algo::estimate_similarity_transform

VXL implementation of similarity transform estimation.

Public Functions

inline virtual bool check_configuration(vital::config_block_sptr) const

Check that the algorithm’s configuration config_block is valid

This checks solely within the provided config_block and not against the current state of the instance. This isn’t static for inheritance reasons.

Parameters:: config – The config block to check configuration of.
Returns:: true if the configuration check passed and false if it didn’t.

virtual vital::similarity_d estimate_transform(std::vector<vital::vector_3d> const &from, std::vector<vital::vector_3d> const &to) const

Estimate the similarity transform between two corresponding point sets

Parameters:

from – List of length N of 3D points in the from space.
to – List of length N of 3D points in the to space.

Throws:

algorithm_exception – When the from and to points sets are misaligned, insufficient or degenerate.

Returns:

An estimated similarity transform mapping 3D points in the from space to points in the to space (i.e. transforms from into to).

Hashed Image Classifier Filter Algorithm

class hashed_image_classifier_filter : public kwiver::vital::algo::image_filter 

Classify an image of features using a sum of linear classifiers.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Perform per-pixel classification.

class priv: Private implementation class.

High Pass Filter Algorithm

class high_pass_filter : public kwiver::vital::algo::image_filter 

VXL High Pass Filtering Process

This method contains basic methods for high pass image filtering on top of input images

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Perform high pass filtering.

class priv: Private implementation class.

Image I/O Algorithm

class image_io : public kwiver::vital::algo::image_io 

A class for using VXL to read and write images.

Public Functions

PLUGGABLE_IMPL (image_io, "Use VXL (vil) to load and save image files.", PARAM_DEFAULT(force_byte, bool, "When loading, convert the loaded data into a byte " "(unsigned char) image regardless of the source data type. " "Stretch the dynamic range according to the stretch options " "before converting. When saving, convert to a byte image " "before writing out the image", false), PARAM_DEFAULT(auto_stretch, bool, "Dynamically stretch the range of the input data such that " "the minimum and maximum pixel values in the data map to " "the minimum and maximum support values for that pixel " "type, or 0.0 and 1.0 for floating point types. If using " "the force_byte option value map between 0 and 255. " "Warning, this can result in brightness and constrast " "varying between images.", false), PARAM_DEFAULT(manual_stretch, bool, "Manually stretch the range of the input data by " "specifying the minimum and maximum values of the data " "to map to the full byte range", false), PARAM_DEFAULT(intensity_range, array2, "The range of intensity values (min, max) to stretch into " "the byte range. This is most useful when e.g. 12-bit " "data is encoded in 16-bit pixels. Only used when manual_stretch is " "set to true.", array2({ 0, 255 })), PARAM_DEFAULT(split_channels, bool, "When writing out images, if it contains more than one image " "plane, write each plane out as a seperate image file. Also, " "when enabled at read time, support images written out in via " "this method.", false)) virtual ~image_io(): Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const: Check that the algorithm’s currently configuration is valid.

std::vector<std::string> plane_filenames(std::string const &filename) const: Get plane filenames for a given written file.

KD Tree Algorithm

class kd_tree : public kwiver::vital::algo::nearest_neighbors

A feature detector that applies a filter to results.

Public Functions

PLUGGABLE_IMPL (kd_tree, "KD Tree search to find nearest points.") virtual ~kd_tree(): Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const: Check that the algorithm’s currently configuration is valid.

virtual void build(std::vector<vital::point_3d> &points) const

Build the search tree

/param [in] points the set of points to build the search tree from

virtual void find_nearest_point(vital::point_3d &point, int K, std::vector<int> &indices, std::vector<double> &distances) const

Find the K nearest neighbors for target point against the points in the search tree.

/param [in] point the point to search against /param [in] K the number of nearest points for each point /param [out] indices the indices for the K nearest points /param [out] distances the distance to each nearest point

virtual void find_nearest_points(std::vector<vital::point_3d> &vec, int K, std::vector<std::vector<int>> &indices, std::vector<std::vector<double>> &distances) const

Find the K nearest neighbors for multiple target points against the points in the search tree.

/param [in] vec the points to search against /param [in] K the number of nearest points for each point /param [out] indices the indices for the K nearest points /param [out] distances the distance to each nearest point

virtual void find_within_radius(vital::point_3d &point, double r, std::vector<int> &indices) const

Find all the nearest neighbors within a radius of a target point against the points in the search tree.

/param [in] point the points to search against /param [in] r the radius of the sphere to search over /param [out] indices the indices for the nearest points

class priv: Private implementation class.

Match Features Constrained Algorithm

class match_features_constrained : public kwiver::vital::algo::match_features

A match_feature algorithm that uses feature position, orientation, and scale constraints

This matching algorithm assumes that the features to be matched are already somewhat well aligned geometrically. The use cases are very similar images (e.g. adjacent frames of video) and features that have been transformed into approximate alignment by a pre-processing step (e.g. image registration)

This algorithm first reduces the search space for each feature using a search radius in the space of location (and optionally orientation and scale) to find only geometrically nearby features. It then looks at the descriptors for the neighbors and finds the best match by appearance.

Public Functions

PLUGGABLE_IMPL (match_features_constrained, "Use VXL to match descriptors under the constraints of similar geometry " "(rotation, scale, position).", PARAM_DEFAULT(scale_thresh, double, "Ratio threshold of scales between matching keypoints (>=1.0)" " -1 turns scale thresholding off", 2.0), PARAM_DEFAULT(angle_thresh, double, "Angle difference threshold between matching keypoints" " -1 turns angle thresholding off", -1.0), PARAM_DEFAULT(radius_thresh, double, "Search radius for a match in pixels", 200.0)) virtual ~match_features_constrained()=default: Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s configuration vital::config_block is valid.

virtual vital::match_set_sptr match(vital::feature_set_sptr feat1, vital::descriptor_set_sptr desc1, vital::feature_set_sptr feat2, vital::descriptor_set_sptr desc2) const

Match one set of features and corresponding descriptors to another

Parameters:

feat1 – [in] the first set of features to match
desc1 – [in] the descriptors corresponding to feat1
feat2 – [in] the second set fof features to match
desc2 – [in] the descriptors corresponding to feat2

Returns:

a set of matching indices from feat1 to feat2

class priv

Morphology Algorithm

class morphology : public kwiver::vital::algo::image_filter 

Convert between VXL image formats.

This can be used, for example, to turn a floating point image into a byte image and vice versa.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Convert to the right type and optionally transform.

class priv

Optimize Cameras Algorithm

class optimize_cameras : public kwiver::vital::algo::optimize_cameras

Public Functions

virtual void optimize(kwiver::vital::camera_perspective_sptr &camera, const std::vector<vital::feature_sptr> &features, const std::vector<vital::landmark_sptr> &landmarks, kwiver::vital::sfm_constraints_sptr constraints = nullptr) const

Optimize a single camera given corresponding features and landmarks.

Optimize a single camera given corresponding features and landmarks

This function assumes that 2D features viewed by this camera have already been put into correspondence with 3D landmarks by aligning them into two parallel vectors

Parameters:

camera – [inout] The camera to optimize.
features – [in] The vector of features observed by camera to use as constraints.
landmarks – [in] The vector of landmarks corresponding to features.
metadata – [in] The optional metadata to constrain the optimization.

Pixel Feature Extractor Algorithm

class pixel_feature_extractor : public kwiver::vital::algo::image_filter 

Extract multiple features from an image.

class priv

Split Image Algorithm

class split_image : public kwiver::vital::algo::split_image 

A class for drawing various information about feature tracks.

Public Functions

PLUGGABLE_IMPL (split_image, "Split a larger image into multiple smaller images") virtual ~split_image()=default: Destructor.

virtual std::vector<kwiver::vital::image_container_sptr> split(kwiver::vital::image_container_sptr img) const: Split image.

Threshold Algorithm

class threshold : public kwiver::vital::algo::image_filter 

Threshold an image using different schemes.

Use either an absolute threshold or one based on percentiles.

Public Functions

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data): Binarize the image at a given percentile threshold.

class priv

Triangulate Landmarks Algorithm

class triangulate_landmarks : public kwiver::vital::algo::triangulate_landmarks

A class for triangulating landmarks from feature tracks and cameras using VXL

Public Functions

PLUGGABLE_IMPL (triangulate_landmarks, "Use VXL (vpgl) to triangulate 3D landmarks from cameras and tracks.") virtual ~triangulate_landmarks()=default: Destructor.

virtual bool check_configuration(vital::config_block_sptr config) const override: Check that the algorithm’s currently configuration is valid.

virtual void triangulate(vital::camera_map_sptr cameras, vital::feature_track_set_sptr tracks, vital::landmark_map_sptr &landmarks) const

Triangulate the landmark locations given sets of cameras and feature tracks

This function only triangulates the landmarks with indicies in the landmark map and which have support in the tracks and cameras

Parameters:

cameras – [in] the cameras viewing the landmarks
tracks – [in] the feature tracks to use as constraints
landmarks – [inout] the landmarks to triangulate