Detector Data Types and Related Algorithms¶
Detected Object Set¶
-
class
detected_object_set
: kwiver::vital::noncopyable¶ Set of detected objects.
This class represents a ordered set of detected objects. The detections are ordered on their basic confidence value.
Reentrancy considerations: Typical usage for a set is for a single detector thread to create a set. It is possible to have an application where two threads are accessing the same set concurrently.
Unnamed Group
-
detected_object_set::iterator
begin
()¶ Detected object set iterators;.
This method returns an iterator for the set of detected objects. The iterator points to a shared pointer to a detected object.
- Return
- An iterator over the objects in this set;
Unnamed Group
-
detected_object_sptr
at
(size_t pos)¶ Return pointer to detected object at specified index.
Returns a reference to the element at specified location pos, with bounds checking.
If pos is not within the range of the container, an exception of type std::out_of_range is thrown.
- Return
- Shared pointer to specified element.
- Parameters
pos
: Position of element to return (from zero).
- Exceptions
std::range
: if position is now within the range of objects in container.
Public Functions
-
detected_object_set
()¶ Create an empty detection set.
This CTOR creates an empty detection set. Detections can be added with the add() method.
-
detected_object_set
(std::vector<detected_object_sptr> const &objs)¶ Create new set of detected objects.
This CTOR creates a detection set using the supplied vector of detection objects. This can be used to create a new detection set from the output of a select() method.
- Parameters
objs
: Vector of detected objects.
-
detected_object_set_sptr
clone
() const¶ Create deep copy.
This method creates a deep copy of this object.
- Return
- Managed copy of this object.
-
void
add
(detected_object_sptr object)¶ Add detection to set.
This method adds a new detection to this set.
- Parameters
object
: Detection to be added to set.
-
void
add
(detected_object_set_sptr detections)¶ Add detection set to set.
This method adds a new detection set to this set.
- Parameters
detections
: Detection set to be added to set.
-
size_t
size
() const¶ Get number of detections in this set.
This method returns the number of detections in the set.
- Return
- Number of detections.
-
bool
empty
() const¶ Returns whether or not this set is empty.
This method returns true if the set is empty, false otherwise.
- Return
- Whether or not the set is empty.
-
detected_object_set_sptr
select
(double threshold = detected_object_type::INVALID_SCORE) const¶ Select detections based on confidence value.
This method returns a vector of detections ordered by confidence value, high to low. If the optional threshold is specified, then all detections from the set that are less than the threshold are not in the selected set. Note that the selected set may be empty.
The returned vector refers to the actual detections in the set, so if you make changes to the selected set, you are also changing the object in the set. If you want a clean set of detections, call clone() first.
- Return
- List of detections.
- Parameters
threshold
: Select all detections with confidence not less than this value. If this parameter is omitted, then all detections are selected.
-
detected_object_set_sptr
select
(const std::string &class_name, double threshold = detected_object_type::INVALID_SCORE) const¶ Select detections based on class_name.
This method returns a vector of detections that have the specified class_name. These detections are ordered by descending score for the name. Note that the selected set may be empty.
The returned vector refers to the actual detections in the set, so if you make changes to the selected set, you are also changing the object in the set. If you want a clean set of detections, call clone() first.
- Return
- List of detections.
- Parameters
class_name
: class namethreshold
: Select all detections with confidence not less than this value. If this parameter is omitted, then all detections with the label are selected.
-
void
scale
(double scale_factor)¶ Scale all detection locations by some scale factor.
This method changes the bounding boxes within all stored detections by scaling them by some scale factor.
- Parameters
scale
: Scale factor
-
void
shift
(double col_shift, double row_shift)¶ Shift all detection locations by some translation offset.
This method shifts the bounding boxes within all stored detections by a supplied column and row shift.
Note: Detections in this set can be shared by multiple sets, so shifting the detections in this set will also shift the detection in other sets that share this detection. If this is going to be a problem, clone() this set before shifting.
- Parameters
col_shift
: Column (a.k.a. x, i, width) translation factorrow_shift
: Row (a.k.a. y, j, height) translation factor
-
kwiver::vital::attribute_set_sptr
attributes
() const¶ Get attributes set.
This method returns a pointer to the attribute set that is attached to this object. It is possible that the pointer is NULL, so check before using it.
- Return
- Pointer to attribute set or NULL
-
void
set_attributes
(attribute_set_sptr attrs)¶ Attach attributes set to this object.
This method attaches the specified attribute set to this object.
- Parameters
attrs
: Pointer to attribute set to attach.
-
detected_object_set::iterator
Detected Object¶
-
class
detected_object
¶ Detected object class.
This class represents a detected object in image space.
There is one object of this type for each detected object. These objects are defined by a bounding box in the image space. Each object has an optional classification object attached.
Public Functions
-
detected_object
(const bounding_box_d &bbox, double confidence = 1.0, detected_object_type_sptr classifications = detected_object_type_sptr())¶ Create detected object with bounding box and other attributes.
- Parameters
bbox
: Bounding box surrounding detected object, in image coordinates.confidence
: Detectors confidence in this detection.classifications
: Optional object classification.
-
detected_object_sptr
clone
() const¶ Create a deep copy of this object.
- Return
- Managed copy of this object.
-
bounding_box_d
bounding_box
() const¶ Get bounding box from this detection.
The bounding box for this detection is returned. This box is in image coordinates. A default constructed (invalid) bounding box is returned if no box has been supplied for this detection.
- Return
- A copy of the bounding box.
-
void
set_bounding_box
(const bounding_box_d &bbox)¶ Set new bounding box for this detection.
The supplied bounding box replaces the box for this detection.
- Parameters
bbox
: Bounding box for this detection.
-
double
confidence
() const¶ Get confidence for this detection.
This method returns the current confidence value for this detection. Confidence values are in the range of 0.0 - 1.0.
- Return
- Confidence value for this detection.
-
void
set_confidence
(double d)¶ Set new confidence value for detection.
This method sets a new confidence value for this detection. Confidence values are in the range of [0.0 - 1.0].
- Parameters
d
: New confidence value for this detection.
-
uint64_t
index
() const¶ Get detection index.
This method returns the index for this detection.
The detection index is a general purpose field that the application can use to individually identify a detection. In some cases, this field can be used to correlate the detection of an object over multiple frames.
- Return
- Detection index fof this detections.
-
void
set_index
(uint64_t idx)¶ Set detection index.
This method sets tne index value for this detection.
The detection index is a general purpose field that the application can use to individually identify a detection. In some cases, this field can be used to correlate the detection of an object over multiple frames.
- Parameters
idx
: Detection index.
-
const std::string &
detector_name
() const¶ Get detector name.
This method returns the name of the detector that created this element. An empty string is returned if the detector name is not set.
- Return
- Name of the detector.
-
void
set_detector_name
(const std::string &name)¶ Set detector name.
This method sets the name of the detector for this detection.
- Parameters
name
: Detector name.
-
detected_object_type_sptr
type
()¶ Get pointer to optional classifications object.
This method returns the pointer to the classification object if there is one. If there is no classification object the pointer is NULL.
- Return
- Pointer to classification object or NULL.
-
void
set_type
(detected_object_type_sptr c)¶ Set new classifications for this detection.
This method supplies a new set of class_names and scores for this detection.
- Parameters
c
: New classification for this detection
-
image_container_sptr
mask
()¶ Get detection mask image.
This method returns the mask image associated with this detection.
- Return
- Pointer to the mask image.
-
void
set_mask
(image_container_sptr m)¶ Set mask image for this detection.
This method supplies a new mask image for this detection.
- Parameters
m
: Mask image
-
detected_object::descriptor_sptr
descriptor
() const¶ Get descriptor vector.
This method returns an optional descriptor vector that was used to create this detection. This is only set for certain object detectors.
- Return
- Pointer to the descriptor vector.
-
void
set_descriptor
(descriptor_sptr d)¶ Set descriptor for this detection.
This method sets a descriptor vector that was used to create this detection. This is only set for certain object detectors.
- Parameters
d
: Descriptor vector
-
Detected Object Type¶
-
class
detected_object_type
¶ Detected object type classification.
This class represents a set of possible types for the detected object. A type for an object is represented by a class_name string and a score.
When an object is classified, there may be several possibilities determined, since the classification process is probabilistic. This class captures the set of possible types along with the relative likelyhood or score.
Note that score values in this object are not constrained to [0.0,1.0] because different detectors use different approaches for scores. These scores can be normalized, but that is up to the user of these values.
Note that the list of possible names is managed through a class static string pool. Every effort has been made to make this pool externally unmutable. Your cooperation is appreciated.
Public Functions
-
detected_object_type
()¶ Create an empty object.
An object is created without class_names or scores.
-
detected_object_type
(const std::vector<std::string> &class_names, const std::vector<double> &scores)¶ Create new object type class.
Create a new object type instance with a set of labels and likelyhoods. The parameters have corresponding ordering, which means that the first label is for the first likelyhood , and so on.
The number of elements in the parameter vectors must be the same.
- Parameters
class_names
: List of names for the possible classes.scores
: Vector of scores for this object.*
- Exceptions
std::invalid_argument
: if the vector lengths differ
-
detected_object_type
(const std::string &class_name, double score)¶ Create new object type class.
Create a new object type instance from a single class name and label.
- Parameters
class_name
: Class namescore
: Probability score for the class
-
bool
has_class_name
(const std::string &class_name) const¶ Determine if class-name is present.
This method determines if the specified class name is present in this object.
- Return
- true if class name is present.
- Parameters
class_name
: Class name to test.
-
double
score
(const std::string &class_name) const¶ Get score for specific class_name.
This method returns the score for the specified class_name. If the name is associated with this object, an exception is thrown.
- Return
- Score for selected class_name.
- Parameters
class_name
: Return score for this entry.
- Exceptions
std::runtime_error
: If supplied class_name is not associated with this object.
-
void
get_most_likely
(std::string &max_name) const¶ Get max class name.
This method returns the most likely class for this object.
If there are no scores associated with this detection, then an exception is thrown
- Parameters
max_name
: Class name with the maximum score.
- Exceptions
std::runtime_error
: If no scores are associated with this detection.
-
void
get_most_likely
(std::string &max_name, double &max_score) const¶ Get max score and name.
This method returns the maximum score or the most likely class for this object. The score value and class_name are returned.
If there are no scores associated with this detection, then an exception is thrown
- Parameters
max_name
: Class name with the maximum score.max_score
: maximum score
- Exceptions
std::runtime_error
: If no scores are associated with this detection.
-
void
set_score
(const std::string &class_name, double score)¶ Set score for a class.
This method sets or updates the score for a type name. Note that the score value is not constrained to [0.0,1.0].
If the class_name specified is not previously associated with this object type, it is added, If it is present, the score is updated.
- Parameters
class_name
: Class name.score
: Score value for class_name
-
void
delete_score
(const std::string &class_name)¶ Remove score and class_name.
This method removes the type entry for the specified class_name. An exception is thrown if this detected object type does not have that class_name.
- Parameters
label
: Class name to remove.
- Exceptions
std::runtime_error
: If supplied class_name is not associated with this object.
-
std::vector<std::string>
class_names
(double threshold = INVALID_SCORE) const¶ Get list of class_names for this detection.
This method returns a vector of class_names that apply to this detection. The names are ordered by decreasing score. If an optional threshold value is supplied, then names with a score not less than that value are included in the returned list.
- Return
- Ordered list of class_names. Note that the list may be empty.
- Parameters
threshold
: If a value is supplied, labels with a score below this value are omitted from the returned list.
-
size_t
size
() const¶ Get number of class names on this object.
This method returns the number of class names that are in this object type.
- Return
- Number of registered class names.
-
detected_object_type::class_const_iterator_t
begin
() const¶ Get start iterator to all class/score pairs.
This method returns an iterator that may be used to iterate over all class/score pairs. The order in which items will be seen by this iterator is unspecified.
- Return
- Start iterator to all class/score pairs.
- See
- end
-
detected_object_type::class_const_iterator_t
end
() const¶ Get end iterator to all class/score pairs.
This method returns an iterator that may be used to end iteration over all class/score pairs.
- Return
- End iterator to all class/score pairs.
- See
- begin
Public Static Functions
-
std::vector<std::string>
all_class_names
()¶ Get list of all class_names in use.
This method returns an ordered vector of all class_name strings. This set of strings represents the superset of all class_names used to classify objects. Strings are added to this set when a previously unseen class_name is passed to the CTOR or
- Return
- Vector of class names.
-
Bounding Box¶
-
template <typename T>
classbounding_box
¶ Coordinate aligned bounding box.
This class represents a coordinate aligned box. The coordinate system places the origin in the upper left.
A bounding box must be constructed with the correct geometry. Once created, the geometry can not be altered.
Public Functions
-
bounding_box
(vector_type const &upper_left, vector_type const &lower_right)¶ Create box from two corner points.
- Parameters
upper_left
: Upper left corner of box.lower_right
: Lower right corner of box.
-
bounding_box
(vector_type const &upper_left, T const &width, T const &height)¶ Create box from point and dimensions.
- Parameters
upper_left
: Upper left corner pointwidth
: Width of box.height
: Height of box.
-
bounding_box
(T xmin, T ymin, T xmax, T ymax)¶ Create a box from four coordinates.
- Parameters
xmin
: Minimum x coordinateymin
: Minimum y coordinatexmax
: Maximum x coordinateymax
: Maximum y coordinate
-
vector_type
center
() const¶ Get center coordinate of box.
- Return
- Center coordinate of box.
-
vector_type
upper_left
() const¶ Get upper left coordinate of box.
- Return
- Upper left coordinate of box.
-
vector_type
lower_right
() const¶ Get lower right coordinate of box.
- Return
- Lower right coordinate of box.
-
T
width
() const¶ Get width of box.
- Return
- Width of box.
-
T
height
() const¶ Get height of box.
- Return
- Height of box.
-
double
area
() const¶ Get area of box.
- Return
- Area of box.
-
Descriptor¶
-
class
descriptor
¶ A representation of a feature descriptor used in matching.
Subclassed by kwiver::vital::descriptor_array_of< double >, kwiver::vital::descriptor_array_of< float >, kwiver::vital::descriptor_array_of< T >
Public Functions
-
virtual
~descriptor
()¶ Destructor.
-
virtual std::type_info const &
data_type
() const = 0¶ Access the type info of the underlying data (double or float)
-
virtual std::size_t
size
() const = 0¶ The number of elements of the underlying type.
-
virtual std::size_t
num_bytes
() const = 0¶ The number of bytes used to represent the data.
-
virtual std::vector<byte>
as_bytes
() const = 0¶ Return the descriptor as a vector of bytes.
This should always work, even if the underlying type is not bytes
-
virtual std::vector<double>
as_double
() const = 0¶ Return the descriptor as a vector of doubles.
Return an empty vector if this makes no sense for the underlying type.
-
bool
operator==
(descriptor const &other) const¶ Equality operator.
-
bool
operator!=
(descriptor const &other) const¶ Inequality operator.
-
virtual
Image Object Detector Algorithm¶
Instantiate with:
kwiver::vital::algo::image_object_detector_sptr detector = kwiver::vital::algo::image_object_detector::create("<impl_name>");
Arrow & Configuration | <impl_name> options | CMake Flag to Enable |
---|---|---|
Example | example_detector | KWIVER_ENABLE_ARROWS |
Hough | hough_circle | KWIVER_ENABLE_OPENCV |
Darknet | darknet | KWIVER_ENABLE_DARKNET |
-
class
image_object_detector
: public kwiver::vital::algorithm_def<image_object_detector>¶ Image object detector base class/.
Subclassed by kwiver::vital::algorithm_impl< create_detection_grid, vital::algo::image_object_detector >, kwiver::vital::algorithm_impl< darknet_detector, vital::algo::image_object_detector >, kwiver::vital::algorithm_impl< example_detector, vital::algo::image_object_detector >, kwiver::vital::algorithm_impl< hough_circle_detector, vital::algo::image_object_detector >, kwiver::vital::algorithm_impl< matlab_image_object_detector, vital::algo::image_object_detector >
Public Functions
-
virtual detected_object_set_sptr
detect
(image_container_sptr image_data) const = 0¶ Find all objects on the provided image.
This method analyzes the supplied image and along with any saved context, returns a vector of detected image objects.
- Return
- vector of image objects found
- Parameters
image_data
: the image pixels
Public Static Functions
-
static std::string
static_type_name
()¶ Return the name of this algorithm.
-
virtual detected_object_set_sptr
Train Detector Algorithm¶
Instantiate with:
kwiver::vital::algo::train_detector_sptr trainer = kwiver::vital::algo::train_detector::create("<impl_name>");
Arrow & Configuration | <impl_name> options | CMake Flag to Enable |
---|---|---|
Darknet | darknet | KWIVER_ENABLE_DARKNET |
-
class
train_detector
: public kwiver::vital::algorithm_def<train_detector>¶ An abstract base class for training object detectors.
Subclassed by kwiver::vital::algorithm_impl< darknet_trainer, vital::algo::train_detector >
Public Functions
-
void
train_from_disk
(vital::category_hierarchy_sptr object_labels, std::vector<std::string> train_image_names, std::vector<kwiver::vital::detected_object_set_sptr> train_groundtruth, std::vector<std::string> test_image_names = std::vector<std::string>(), std::vector<kwiver::vital::detected_object_set_sptr> test_groundtruth = std::vector<kwiver::vital::detected_object_set_sptr>())¶ Train a detection model given a list of images and detections.
This varient is geared towards offline training.
- Parameters
object_labels
: object category labels for trainingtrain_image_list
: list of train image filenamestrain_groundtruth
: annotations loaded for each imagetest_image_list
: list of test image filenamestest_groundtruth
: annotations loaded for each image
-
void
train_from_memory
(vital::category_hierarchy_sptr object_labels, std::vector<kwiver::vital::image_container_sptr> train_images, std::vector<kwiver::vital::detected_object_set_sptr> train_groundtruth, std::vector<kwiver::vital::image_container_sptr> test_images = std::vector<kwiver::vital::image_container_sptr>(), std::vector<kwiver::vital::detected_object_set_sptr> test_groundtruth = std::vector<kwiver::vital::detected_object_set_sptr>())¶ Train a detection model given images and detections.
This varient is geared towards online training, and is not required to be defined.
- Exceptions
runtime_exception
: if not defined.
- Parameters
object_labels
: object category labels for trainingtrain_images
: vector of input train imagestrain_groundtruth
: annotations loaded for each train imagetest_images
: optional vector of input test imagestest_groundtruth
: optional annotations loaded for each test image
Public Static Functions
-
static std::string
static_type_name
()¶ Return the name of this algorithm.
-
void
Detected Object Filter Algorithm¶
Instantiate with:
kwiver::vital::algo::detected_object_filter_sptr filter = kwiver::vital::algo::detected_object_filter::create("<impl_name>");
Arrow & Configuration | <impl_name> options | CMake Flag to Enable |
---|---|---|
Core | class_probablity_filter | KWIVER_ENABLE_ARROWS |
-
class
detected_object_filter
: public kwiver::vital::algorithm_def<detected_object_filter>¶ An abstract base class for filtering sets of detected objects.
A detected object filter accepts a set of detections and produces another set of detections. The output set may be different from the input set. It all depends on the actual implementation. In any case, the input detection set shall be unmodified.
Subclassed by kwiver::vital::algorithm_impl< class_probablity_filter, vital::algo::detected_object_filter >
Public Functions
-
virtual detected_object_set_sptr
filter
(const detected_object_set_sptr input_set) const = 0¶ Filter set of detected objects.
This method applies a filter to the input set to create an output set. The input set of detections is unmodified.
- Return
- Filtered set of detections.
- Parameters
input_set
: Set of detections to be filtered.
Public Static Functions
-
static std::string
static_type_name
()¶ Return the name of this algorithm.
-
virtual detected_object_set_sptr
Draw Detected Object Set Algorithm¶
Instantiate with:
kwiver::vital::algo::draw_detected_object_set_sptr draw = kwiver::vital::algo::draw_detected_object_set::create("<impl_name>");
Arrow & Configuration | <impl_name> options | CMake Flag to Enable |
---|---|---|
OpenCV | ocv | KWIVER_ENABLE_OPENCV |
-
class
draw_detected_object_set
: public kwiver::vital::algorithm_def<draw_detected_object_set>¶ An abstract base class for algorithms which draw tracks on top of images in various ways, for analyzing results.
Subclassed by kwiver::vital::algorithm_impl< draw_detected_object_set, vital::algo::draw_detected_object_set >
Public Functions
-
virtual kwiver::vital::image_container_sptr
draw
(kwiver::vital::detected_object_set_sptr detected_set, kwiver::vital::image_container_sptr image) = 0¶ Draw detected object boxes on Image.
This method draws the detections on a copy of the image. The input image is unmodified. The actual boxes that are drawn are controlled by the configuration for the implementation.
- Return
- Image with boxes and other annotations added.
- Parameters
detected_set
: Set of detected objectsimage
: Boxes are drawn in this image
Public Static Functions
-
static std::string
static_type_name
()¶ Return the name of this algorithm.
-
virtual kwiver::vital::image_container_sptr
Detected Object Set Input Algorithm¶
Instantiate with:
kwiver::vital::algo::detected_object_set_input_sptr detec_in = kwiver::vital::algo::detected_object_set_input::create("<impl_name>");
Arrow & Configuration | <impl_name> options | CMake Flag to Enable |
---|---|---|
CSV | csv | KWIVER_ENABLE_ARROWS |
KW18 | kw18 | KWIVER_ENABLE_ARROWS |
-
class
detected_object_set_input
: public kwiver::vital::algorithm_def<detected_object_set_input>¶ Read detected object sets.
This class is the abstract base class for the detected object set writer.
Detection sets from multiple images are stored in a single file with enough information to recreate a unique image identifier, usually the file name, and an associated set of detections.
Subclassed by kwiver::vital::algorithm_impl< detected_object_set_input_csv, vital::algo::detected_object_set_input >, kwiver::vital::algorithm_impl< detected_object_set_input_kpf, vital::algo::detected_object_set_input >, kwiver::vital::algorithm_impl< detected_object_set_input_kw18, vital::algo::detected_object_set_input >, kwiver::vital::algorithm_impl< detected_object_set_input_simulator, vital::algo::detected_object_set_input >
Public Functions
-
void
open
(std::string const &filename)¶ Open a file of detection sets.
This method opens a detection set file for reading.
- Parameters
filename
: Name of file to open
- Exceptions
kwiver::vital::path_not_exists
: Thrown when the given path does not exist.kwiver::vital::path_not_a_file
: Thrown when the given path does not point to a file (i.e. it points to a directory).kwiver::vital::file_not_found_exception
:
-
void
use_stream
(std::istream *strm)¶ Read detections from an existing stream.
This method specifies the input stream to use for reading detections. Using a stream is handy when the detections are available in a stream format.
- Parameters
strm
: input stream to use
-
void
close
()¶ Close detection set file.
The currently open detection set file is closed. If there is no currently open file, then this method does nothing.
-
virtual bool
read_set
(kwiver::vital::detected_object_set_sptr &set, std::string &image_name) = 0¶ Read next detected object set.
This method reads the next set of detected objects from the file. False is returned when the end of file is reached.
- Return
- true if detections are returned, false if end of file.
- Parameters
set
: Pointer to the new set of detections. Set may be empty if there are no detections on an image.image_name
: Name of the image that goes with the detections. This string may be empty depending on the source format.
-
bool
at_eof
() const¶ Determine if input file is at end of file.
This method reports the end of file status for a file open for reading.
- Return
- true if file is at end.
Public Static Functions
-
static std::string
static_type_name
()¶ Return the name of this algorithm.
-
void
Detected Object Set Output Algorithm¶
Instantiate with:
kwiver::vital::algo::detected_object_set_output_sptr detec_out = kwiver::vital::algo::detected_object_set_output::create("<impl_name>");
Arrow & Configuration | <impl_name> options | CMake Flag to Enable |
---|---|---|
CSV | csv | KWIVER_ENABLE_ARROWS |
KW18 | kw18 | KWIVER_ENABLE_ARROWS |
-
class
detected_object_set_output
: public kwiver::vital::algorithm_def<detected_object_set_output>¶ Read and write detected object sets.
This class is the abstract base class for the detected object set reader and writer.
Detection sets from multiple images are stored in a single file with enough information to recreate a unique image identifier, usually the file name, and an associated wet of detections.
Subclassed by kwiver::vital::algorithm_impl< detected_object_set_output_csv, vital::algo::detected_object_set_output >, kwiver::vital::algorithm_impl< detected_object_set_output_kpf, vital::algo::detected_object_set_output >, kwiver::vital::algorithm_impl< detected_object_set_output_kw18, vital::algo::detected_object_set_output >, kwiver::vital::algorithm_impl< matlab_detection_output, vital::algo::detected_object_set_output >
Public Functions
-
void
open
(std::string const &filename)¶ Open a file of detection sets.
This method opens a detection set file for writing.
- Parameters
filename
: Name of file to open
- Exceptions
kwiver::vital::path_not_exists
: Thrown when the given path does not exist.kwiver::vital::path_not_a_file
: Thrown when the given path does not point to a file (i.e. it points to a directory).
-
void
use_stream
(std::ostream *strm)¶ Write detections to an existing stream.
This method specifies the output stream to use for writing detections. Using a stream is handy when the detections output is available in a stream format.
- Parameters
strm
: output stream to use
-
void
close
()¶ Close detection set file.
The currently open detection set file is closed. If there is no currently open file, then this method does nothing.
-
virtual void
write_set
(const kwiver::vital::detected_object_set_sptr set, std::string const &image_path) = 0¶ Write detected object set.
This method writes the specified detected object set and image name to the currently open file.
- Parameters
set
: Detected object setimage_path
: File path to image associated with the detections.
Public Static Functions
-
static std::string
static_type_name
()¶ Return the name of this algorithm.
-
void
Code Example¶
#include "vital/types/image.h"
#include "vital/types/image_container.h"
#include "vital/types/detected_object.h"
#include "vital/types/detected_object_set.h"
#include "vital/algo/image_io.h"
#include "vital/algo/image_object_detector.h"
#include "vital/algo/draw_detected_object_set.h"
#include "vital/algo/detected_object_set_input.h"
#include "vital/algo/detected_object_set_output.h"
#include "vital/plugin_loader/plugin_manager.h"
// We will be calling some OpenCV code, so we need to include
// some OpenCV related files
#include <opencv2/highgui/highgui.hpp>
#include "arrows/ocv/image_container.h"
#include <kwiversys/SystemTools.hxx>
void how_to_part_02_detections()
{
// Initialize KWIVER and load up all plugins
kwiver::vital::plugin_manager::instance().load_all_plugins();
// Many vision algorithms are used to detect and identify items in an image.
// Detectors are any class that implements the kwiver::vital::algo::image_object_detector interface
// In this example we will explore the detection data types.
// In the following section we will create dummy data in the data types in lieu of running a detection algorithm
// First, Load an image (see how_to_part_01_images)
kwiver::vital::algo::image_io_sptr ocv_io = kwiver::vital::algo::image_io::create("ocv");
kwiver::vital::image_container_sptr ocv_img = ocv_io->load("./soda_circles.jpg");
// Now let's run a detection algorithm that comes with kwiver
kwiver::vital::algo::image_object_detector_sptr detector = kwiver::vital::algo::image_object_detector::create("hough_circle");
kwiver::vital::detected_object_set_sptr hough_detections = detector->detect(ocv_img);
// We can take this detection set and create a new image with the detections overlaid on the image
kwiver::vital::algo::draw_detected_object_set_sptr drawer = kwiver::vital::algo::draw_detected_object_set::create("ocv");
drawer->set_configuration(drawer->get_configuration());// This will default the configuration
kwiver::vital::image_container_sptr hough_img = drawer->draw(hough_detections, ocv_img);
// Let's see what it looks like
cv::Mat hough_mat = kwiver::arrows::ocv::image_container::vital_to_ocv(hough_img->get_image(), kwiver::arrows::ocv::image_container::RGB_COLOR);
cv::namedWindow("Hough Detections", cv::WINDOW_AUTOSIZE);// Create a window for display.
cv::imshow("Hough Detections", hough_mat); // Show our image inside it.
cv::waitKey(5);
kwiversys::SystemTools::Delay(2000); // Wait for 2s
cvDestroyWindow("Hough Detections");
// Next, let's look at the detection data structures and we can make them
// General detection data is defined by the detected_object class
// Detectors will take in an image and return a detected_object_set_sptr object
// A detected_object_set_sptr is comprised of the following data:
// A bounding box
// bounding_box_d is a double based box where the top left and bottom right corners are specificed as TODO pixel index?
// The top left corner is the anchor. A bounding_box_i is interger based to associate corners to pixels in the image
kwiver::vital::bounding_box_d bbox1(ocv_img->width()*0.25, ocv_img->height()*0.25,
ocv_img->width()*0.75, ocv_img->height()*0.75);
// The confidence value is the confidence associated with the detection.
// It should be a probability (0..1) that the detector is sure that it has identified what it is supposed to find.
double confidence1 = 1.0;
// A Classification
// The detected_object_type is created by a classifier which is sometimes part of the detector.
// It is a group of name / value pairs.The name being the name of the class.
// The score is the probability that the object is that class.
// It is optional and not required for a detected object although most examples provide one just to be complete.
kwiver::vital::detected_object_type_sptr type1(new kwiver::vital::detected_object_type());
// This can have multiple entries / scores
type1->set_score("car", 0.03);
type1->set_score("fish", 0.52);
type1->set_score("flag pole", 0.23);
// Put it all together to make a detection
kwiver::vital::detected_object_sptr detection1(new kwiver::vital::detected_object(bbox1, confidence1, type1));
detection1->set_detector_name("center");
// Let's add a few more detections to our detection set and write it out in various formats
kwiver::vital::bounding_box_d bbox2(ocv_img->width()*0.05, ocv_img->height()*0.05,
ocv_img->width()*0.55, ocv_img->height()*0.55);
double confidence2 = 0.50;
kwiver::vital::detected_object_type_sptr type2(new kwiver::vital::detected_object_type());
type2->set_score("car", 0.04);
type2->set_score("fish", 0.12);
type2->set_score("flag pole", 0.67);
kwiver::vital::detected_object_sptr detection2(new kwiver::vital::detected_object(bbox2, confidence2, type2));
detection2->set_detector_name("upper left");
kwiver::vital::bounding_box_d bbox3(ocv_img->width()*0.45, ocv_img->height()*0.45,
ocv_img->width()*0.95, ocv_img->height()*0.95);
double confidence3 = 0.75;
kwiver::vital::detected_object_type_sptr type3(new kwiver::vital::detected_object_type());
type3->set_score("car", 0.22);
type3->set_score("fish", 0.08);
type3->set_score("flag pole", 0.07);
kwiver::vital::detected_object_sptr detection3(new kwiver::vital::detected_object(bbox3, confidence3, type3));
detection3->set_detector_name("lower right");
// Group multiple detections for an image in a set object
kwiver::vital::detected_object_set_sptr detections(new kwiver::vital::detected_object_set());
detections->add(detection1);
detections->add(detection2);
detections->add(detection3);
kwiver::vital::image_container_sptr img_detections = drawer->draw(detections, ocv_img);
// Let's see what it looks like
cv::Mat mat = kwiver::arrows::ocv::image_container::vital_to_ocv(img_detections->get_image(), kwiver::arrows::ocv::image_container::RGB_COLOR);
cv::namedWindow("Detections", cv::WINDOW_AUTOSIZE);// Create a window for display.
cv::imshow("Detections", mat); // Show our image inside it.
cv::waitKey(5);
kwiversys::SystemTools::Delay(2000); // Wait for 2s
cvDestroyWindow("Detections");
kwiver::vital::algo::detected_object_set_output_sptr kpf_writer = kwiver::vital::algo::detected_object_set_output::create("kpf_output");
kwiver::vital::algo::detected_object_set_input_sptr kpf_reader = kwiver::vital::algo::detected_object_set_input::create("kpf_input");
if (kpf_writer == nullptr)
{
std::cerr << "Make sure you have built the kpf arrow, which requires fletch to have yaml" << std::endl;
}
else
{
kpf_writer->open("detected_object_set.kpf");
kpf_writer->write_set(detections, "");
// Now let's read the kpf data back in
std::string image_name;
kwiver::vital::detected_object_set_sptr kpf_detections;
kpf_reader->open("detected_object_set.kpf");
kpf_reader->read_set(kpf_detections, image_name);
auto ie = kpf_detections->cend();
for (auto det = kpf_detections->cbegin(); det != ie; ++det)
{
const kwiver::vital::bounding_box_d bbox((*det)->bounding_box());
std::stringstream ss;
ss << "detector_name " << (*det)->detector_name() << "\n"
<< "bounding box :" << "x1(" << bbox.min_x() << ") y1(" << bbox.min_y() << ") x2(" << bbox.max_x() << ") y2(" << bbox.max_y() << ") \n"
<< "confidence : " << (*det)->confidence() << "\n"
<< "classifications : " << "\n";
for (auto t : *(*det)->type())
ss << "\t type : " << t.first << " " << t.second;
std::cout << ss.str();
}
}
}