Image Data Types and Related Algorithms
Image
-
class image
The representation of an in-memory image.
This base image class represents an image with a dynamic data type. The underlying data type can be queried using pixel_traits(). To properly access individual pixels the data type must be known. The templated at<T>() member function provides direct access to pixels. Alternatively, cast the image itself into an image_of object. The typed image_of class is a bit easier to work with once the type is known, but this base class is useful in APIs that may operate on images of various types.
Memory Management
This image class supports two modes of memory management. Either the image owns its memory or it does not. If the image owns its memory the image::memory() function will return a shared pointer to that image_memory object. Otherwise, image::memory() will return nullptr. In both cases, image::first_pixel() returns a pointer to the first pixel of the memory that appears in the image. The address of the first pixel does not need to match the starting address of the image_memory. There can be multiple different views into the same memory (e.g. a cropped image view) and these views may use the same memory object with a different offsets to the first pixel, size, and step parameters.
Typically the image manages its own memory in a reference counted image_memory object. Creating a new image will allocate this memory, which can be accessed from image::memory(). Copying an image will make a shallow copy refering to the same memory object, and the memory will be deleted when all images are done with it.
There is a special constructor that allows construction of an image as a veiw into some existing memory. In this case the image does not own the memory and image::memory() will return nullptr. The user must ensure that the memory exists for the lifetime of the image.
Subclassed by kwiver::vital::image_of< uint8_t >, kwiver::vital::image_of< uint16_t >, kwiver::vital::image_of< T >
Public Functions
-
image(const image_pixel_traits &pt = image_pixel_traits())
Default Constructor.
Default Constructor
- Parameters:
pt – Change the pixel traits of the image
-
image(size_t width, size_t height, size_t depth = 1, bool interleave = false, const image_pixel_traits &pt = image_pixel_traits())
Constructor that allocates image memory.
Constructor that allocates image memory
Create a new blank (empty) image of specified size.
- Parameters:
width – Number of pixels in width
height – Number of pixel rows
depth – Number of image channels
pt – data type traits of the image pixels
interleave – Set if the pixels are interleaved
-
image(const void *first_pixel, size_t width, size_t height, size_t depth, ptrdiff_t w_step, ptrdiff_t h_step, ptrdiff_t d_step, const image_pixel_traits &pt = image_pixel_traits())
Constructor that points at existing memory.
Constructor that points at existing memory
Create a new image from supplied memory.
- Parameters:
first_pixel – Address of the first pixel in the image. This does not have to be the lowest memory address of the image memory.
width – Number of pixels wide
height – Number of pixels high
depth – Number of image channels
w_step – pointer increment to get to next pixel column
h_step – pointer increment to get to next pixel row
d_step – pointer increment to get to next image channel
pt – data type traits of the image pixels
-
image(const image_memory_sptr &mem, const void *first_pixel, size_t width, size_t height, size_t depth, ptrdiff_t w_step, ptrdiff_t h_step, ptrdiff_t d_step, const image_pixel_traits &pt = image_pixel_traits())
Constructor that shares memory with another image.
Constructor that shares memory with another image
Create a new image from existing image.
- Parameters:
mem – Shared memory block to be used
first_pixel – Address of the first pixel in the image. This does not have to be the lowest memory address of the image memory.
width – Number of pixels wide
height – Number of pixels high
depth – Number of image channels
w_step – pointer increment to get to next pixel column
h_step – pointer increment to get to next pixel row
d_step – pointer increment to get to next image channel
pt – data type traits of the image pixels
-
image(const image &other)
Copy Constructor.
Copy Constructor
The new image will share the same memory as the old image
- Parameters:
other – The other image.
-
inline const image_memory_sptr &memory() const
Const access to the image memory
Access to the image memory
In most cases, when interacting with image data, you should use the first_pixel() function instead of memory(). The memory() function provides access to the underlying reference counted memory for advanced memory management applications. It returns a block of data that contains the image somewhere within. The block of data may contain only the pixels in this image, but it could also contain much more hidden data if the image is a crop or subsampling of an original image that was larger. The blocks of memory are typically shared between copies of this image, and each copy may have a different view into the memory.
This function may also return
nullptr
for a valid image that is a view into some external memory (first_pixel() is still valid in this case). Use caution when accessing the memory directly and always check that the memory is notnullptr
. Never assume that the image data must be contained in the memory block returned by this function.
-
inline image_memory_sptr memory()
Access to the image memory
In most cases, when interacting with image data, you should use the first_pixel() function instead of memory(). The memory() function provides access to the underlying reference counted memory for advanced memory management applications. It returns a block of data that contains the image somewhere within. The block of data may contain only the pixels in this image, but it could also contain much more hidden data if the image is a crop or subsampling of an original image that was larger. The blocks of memory are typically shared between copies of this image, and each copy may have a different view into the memory.
This function may also return
nullptr
for a valid image that is a view into some external memory (first_pixel() is still valid in this case). Use caution when accessing the memory directly and always check that the memory is notnullptr
. Never assume that the image data must be contained in the memory block returned by this function.
-
size_t size() const
The size of the image data in bytes.
The size of the image managed data in bytes
This size includes all allocated image memory, which could be larger than width*height*depth*bytes_per_pixel.
Note
This size only accounts for memory which is owned by the image. If this image was constructed as a view into third party memory then the size is reported as 0.
-
inline const void *first_pixel() const
Const access to the pointer to first image pixel
Access to the pointer to first image pixel
Returns a raw void pointer to the first pixel in the image. This is the starting point for iterating through the image using offsets of w_step(), h_step(), and d_step(). See also image_of::first_pixel() for a variant of this function that returns a pointer to the underlying pixel type.
See also
image_of::first_pixel()
Note
the address returned may differ from the starting address returned by image::memory() if the image is a window into a larger block of image memory.
Note
If the address returned is not
nullptr
but image::memory() returnsnullptr
, then this image is a view into external memory not owned by this image object.
-
inline void *first_pixel()
Access to the pointer to first image pixel
Returns a raw void pointer to the first pixel in the image. This is the starting point for iterating through the image using offsets of w_step(), h_step(), and d_step(). See also image_of::first_pixel() for a variant of this function that returns a pointer to the underlying pixel type.
See also
image_of::first_pixel()
Note
the address returned may differ from the starting address returned by image::memory() if the image is a window into a larger block of image memory.
Note
If the address returned is not
nullptr
but image::memory() returnsnullptr
, then this image is a view into external memory not owned by this image object.
-
inline size_t width() const
The width of the image in pixels.
-
inline size_t height() const
The height of the image in pixels.
-
inline size_t depth() const
The depth (or number of channels) of the image.
-
inline const image_pixel_traits &pixel_traits() const
The trait of the pixel data type.
-
inline ptrdiff_t w_step() const
The the step in memory to next pixel in the width direction.
-
inline ptrdiff_t h_step() const
The the step in memory to next pixel in the height direction.
-
inline ptrdiff_t d_step() const
The the step in memory to next pixel in the depth direction.
-
bool is_contiguous() const
Return true if the pixels accessible in this image form a contiguous memory block
-
bool operator==(image const &other) const
Equality operator.
Equality operator
Compares this image to another image to test equality.
See also
For deep equality comparison see equal_content
Note
This function computes only “shallow” equality. That is, the images are considered equal if they point to the same memory and have the dimensions and pixel step sizes. Deep equality testing requires stepping through and testing that the values of each pixel are the same even if the memory and possibly memory layout differ.
- Parameters:
other – image to compare with
-
inline bool operator!=(image const &other) const
Inequality operator
Compares this image to another image to test inequality.
Note
This function computes only “shallow” inequality. Refer to the equality operator (==) for details.
- Parameters:
other – image to compare with
-
template<typename T>
inline T &at(size_t i, size_t j) Access pixels in the first channel of the image
- Parameters:
i – width position (x)
j – height position (y)
-
template<typename T>
inline const T &at(size_t i, size_t j) const Const access pixels in the first channel of the image.
-
template<typename T>
inline T &at(size_t i, size_t j, size_t k) Access pixels in the image (width, height, channel)
-
template<typename T>
inline const T &at(size_t i, size_t j, size_t k) const Const access pixels in the image (width, height, channel)
-
void set_size(size_t width, size_t height, size_t depth)
Set the size of the image.
Set the size of the image.
If the size has not changed, do nothing. Otherwise, allocate new memory matching the new size.
- Parameters:
width – a new image width
height – a new image height
depth – a new image depth
-
image crop(size_t x_offset, size_t y_offset, size_t width, size_t height) const
Get a cropped view of the image.
Get a cropped view of the image.
Get a cropped view of the image. The cropped view shares memory with the original image so no deep copy is done.
- Parameters:
x_offset – start of the crop region in x (width)
y_offset – start of the crop region in y (height)
width – width of the crop region
height – height of the crop region
-
image(const image_pixel_traits &pt = image_pixel_traits())
Time Stamp
-
class timestamp
Frame time.
This class represents a timestamp for a single video frame. The time is stored in micro-seconds and frame numbers start at one.
A timestamp has the notion of valid time and valid frame. This is useful when dealing with interpolated timestamps. In this case, a timestamp may have a time, but no frame.
When comparing timestamps, they must be from the same domain. If not, then they are not comparable and all relative operators return false.
If both timestamps have a time, then they are ordered by that value. If both do not have time but both have frame numbers, they are ordered by frame number. If the timestamps do not have some way of being compared, all relational operators return false.
Public Functions
-
timestamp()
Default constructor.
Created an invalid timestamp.
-
explicit timestamp(time_usec_t t, frame_id_t f)
Constructor.
Creates a valid timestamp with specified time and frame number.
- Parameters:
t – Time for timestamp in micro-seconds
f – Frame number for timestamp
-
inline bool is_valid() const
Is timestamp valid.
Both the time and frame must be set for a timestamp to be totally valid.
- Returns:
true if both time and frame are valid
-
inline bool has_valid_time() const
Timestamp has valid time.
Indicates that the time has been set for this timestamp.
- Returns:
true if time has been set
-
inline bool has_valid_frame() const
Timestamp has valid frame number.
Indicates that the frame number has been set for this timestamp.
- Returns:
true if frame number has been set
-
inline time_usec_t get_time_usec() const
Get time from timestamp.
The time portion of the timestamp is returned in micro-seconds. The value will be undetermined if the timestamp does not have a valid time.
See also
- Returns:
Frame time in micro-seconds
-
double get_time_seconds() const
Get time in seconds.
The time portion of the timestamp is returned in seconds and fractions.
- Returns:
time in seconds.
-
inline frame_id_t get_frame() const
Get frame number from timestamp.
The frame number value from the timestamp is returned. The first frame in a sequence is usually one. The frame number will be undetermined if the timestamp does not have a valid frame number set.
See also
- Returns:
Frame number.
-
timestamp &set_time_usec(time_usec_t t)
Set time portion of timestamp.
- Parameters:
t – Time for frame.
-
timestamp &set_time_seconds(double t)
Set time portion of timestamp.
- Parameters:
t – Time for frame in seconds.
-
timestamp &set_time_domain_index(int dom)
Set time domain index for this timestamp.
- Parameters:
dom – Time domain index
- Returns:
Reference to this object.
-
std::string pretty_print() const
Format object in a readable manner.
This method formats a time stamp in a readable and recognizable manner suitable form debugging and logging.
- Returns:
formatted timestamp
-
timestamp()
Image Container
-
class image_container
An abstract representation of an image container.
This class provides an interface for passing image data between algorithms. It is intended to be a wrapper for image classes in third-party libraries and facilitate conversion between various representations. It provides limited access to the underlying data and is not intended for direct use in image processing algorithms.
Subclassed by kwiver::arrows::gdal::image_container, kwiver::arrows::ocv::image_container, kwiver::arrows::qt::image_container, kwiver::arrows::vxl::image_container, kwiver::vital::simple_image_container
Public Functions
-
virtual ~image_container() = default
Destructor.
-
virtual size_t size() const = 0
The size of the image data in bytes.
The size of the image data in bytes
This size includes all allocated image memory, which could be larger than width*height*depth.
-
virtual size_t width() const = 0
The width of the image in pixels.
-
virtual size_t height() const = 0
The height of the image in pixels.
-
virtual size_t depth() const = 0
The depth (or number of channels) of the image.
-
inline virtual image get_image(unsigned x_offset, unsigned y_offset, unsigned width, unsigned height) const
Get an in-memory image class to access a sub-image of the data.
-
inline virtual metadata_sptr get_metadata() const
Get metadata associated with this image.
-
inline virtual void set_metadata(metadata_sptr md)
Set metadata associated with this image.
-
virtual ~image_container() = default
Image I/O Algorithm
Instantiate with:
kwiver::vital::algo::image_io_sptr img_io = kwiver::vital::algo::image_io::create("<impl_name>");
Arrow & Configuration |
<impl_name> options |
CMake Flag to Enable |
---|---|---|
ocv |
KWIVER_ENABLE_OPENCV |
|
vxl |
KWIVER_ENABLE_VXL |
-
class image_io : public kwiver::vital::algorithm
An abstract base class for reading and writing images
This class represents an abstract interface for reading and writing images.
A note about the basic capabilities:
HAS_TIME - This capability is set to true if the image metadata supplies a timestamp. If a timestamp is supplied, it is made available in the metadata for the image. If the timestamp is not supplied, then the metadata will not have the timestamp set.
Subclassed by kwiver::arrows::gdal::image_io, kwiver::arrows::ocv::image_io, kwiver::arrows::qt::image_io, kwiver::arrows::vxl::image_io
Public Functions
-
kwiver::vital::image_container_sptr load(std::string const &filename) const
Load image from the file
- Throws:
kwiver::vital::path_not_exists – Thrown when the given path does not exist.
kwiver::vital::path_not_a_file – Thrown when the given path does not point to a file (i.e. it points to a directory).
- Parameters:
filename – the path to the file to load
- Returns:
an image container refering to the loaded image
-
void save(std::string const &filename, kwiver::vital::image_container_sptr data) const
Save image to a file
Image file format is based on file extension.
- Throws:
kwiver::vital::path_not_exists – Thrown when the expected containing directory of the given path does not exist.
kwiver::vital::path_not_a_directory – Thrown when the expected containing directory of the given path is not actually a directory.
- Parameters:
filename – the path to the file to save
data – the image container refering to the image to write
-
kwiver::vital::metadata_sptr load_metadata(std::string const &filename) const
Get the image metadata
- Throws:
kwiver::vital::path_not_exists – Thrown when the given path does not exist.
kwiver::vital::path_not_a_file – Thrown when the given path does not point to a file (i.e. it points to a directory).
- Parameters:
filename – the path to the file to read
- Returns:
pointer to the loaded metadata
-
algorithm_capabilities const &get_implementation_capabilities() const
Return capabilities of concrete implementation.
This method returns the capabilities for the current image reader/writer.
- Returns:
Reference to supported image capabilities.
-
kwiver::vital::image_container_sptr load(std::string const &filename) const
Convert Image Algorithm
Instantiate with:
kwiver::vital::algo::convert_image_sptr img_bypas = kwiver::vital::algo::convert_image::create("<impl_name>");
Arrow & Configuration |
<impl_name> options |
CMake Flag to Enable |
---|---|---|
bypass |
KWIVER_ENABLE_ARROWS |
-
class convert_image : public kwiver::vital::algorithm
An abstract base class for converting base image type.
Arrows that implement this interface convert the input image type (e.g. BGR 16) to a different type (e.g. RGB 8). Concrete implementations usually work with a single image representation, such as VXL or OCV.
If you are lookling for an interface for an image transform that will change the value of a pixel, then use the image_filter interface.
Subclassed by kwiver::arrows::core::convert_image_bypass
Public Functions
-
virtual void set_configuration(kwiver::vital::config_block_sptr config)
Set this algorithm’s properties via a config block.
-
virtual bool check_configuration(kwiver::vital::config_block_sptr config) const
Check that the algorithm’s currently configuration is valid.
Check that the algorithm’s current configuration is valid.
-
virtual kwiver::vital::image_container_sptr convert(kwiver::vital::image_container_sptr img) const = 0
Convert image base type.
-
virtual void set_configuration(kwiver::vital::config_block_sptr config)
Image Filter Algorithm
Instantiate with:
kwiver::vital::algo::image_filter_sptr img_filter = kwiver::vital::algo::image_filter::create("<impl_name>");
Arrow & Configuration |
<impl_name> options |
CMake Flag to Enable |
---|---|---|
N/A |
N/A |
N/A |
** Currently there are no arrows implementing the image_filter algorithm **
-
class image_filter : public kwiver::vital::algorithm
Abstract base class for image set filter algorithms.
This interface supports arrows/algorithms that do a pixel by pixel image modification, such as image enhancement. The resultant image must be the same size as the input image.
Subclassed by kwiver::arrows::vxl::aligned_edge_detection, kwiver::arrows::vxl::average_frames, kwiver::arrows::vxl::color_commonality_filter, kwiver::arrows::vxl::convert_image, kwiver::arrows::vxl::hashed_image_classifier_filter, kwiver::arrows::vxl::high_pass_filter, kwiver::arrows::vxl::morphology, kwiver::arrows::vxl::pixel_feature_extractor, kwiver::arrows::vxl::threshold
Public Functions
-
image_filter()
Return the name of this algorithm.
-
virtual kwiver::vital::image_container_sptr filter(kwiver::vital::image_container_sptr image_data) = 0
Filter a input image and return resulting image
This method implements the filtering operation. The method does not modify the image in place. The resulting image must be a newly allocated image which is the same size as the input image.
- Parameters:
image_data – Image to filter.
- Returns:
a filtered version of the input image
-
image_filter()
Split Image Algorithm
Instantiate with:
kwiver::vital::algo::split_image_sptr img_split = kwiver::vital::algo::split_image::create("<impl_name>");
Arrow & Configuration |
<impl_name> options |
CMake Flag to Enable |
---|---|---|
ocv |
KWIVER_ENABLE_OPENCV |
|
vxl |
KWIVER_ENABLE_VXL |
-
class split_image : public kwiver::vital::algorithm
An abstract base class for converting base image type.
Subclassed by kwiver::arrows::ocv::split_image, kwiver::arrows::vxl::split_image
Public Functions
-
virtual std::vector<kwiver::vital::image_container_sptr> split(kwiver::vital::image_container_sptr img) const = 0
Split image.
-
virtual std::vector<kwiver::vital::image_container_sptr> split(kwiver::vital::image_container_sptr img) const = 0
Video Input Algorithm
Instantiate with:
kwiver::vital::algo::video_input_sptr img_bypas = kwiver::vital::algo::video_input::create("<impl_name>");
Arrow & Configuration |
<impl_name> options |
CMake Flag to Enable |
---|---|---|
vidl_ffmpeg |
KWIVER_ENABLE_VXL |
-
class convert_image : public kwiver::vital::algorithm
An abstract base class for converting base image type.
Arrows that implement this interface convert the input image type (e.g. BGR 16) to a different type (e.g. RGB 8). Concrete implementations usually work with a single image representation, such as VXL or OCV.
If you are lookling for an interface for an image transform that will change the value of a pixel, then use the image_filter interface.
Subclassed by kwiver::arrows::core::convert_image_bypass
Public Functions
-
virtual void set_configuration(kwiver::vital::config_block_sptr config)
Set this algorithm’s properties via a config block.
-
virtual bool check_configuration(kwiver::vital::config_block_sptr config) const
Check that the algorithm’s currently configuration is valid.
Check that the algorithm’s current configuration is valid.
-
virtual kwiver::vital::image_container_sptr convert(kwiver::vital::image_container_sptr img) const = 0
Convert image base type.
-
virtual void set_configuration(kwiver::vital::config_block_sptr config)