RodTracker.backend.reconstruction

Classes and methods called in RodTracked GUI for performing 3D reconstruction and tracking of particles, as well as plotting of the resulting metrics (reprojection errors, frame-to-frame distance, etc).

Author: Adrian Niemann (adrian.niemann@ovgu.de)

Date: 2022-2024

class Plotter(data: DataFrame, **kwargs)

Bases: QRunnable

Object to run the generation of 3D reconstruction and tracking plots.

This object should run the generation of evaluation plots for the 3D reconstruction of the whole or a portion of a dataset. It is capable of running it in a thread different from the main thread to not keep the application responsive.

Parameters:

data (DataFrame) – Slice of the main rod position dataset that shall be used for plotting.

data

Slice of the main rod position dataset that will be used for plotting.

Type:

DataFrame

signals

Signals that can be emitted during the running of a Plotter object. Their purpose is to report errors and (intermediate) results.

Type:

PlotterSignals

kwargs

Keyword arguments that will be used by the plotting functions. The currrently recognized keywords are:

"colors", "start_frame", "end_frame", "position_scaling", "cam_ids", "calibration", and "transformation"

Type:

dict

generate_plots(data: DataFrame, **kwargs)

Wrapper function for all plotting functions that unpacks options and then runs the plotting with the appropriate settings.

Parameters:

data (DataFrame) – Slice of the main rod position dataset that will be used for plotting.

Hint

Emits

• PlotterSignals.error (potentially repeatedly)

• PlotterSignals.result_plot (potentially repeatedly)

plot_displacements_3d(data: DataFrame, colors: List[str] | None = None, start_frame: int | None = None, end_frame: int | None = None)

Plot the frame-wise (minimum) displacement per rod and average of rods for multiple colors.

From the 3D positions of rods the between frames displacement is calculated for each of the given rods. Both rod endpoint combinations are used to calculate the displacement and the respective minimum is chosen for plotting. The resulting plot then consists of one line per given particle, as well as, the average displacement of all particles between the frames. One Figure is generated for every color given in colors.

Parameters:

• data (pd.DataFrame) –

  (Slice of) a dataset of rod position data. Must contain the columns:

  color, "particle", "frame", "x1", "y1", "z1", "x2", "y2", "z2"

• colors (List[str], optional) –

  List of colors present in data that for which a plot shall be created. If this is not specified, i.e. set to None, all colors in data will be used.

  By default None.

• start_frame (int, optional) –

  First frame that shall be included in the generated plots. If this is not specified, i.e. set to None, the lowest frame number found in data will be used.

  By default None.

• end_frame (int, optional) –

  Last frame that shall be included in the generated plots. If this is not specified, i.e. set to None, the highest frame number found int data will be used.

  By default None.

See also

ParticleDetection.reconstruct_3D.visualization.displacement_fwise(), ParticleDetection.utils.data_loading.extract_3d_data()

Hint

Emits

• PlotterSignals.result_plot

plot_reprojection_errors(data: DataFrame, cam_ids: List[str], calibration: dict | None = None, position_scaling: float = 1.0, transformation: dict | None = None)

Plot a histogram of reprojection errors.

Plot a histogram of reprojection errors from 3D coordinates to 2D image coordinates for all rods present in the given DataFrame.

Parameters:

• data (pd.DataFrame) –

  (Slice of) a dataset of rod position data. Must contain the columns:

  'x1', 'y1', 'z1', 'x2', 'y2', 'z2', 'x1_{cam_id1}', 'y1_{cam_id1}', 'x2_{cam_id1}', 'y2_{cam_id1}', 'x1_{cam_id2}', 'y1_{cam_id2}', 'x2_{cam_id2}', 'y2_{cam_id2}'

• cam_ids (List[str]) – The IDs are used to identify the 2D data columns.

• calibration (dict, optional) –

  Stereo camera calibration data for the camera setup given in data. The calculation reprojection errors depends on this and will therefore fail if this is not given, i.e. set to None.

  By default None.

• position_scaling (float, optional) –

  Scaling value by which to multiply the given data, in case it has been saved in a scaled form. If this is not specified, i.e. is set to None, a scaling factor of 1.0 is chosen and the data will therefore not be changed.

  By default 1.0.

• transformation (dict, optional) –

  Transformation from the first camera’s coordinate system to the experiment/world coordinate system. Providing a transformation might not be essential, depending on the given data. If data is given in the first camera’s coordinate system this value should either not be set or a neutral transformation, that does not change the coordinate system.

  By default None.

See also

reprojection_errors_hist(), reproject_data(), reproject_points()

Hint

Emits

• PlotterSignals.result_plot

plot_rod_lengths(data: DataFrame, position_scaling: float | None = None)

Plot a histogram of rod lengths.

Plot a histogram of all rod lengths present in the given DataFrame. The DataFrame therefore must contain a column called l that contains this data.

Parameters:

• data (DataFrame) –

  (Slice of) a dataset of rod position data. Must contain the column:

  l

• position_scaling (float, optional) –

  Scaling value by which to multiply the given data, in case it has been saved in a scaled form. If this is not specified, i.e. is set to None, a scaling factor of 1.0 is chosen and the data will therefore not be changed.

  By default None.

See also

ParticleDetection.reconstruct_3D.visualization.length_hist()

Hint

Emits

• PlotterSignals.result_plot

static reproject_data(data: DataFrame, cam_ids: List[str], calibration: dict, position_scaling: float = 1.0, transformation: dict | None = None)

Calculate reprojection errors for each row in a rod position data DataFrame.

Parameters:

• data (pd.DataFrame) –

  (Slice of) a dataset of rod position data. Must contain the columns:

  'x1', 'y1', 'z1', 'x2', 'y2', 'z2', 'x1_{cam_id1}', 'y1_{cam_id1}', 'x2_{cam_id1}', 'y2_{cam_id1}', 'x1_{cam_id2}', 'y1_{cam_id2}', 'x2_{cam_id2}', 'y2_{cam_id2}'

• cam_ids (List[str]) – The IDs are used to identify the 2D data columns.

• calibration (dict) – Stereo camera calibration data for the camera setup given in data. The calculation reprojection errors depends on this and will therefore fail and return if this is not given, i.e. set to None.

• position_scaling (float, optional) –

  Scaling value by which to multiply the given data, in case it has been saved in a scaled form. If this is not specified, i.e. is set to None, a scaling factor of 1.0 is chosen and the data will therefore not be changed.

  By default 1.0.

• transformation (dict, optional) –

  Transformation from the first camera’s coordinate system to the experiment/world coordinate system. Providing a transformation might not be essential, depending on the given data. If data is given in the first camera’s coordinate system this value should either not be set or a neutral transformation, that does not change the coordinate system.

  By default None.

Returns:

ndarray | None – Absolute reprojection errors, with one value per row in data. This function returns None, if either no calibration data is given or data does not contain all necessary columns.

See also

reproject_points()

run()

Run the plotting of 3D reconstruction and tracking plots in this Plotter object.

This function is not intended to be run directly but by invoking it via a QThreadPool.start(plotter) call.

Hint

Emits

• PlotterSignals.error (potentially repeatedly)

• PlotterSignals.result_plot (potentially repeatedly)

class PlotterSignals

Bases: QObject

Helper object to provide Plotter access to pyqtSignal.

error

Signal for propagating errors occuring in the worker’s thread.

The transferred tuple should contain the following values:

[0]: Exception type

[1]: Exception value

[2]: Exception traceback

See also

sys.exc_info()

Type:

pyqtSignal(tuple)

result_plot

Transfers a result Figure for display elsewhere.

Type:

pyqtSignal(Figure)

class Reconstructor(data: DataFrame, frames: List[int], calibration: dict, transformation: dict, cams: List[str], color: str)

Bases: QRunnable

Object for running the reconstruction of 3D rod coordinates in a thread different from the main thread.

This object runs the reconstruction of 3D coordinates of rods from their 2D stereo camera coordinates. The assignment of rod numbers remains untouched in with this method, i.e. there is NO automatic tracking of particles over multiple frames and NO per frame reprojection error optimization.

Parameters:

• data (DataFrame) –

  datapd.DataFrame

  (Slice of) a dataset of rod position data. Must contain at least following the columns:

  'x1_{cam_id1}', 'y1_{cam_id1}', 'x2_{cam_id1}', 'y2_{cam_id1}', 'x1_{cam_id2}', 'y1_{cam_id2}', 'x2_{cam_id2}', 'y2_{cam_id2}', 'frame'

• frames (List[int]) – Frames the reconstruction of rods will be performed on.

• calibration (dict) –

  Stereocamera calibration parameters with the required fields:

  "CM1": camera matrix of cam1

  "R": rotation matrix between cam1 & cam2

  "T": translation vector between cam1 & cam2

  "CM2": camera matrix of cam2

• transformation (dict) –

  Coordinate system transformation matrices from camera 1 coordinates to world/experiment coordinates. Must contain the following fields:

  "rotation", "translation"

  If no transformation is desired, either set this value to None or use as neutral transformation.

• cams (List[str]) – IDs of the cameras from whos images the 2D position data was generated.

• color (str) – Color of the rods in data. This value will also be written to the output DataFrame.

data

(Slice of) a dataset of rod position data. Must contain at least following the columns:

'x1_{cam_id1}', 'y1_{cam_id1}', 'x2_{cam_id1}', 'y2_{cam_id1}', 'x1_{cam_id2}', 'y1_{cam_id2}', 'x2_{cam_id2}', 'y2_{cam_id2}', 'frame'

Type:

DataFrame

frames

Frames the reconstruction of rods will be performed on.

Type:

List[int]

calibration

Stereocamera calibration parameters with the required fields:

"CM1": camera matrix of cam1

"R": rotation matrix between cam1 & cam2

"T": translation vector between cam1 & cam2

"CM2": camera matrix of cam2

Type:

dict

transformation

Coordinate system transformation matrices from camera 1 coordinates to world/experiment coordinates. Must contain the following fields:

"rotation", "translation"

If no transformation is desired, either set this value to None or use as neutral transformation.

Type:

dict

cams

IDs of the cameras from whos images the 2D position data was generated.

Type:

List[str]

color

Color of the rods in data. This value will also be written to the output DataFrame.

Type:

str

signals

Signals that can be emitted during the running of a Reconstructor object. Their purpose is to report errors, progress, and (intermediate) results.

Type:

TrackerSignals

See also

match_frame()

run()

Run the reconstruction of 3D rod coordinates with the parameters set in this Reconstructor object.

This function is not intended to be run directly but by invoking it via a QThreadPool.start(reconstructor) call.

See also

match_frame()

Hint

Emits

• TrackerSignals.error

• TrackerSignals.progress

• TrackerSignals.result

class Tracker(data: DataFrame, frames: List[int], calibration: dict, transformation: dict, cams: List[str], color: str)

Bases: Reconstructor

Object for running the tracking of rods and reconstruction of 3D rod coordinates in a thread different from the main thread.

This object runs the tracking of rods and simultaneous reconstruction of 3D coordinates from their 2D stereo camera coordinates. Rod numbers are reassigned in this process.

Parameters:

• data (DataFrame) –

  datapd.DataFrame

  (Slice of) a dataset of rod position data. Must contain at least following the columns:

  'x1_{cam_id1}', 'y1_{cam_id1}', 'x2_{cam_id1}', 'y2_{cam_id1}', 'x1_{cam_id2}', 'y1_{cam_id2}', 'x2_{cam_id2}', 'y2_{cam_id2}', 'frame'

• frames (List[int]) – Frames the reconstruction of rods will be performed on.

• calibration (dict) –

  Stereocamera calibration parameters with the required fields:

  "CM1": camera matrix of cam1

  "R": rotation matrix between cam1 & cam2

  "T": translation vector between cam1 & cam2

  "CM2": camera matrix of cam2

• transformation (dict) –

  Coordinate system transformation matrices from camera 1 coordinates to world/experiment coordinates. Must contain the following fields:

  "rotation", "translation"

  If no transformation is desired, either set this value to None or use as neutral transformation.

• cams (List[str]) – IDs of the cameras from whos images the 2D position data was generated.

• color (str) – Color of the rods in data. This value will also be written to the output DataFrame.

data

(Slice of) a dataset of rod position data. Must contain at least following the columns:

'x1_{cam_id1}', 'y1_{cam_id1}', 'x2_{cam_id1}', 'y2_{cam_id1}', 'x1_{cam_id2}', 'y1_{cam_id2}', 'x2_{cam_id2}', 'y2_{cam_id2}', 'frame'

Type:

DataFrame

frames

Frames the reconstruction of rods will be performed on.

Type:

List[int]

calibration

Stereocamera calibration parameters with the required fields:

"CM1": camera matrix of cam1

"R": rotation matrix between cam1 & cam2

"T": translation vector between cam1 & cam2

"CM2": camera matrix of cam2

Type:

dict

transformation

Coordinate system transformation matrices from camera 1 coordinates to world/experiment coordinates. Must contain the following fields:

"rotation", "translation"

If no transformation is desired, either set this value to None or use as neutral transformation.

Type:

dict

cams

IDs of the cameras from whos images the 2D position data was generated.

Type:

List[str]

color

Color of the rods in data. This value will also be written to the output DataFrame.

Type:

str

signals

Signals that can be emitted during the running of a Tracker object. Their purpose is to report errors, progress, and (intermediate) results.

Type:

TrackerSignals

See also

match_frame()

run()

Run the tracking of rods coordinates with the parameters set in this Tracker object.

This function is not intended to be run directly but by invoking it via a QThreadPool.start(reconstructor) call.

See also

match_frame()

Hint

Emits

• TrackerSignals.error

• TrackerSignals.progress

• TrackerSignals.result

class TrackerSignals

Bases: QObject

Helper object to provide Reconstructor and class:Tracker access to pyqtSignal.

error

Signal for propagating errors occuring in the worker’s thread.

The transferred tuple should contain the following values:

[0]: Exception type

[1]: Exception value

[2]: Exception traceback

See also

sys.exc_info()

Type:

pyqtSignal(tuple)

progress

Reports the progress of the started computation.

The progress is reported as the ratio of finished iterations over all iterations, so \(\in [0, 1]\).

Type:

pyqtSignal(float)

result

Reports the result of completed reconstructions.

Type:

pyqtSignal(DataFrame)