ParticleDetection.reconstruct_3D.match2D

Functions to reconstruct 3D rod endpoints from images of a stereocamera system on a per-frame basis, without knowledge about positions in other frames. Some functions are directly ported from a MATLAB implementation to python.

Authors: Adrian Niemann (adrian.niemann@ovgu.de), Dmitry Puzyrev (dmitry.puzyrev@ovgu.de)

Date: 01.11.2022

match_complex(data: DataFrame, frame_numbers: Iterable[int], color: str, calibration: dict, transform: dict, cam1_name='gp1', cam2_name='gp2', renumber: bool = True)

Matches and triangulates rods from a DataFrame.

The function matches rod endpoints per frame such that the reprojection error is minimal.

Parameters

• data (DataFrame) – Dataset of rod positions.

• frame_numbers (Iterable[int]) – An iterable of frame numbers present in the data.

• color (str) – Color of the rods in data to match.

• calibration (dict) –

  Stereocamera calibration parameters with the required fields:

  "CM1": camera matrix of cam1

  "R": rotation matrix between cam1 & cam2

  "T": translation vector between cam1 & cam2

  "CM2": camera matrix of cam2

• transform (dict) –

  Coordinate system transformation matrices from camera 1 coordinates to world/experiment coordinates. Must contain the following fields:

  "M_rotate_x", "M_rotate_y", "M_rotate_z", "M_trans", "M_trans2"

• cam1_name (str, optional) –

  First camera’s identifier in the given dataset.

  By default "gp1".

• cam2_name (str, optional) –

  Second camera’s identifier in the given dataset.

  By default "gp2".

• renumber (bool, optional) –

  Flag, whether to keep the already assigned combinations between camera 1 and camera 2.

  True: Only the endpoint combinations are (re-)evaluated.

  False: Rod combinations between camera 1 and camera 1 as well as their respective endpoint combinations are (re-)evaluated.

  By default True.

Returns

Tuple[DataFrame, ndarray, ndarray] – Returns the endpoint matched DataFrame together with the reprojection errors and the resulted rod lengths.

match_csv_complex(input_folder, output_folder, colors, cam1_name='gp1', cam2_name='gp2', frame_numbers=None, calibration_file=None, transformation_file=None, rematching=True)

Matches and triangulates rods from *.csv data files.

The function matches rod endpoints per frame such that the reprojection error is minimal. It takes *.csv files with the columns from DEFAULT_COLUMNS as input and also outputs the results in this format.

Parameters

• input_folder (str) – Folder containing the *.csv files for all colors given in colors.

• output_folder (str) – Folder to write the output to. The parent folder of this must exist already.

• colors (Iterable[str]) – Names of the colors present in the dataset. See DEFAULT_CLASSES.

• cam1_name (str, optional) –

  First camera’s identifier in the given dataset.

  By default "gp1".

• cam2_name (str, optional) –

  Second camera’s identifier in the given dataset.

  By default "gp2".

• frame_numbers (Iterable[int], optional) – An iterable of frame numbers present in the data. By default None.

• calibration_file (str, optional) –

  Path to a *.json file with stereocalibration data for the cameras which produced the images for the rod position data.

  By default the calibration constructed with Matlab for GP1 and GP2 is used.

• transformation_file (str, optional) –

  Path to a *.json file with transformation matrices expressing the transformation from the first camera’s coordinate system to the world/box coordinate system.

  By default the transformation constructed with Matlab is used.

Returns

ndarray, ndarray – Reprojection errors, rod lengths of the matched rod endpoints.

match_frame(data: DataFrame, cam1_name: str, cam2_name: str, frame: int, color: str, calibration: dict, P1: ndarray, P2: ndarray, rot: Rotation, trans: ndarray, r1: ndarray, r2: ndarray, t1: ndarray, t2: ndarray, renumber: bool = True)

Matches and triangulates rods from a DataFrame.

Parameters

• data (DataFrame) – Dataset of rod positions.

• cam1_name (str) – First camera’s identifier in the given dataset, e.g. "gp1".

• cam2_name (str) – Second camera’s identifier in the given dataset, e.g. "gp2".

• frame (int) – Frame in data who’s endpoints shall be (re-)evaluated.

• color (str) – Color of the rods in data to match.

• calibration (dict) –

  Stereocamera calibration parameters with the required fields:

  "CM1": camera matrix of cam1

  "R": rotation matrix between cam1 & cam2

  "T": translation vector between cam1 & cam2

  "CM2": camera matrix of cam2

• P1 (ndarray) – Projection matrix for camera 1.

• P2 (ndarray) – Projection matrix for camera 2.

• rot (Rotation) – Rotation from camera 1 coordinates to world/experiment coordinates.

• trans (ndarray) – Translation vector as part of the transformation to world/experiment coordinates.

• r1 (ndarray) – Rotation matrix of camera 1.

• r2 (ndarray) – Rotation matrix of camera 2.

• t1 (ndarray) – Translation vector of camera 1.

• t2 (ndarray) – Translation vector of camera 2.

• renumber (bool, optional) –

  Flag, whether to keep the already assigned combinations between camera 1 and camera 2.

  True: Only the endpoint combinations are (re-)evaluated.

  False: Rod combinations between camera 1 and camera 1 as well as their respective endpoint combinations are (re-)evaluated.

  By default True.

Returns

Tuple[DataFrame, ndarray, ndarray] – Returns the DataFrame with (re-)matched endpoints for frame of color. Additionally, returns the assignment costs, i.e. the sum of end point reprojection errors per rod. Lastly, returns the lengths of the reconstructed rods.

match_matlab_complex(cam1_folder, cam2_folder, output_folder, colors, frame_numbers, calibration_file=None, transformation_file=None, cam1_convention='{idx:05d}_{color:s}.mat', cam2_convention='{idx:05d}_{color:s}.mat')

Match rod endpoints per frame such that the reprojection error is minimal. This function takes the same input file format and outputs the same file formats as the previous implementation in MATLAB. The inputs and outputs are equivalent to match_matlab_simple() but makes heavy use of matrix operations to increase computational efficiency.

Parameters

• cam1_folder (str) – Folder with rod data of the first camera.

• cam2_folder (str) – Folder with rod data of the second camera

• output_folder (str) – Folder to write the output to. The parent folder of this must exist already.

• colors (List[str]) – Names of the colors present in the dataset.

• frame_numbers (Iterable[int]) – An iterable of frame numbers present in the data.

• calibration_file (str, optional) –

  Path to a *.json file with stereocalibration data for the cameras which produced the images for the rod position data.

  By default the calibration constructed with Matlab for GP1 and GP2 is used.

• transformation_file (str, optional) –

  Path to a *.json file with transformation matrices expressing the transformation from the first camera’s coordinate system to the world/box coordinate system.

  By default the transformation constructed with Matlab is used.

• cam1_convention (str, optional) –

  Naming convention for the first camera’s position data files defined by a formattable string, that accepts some of the following variables: {idx, color}.

  By default "{idx:05d}_{color:s}.mat".

• cam2_convention (str, optional) –

  Naming convention for the second camera’s position data files defined by a formattable string, that accepts some of the following variables: {idx, color}.

  By default "{idx:05d}_{color:s}.mat".

Returns

ndarray, ndarray – Reprojection errors, rod lengths of the matched rod endpoints.

Note

This function currently saves the 3D points in the first camera’s coordinate system, NOT the world/box coordinate system.

Warning

Deprecated since version 0.4.0: Use match_csv_complex() instead.

match_matlab_simple(cam1_folder, cam2_folder, output_folder, colors, frame_numbers, calibration_file=None, transformation_file=None, cam1_convention='{idx:05d}_{color:s}.mat', cam2_convention='{idx:05d}_{color:s}.mat')

Ported Matlab script from match_rods_2020mix_gp12_cl1.m. This function takes the same input file format and outputs the same file formats as the previous implementation in MATLAB. Use this function for a consistent behaviour to previous data processings.

:param See match_matlab_complex():

Returns

See match_matlab_complex()

Note

This function currently saves the 3D points in the first camera’s coordinate system, NOT the world/box coordinate system.

Warning

Deprecated since version 0.4.0: Use match_csv_complex() instead.

reorder_endpoints_csv(input_folder: str, output_folder: str, colors: Iterable[str], cam1_name: str = 'gp1', cam2_name: str = 'gp2', frame_numbers: Optional[Iterable[int]] = None)

Reorders endpoints from *.csv data files.

The function reorders rod endpoints per frame such that the endpoints displacement between consecutive frames is minimal. It does not change the rod number assignment and does not match the 2D endpoint coordinates into 3D acoording to the reprojection error. Usually this step is performed after rematching or instead of it if rematching is not necessary.

It takes *.csv files with the columns from DEFAULT_COLUMNS as input and also outputs the results in this format.

Parameters

• input_folder (str) – Folder containing the *.csv files for all colors given in colors.

• output_folder (str) – Folder to write the output to. The parent folder of this must exist already.

• colors (Iterable[str]) – Names of the colors present in the dataset. See DEFAULT_CLASSES.

• cam1_name (str, optional) –

  First camera’s identifier in the given dataset.

  By default "gp1".

• cam2_name (str, optional) –

  Second camera’s identifier in the given dataset.

  By default "gp2".

• frame_numbers (Iterable[int], optional) –

  An iterable of frame numbers present in the data.

  By default None.

Returns

Tuple[ndarray] – Returns the assignment costs, i.e. the sum of end point reprojection errors per rod.