miop¶
- class miop.CameraMvmtFromMetadata[source]¶
Bases:
DAGNodeComputes camera intrinsics and extrinsics (rotation matrices) from metadata.
- rotation_matrices¶
Rotation matrices for each image in each face.
- Type:
list of list of np.ndarray
- camera_position¶
- List of dictionaries for each face containing:
‘rotations’: array of 3x3 rotation matrices (extrinsics)
‘intrinsics’: placeholder list for intrinsics (scaling and shear [ones, [1., 0.]])
- Type:
list of dict
- choose_axis(rot_z)[source]¶
Determines the tilt axis based on rotation in the (x, y) plane.
- Parameters:
rot_z (float) – Rotation angle in the (x, y) plane in radians.
- Returns:
A 3D unit vector representing the axis of rotation, one of: - [1., 0., 0.] (default: horizontal) - [-1., 0., 0.] (rot_z ≈ 180°) - [0., 1., 0.] (rot_z ≈ -90°) - [0., -1., 0.] (rot_z ≈ 90°)
- Return type:
list of float
- eval(img_collection)[source]¶
Computes camera extrinsics (rotation matrices) for each image in the image collection using the tilt angle and inferred tilt axis from metadata.
- Parameters:
img_collection (ImageCollection) – The collection of images with metadata from which to compute rotations.
- Returns:
- One dictionary per face, containing:
’rotations’: np.ndarray of shape (N, 3, 3), rotation matrices for N images
’intrinsics’: placeholder value ([ones, [1., 0.]])
- Return type:
list of dict
- get_rotation_matrix(angle, axis)[source]¶
Computes a 3x3 rotation matrix given an angle and rotation axis using Rodrigues’ formula.
- Parameters:
angle (float) – Rotation angle in radians.
axis (list of float) – 3D axis of rotation (e.g., [1., 0., 0.] for X-axis).
- Returns:
A 3x3 rotation matrix representing the rotation about the given axis.
- Return type:
np.ndarray
- class miop.CorrespondingPointsFromDisparity(n_points=10000)[source]¶
Bases:
DAGNodeExtract corresponding points from disparity maps.
This class takes disparity maps (representing the horizontal and vertical displacements between images) as input and computes the corresponding points in the second image based on the disparity information.
- n_points¶
Number of points to extract from the disparity maps.
- Type:
int
- eval(disparity_maps)[source]¶
Compute updated corresponding points using disparity maps.
This method iterates over the disparity maps and applies the displacement values from the disparity maps to a grid of points in the first image, removing points that lie outside the image border.
- Parameters:
disparity_maps (list of list of np.ndarray) – A nested list of disparity maps. Each face (outer list) contains a list of disparity maps. Each disparity map is a (2, H, W) array representing the x and y displacements respectively.
- Returns:
Valid corresponding points across all disparity maps.
- Return type:
list of list of np.ndarray
- class miop.CorrespondingPointsFromMatches(atol)[source]¶
Bases:
DAGNodeIdentify corresponding points across an image collection using feature matches.
This class takes a series of matched points from multiple images (typically, the output of a feature matching algorithm) and finds feature points that have consistent correspondences across image pairs (e.g., a <-> b, b <-> c implies a <-> b <-> c).
- atol¶
Absolute tolerance for coordinates to be considered equivalent (used for rounding).
- Type:
float
- eval(matches)[source]¶
Filter and propagate consistent feature matches across an image sequence.
- Parameters:
matches (list of tuple of np.ndarray) – A list of match pairs. Each element is a tuple of two (N, 2) numpy arrays (pts_1, pts_2) representing keypoints matched between two consecutive images.
- Returns:
A list of keypoints corresponding across all images, one per image in the sequence.
- Return type:
list of np.ndarray
- find_duplicates(arr)[source]¶
Find duplicate (non-unique) 2D points in an array.
- Parameters:
arr (np.ndarray) – A 2D array of shape (n, 2), where n is the number of keypoints.
- Returns:
Indices of duplicate points.
- Return type:
np.ndarray
- round_to_nearest_multiple(arr, atol)[source]¶
Round coordinates to the nearest multiple of the given absolute tolerance.
- Parameters:
arr (np.ndarray) – Input coordinate array.
atol (float) – Absolute tolerance for rounding.
- Returns:
Array with coordinates rounded to the nearest multiple of atol.
- Return type:
np.ndarray
- class miop.DenseRoma(num_matches=5000, tiny=False)[source]¶
Bases:
DAGNodePerform dense feature matching between image pairs using the RoMa model.
RoMa can be installed from the researcher’s repository: https://github.com/Parskatt/RoMa
- eval(img_collection)[source]¶
Run RoMa matching on an image collection.
- Parameters:
img_collection (ImageCollection) – A collection of faces, each containing a list of image pairs to match.
- Returns:
A nested list of matched keypoints for each face. Each element contains a list of two (N, 2) arrays representing the matched keypoints in the two images.
- Return type:
List[List[np.ndarray]]
- plot_images(img1, img2)[source]¶
Plot two images side by side.
- Parameters:
img1 (np.ndarray) – First image. Shape (H, W, 3) or (H, W).
img2 (np.ndarray) – Second image. Shape (H, W, 3) or (H, W).
- plot_keypoints(kpts, colors='lime')[source]¶
Plot keypoints on the currently displayed images.
- Parameters:
kpts (List[np.ndarray]) – List of (N, 2) arrays containing keypoint coordinates.
colors (str or List[str], optional) – Color or list of colors for the keypoints. Default is “lime”.
- class miop.Factorization(reproErrorTh, normalized_view, max_matches=0, batch_size=4)[source]¶
Bases:
DAGNodeBuilds a measurement matrix from corresponding points and factorizes it to recover the movement of the cameras.
- reproErrorTh¶
Threshold for the reprojection error.
- Type:
float
- max_matches¶
Maximum number of matches to consider (default is 0, meaning no limit).
- Type:
int
- batch_size¶
Number of images to process in each batch (default is 4).
- Type:
int
- rot_angles¶
List of rotation angles for the camera movements. Set in eval().
- Type:
list
- build_measurement_matrix(matches)[source]¶
Constructs a measurement matrix from corresponding points in multiple images.
- Parameters:
matches (numpy.ndarray) – Array of shape (m, n, 2), where m is number of images, n is number of points, and 2 corresponds to (x, y) coordinates.
- Returns:
- meas_matnumpy.ndarray
The constructed measurement matrix of shape (2*m, n).
- centersnumpy.ndarray
The centroids of the matches used for centering the points, shape (m, 2).
- Return type:
tuple
- compute_rmse(distances)[source]¶
Computes the root mean square error (RMSE) from a list of distances.
- Parameters:
distances (array-like) – List or array of distance errors.
- Returns:
The computed RMSE value.
- Return type:
float
- eval(matches)[source]¶
Computes camera parameters from corresponding matches using factorization on batches of images.
The camera movements for each batch are propagated to the rest of the face. The first camera of each face is used as relative origin with zero tilt. The tilt angles are forced positive, so the image collection should have monotonically increasing tilt angles.
- Parameters:
matches (list of list of numpy.ndarray) – Corresponding points across the image collection. The first list level represents faces.
- Returns:
A list containing the computed camera parameters for each face.
- Return type:
list
- factorization_sc(W, decomp='eigen', normRot=0)[source]¶
Performs factorization using the scaling and camera motion model described by Poelman and Kanade.
- Parameters:
W (numpy.ndarray) – The measurement matrix containing point correspondences across frames.
decomp (str, optional) – Decomposition method to use: ‘eigen’ or ‘cholesky’ (default is ‘eigen’).
normRot (int, optional) – Index of the camera used as reference for normalization (default is 0).
- Returns:
- RotationMatriceslist of numpy.ndarray
List of rotation matrices for each frame.
- Shapenumpy.ndarray
The estimated 3D shape matrix.
- kiNewlist
Normalized scaling factors for each frame.
- paramsnumpy.ndarray
Additional parameters (2-element array).
- Qnumpy.ndarray
The matrix used for correction.
- kilist
Original scaling factors for each frame.
- Return type:
tuple
- factorize(measurementMatrix, centroids)[source]¶
Factorizes the measurement matrix to estimate camera rotations and intrinsics.
- Parameters:
measurementMatrix (numpy.ndarray) – The measurement matrix to factorize.
centroids (numpy.ndarray) – Centroids of matched points for adjustment during factorization.
- Returns:
Dictionary containing: - ‘rotations’: numpy.ndarray of estimated rotation matrix. - ‘intrinsics’: list containing normalized scaling factors and parameters.
- Return type:
dict
- gT(a, b)[source]¶
Computes the constraint vector used in the metric upgrade process.
- Parameters:
a (np.ndarray) – An array of shape (frames, 3) representing camera motion vectors.
b (np.ndarray) – An array of shape (frames, 3) representing camera motion vectors.
- Returns:
A (frames, 6) array representing the constraint vectors for each frame.
- Return type:
np.ndarray
- gT1(a, b)[source]¶
Constructs a vector used in solving the quadratic equations for factorization based on Tomasi/Kanade and Poelman/Kanade methods.
- Parameters:
a (np.ndarray) – A vector of shape (3,) representing the motion of the first frame.
b (np.ndarray) – A vector of shape (3,) representing the motion of the first frame.
- Returns:
A vector of shape (6,) containing the constraint terms.
- Return type:
np.ndarray
- print_rotation_angles(R, name='SC')[source]¶
Prints rotation angles for each rotation matrix in degrees.
- Parameters:
R (numpy.ndarray) – Rotation matrices array, shape (num_cameras, 3, 3).
name (str, optional) – Name to prefix the print statements (default is ‘SC’).
- Returns:
List of rotation angles [phiX, phiY, phiZ] in degrees for each rotation matrix.
- Return type:
list of numpy.ndarray
- print_scaling_factors(scaling_factors)[source]¶
Normalizes and prints the relative scaling factors of each view.
- Parameters:
scaling_factors (numpy.ndarray) – Scaling factors for each camera view.
- Returns:
Normalized scaling factors with respect to the normalized view.
- Return type:
numpy.ndarray
- remove_outliers_repro(W, reW, thFac)[source]¶
Removes outlier points based on reprojection error by applying a threshold.
- Parameters:
W (np.ndarray) – The original 2D measurement matrix of shape (2*num_views, num_points).
reW (np.ndarray) – The reprojected 2D measurement matrix of shape (2*num_views, num_points).
thFac (float) – The reprojection error threshold in pixels.
- Returns:
Wnew (np.ndarray): Cleaned measurement matrix after removing outliers.
WreNew (np.ndarray): Corresponding reprojected measurement matrix.
- Return type:
tuple of np.ndarray
- reprojected_measurement_matrix(result)[source]¶
Reprojects the 3D structure back to 2D image space using the estimated camera rotations and scaling.
- Parameters:
result (tuple) – The factorization result containing: - rotations (list of np.ndarray): Camera rotation matrices. - structure (np.ndarray): 3D structure. - scalings (list of float): Scaling factors per image. - params (tuple): Affine parameters (c, s).
- Returns:
reW (np.ndarray): The reprojected 2D measurement matrix of shape (2*num_views, num_points).
P_trunc (list of np.ndarray): List of projection matrices for each view.
- Return type:
tuple
- rotation_angles_arctan(R)[source]¶
Extract rotation angles (in degrees) from a rotation matrix using arctangent.
- Parameters:
R (numpy.ndarray) – A 3x3 rotation matrix.
- Returns:
Rotation angles [x, y, z] in degrees.
- Return type:
numpy.ndarray
- signed_error(reW, W)[source]¶
Computes the signed reprojection error between the reprojected and original measurement matrices.
- Parameters:
reW (np.ndarray) – The reprojected 2D measurement matrix of shape (2*num_views, num_points).
W (np.ndarray) – The original 2D measurement matrix of shape (2*num_views, num_points).
- Returns:
A (2, num_points) array containing the signed errors in u and v directions.
- Return type:
np.ndarray
- class miop.ImageCollection(metadata_type='none', metadata=None, downsample=1.0, crop_dim=[], two_by_two=False, use_segmentation=False, max_dim=(800, 800), verbose=True)[source]¶
Bases:
DAGNodeRepresents a collection of images, with support for downsampling, cropping, rotating, segmentation, and pairing based on image metadata.
- metadata_type¶
Type of metadata (‘none’, ‘fei’, or ‘custom’).
- Type:
str
- metadata¶
- Metadata associated with each image. Each dict may contain:
‘StageTa’: Tilt angle in radians.
‘tilt_axis’: Tilt axis as a 3D vector (e.g., [1., 0., 0.]).
‘StageR’: Rotation angle of the microscope stage in the (x, y) plane, in radians.
- Type:
list of dict
- downsample_factor¶
Factor by which to downsample all images.
- Type:
float
- crop_dim¶
Crop dimensions per face. Format: [[[width, height], [x, y]], …]
- Type:
list
- two_by_two¶
If True, images are grouped into fixed pairs: (1,2), (3,4), etc. If False, grouped as overlapping pairs: (1,2), (2,3), etc.
- Type:
bool
- use_segmentation¶
Whether to use segmentation masks.
- Type:
bool
- max_dim¶
Maximum image dimensions as (width, height).
- Type:
tuple of int
- verbose¶
If True, prints warnings and logs.
- Type:
bool
- paths_to_images¶
File paths to the images in the collection.
- Type:
list of str
- paths_to_masks¶
File paths to the corresponding segmentation masks.
- Type:
list of str
- masks¶
Mapping from Image objects to their segmentation masks.
- Type:
dict
- faces¶
Grouped image pairs, organized by face.
- Type:
list of tuple
- apply_to_collection(func, *args, **kwargs)[source]¶
Applies a function to each image in the collection.
- Parameters:
func (callable) – Function to apply to each image.
*args – Additional positional arguments to pass to the function.
**kwargs – Additional keyword arguments to pass to the function.
- Returns:
List of results from the function.
- Return type:
list
- apply_to_face(face_idx, func, *args, **kwargs)[source]¶
Applies a function to all images within a face.
- Parameters:
face_idx (int) – Index of the face.
func (callable) – Function to apply.
*args – Additional positional arguments.
**kwargs – Additional keyword arguments.
- Returns:
List of results from the function.
- Return type:
list
- crop(*crop_arg)[source]¶
Crops images in the collection.
- Parameters:
crop_arg (list) –
List of crop parameters. Format per face: [[width, height], [x_offset, y_offset]]
If only one crop_arg is provided, it is applied to all faces.
- downsample(factor_x, factor_y)[source]¶
Downsamples all images in the collection.
- Parameters:
factor_x (float) – Downsampling factor for width.
factor_y (float) – Downsampling factor for height.
- eval(paths_to_images, paths_to_masks=None)[source]¶
Loads and preprocesses the image collection: reading, cropping, downsampling, rotating, and handling segmentation.
- Parameters:
paths_to_images (list of str) – Paths to the image files.
paths_to_masks (list of str, optional) – Paths to the segmentation mask directories.
- Returns:
The processed image collection.
- Return type:
- register_pair(paths_to_images)[source]¶
Not implemented. Used to manually add image pairs to the collection.
- Raises:
NotImplementedError –
- rotate(angle=None, rad=False)[source]¶
Rotates all images of the collection according to angle or rotation_z from the metadata.
- Parameters:
angle (float or None, optional) – Angle to rotate. If None, uses rotation_z from metadata.
rad (bool, optional) – If True, angle is in radians. Default is False.
- save_collection(path_to_dir)[source]¶
Saves all images to the specified directory.
- Parameters:
path_to_dir (str) – Directory path where images will be saved.
- Returns:
Results from saving images.
- Return type:
list
- show(crop_utility=True, use_segmentation=False)[source]¶
Displays all images in the collection, optionally overlaying crop regions and segmentation masks.
- Parameters:
crop_utility (bool, optional) – Whether to display the cropping utility overlay (default is True).
use_segmentation (bool, optional) – Whether to overlay segmentation masks (default is False).
- class miop.Interpolate[source]¶
Bases:
DAGNodeDensify sparse feature matches into full disparity maps using interpolation.
Interpolation is done with scipy.interpolate.griddata(method=’linear’).
- eval(matches)[source]¶
Generate dense disparity maps from sparse matches.
- Parameters:
matches (List[List[np.ndarray]]) – A nested list of shape (M, N), where M is the number of faces and N is the number of image pairs. Each entry is a (2, N, 2) array of matched keypoints, where the first element contains keypoints from image 1 and the second from image 2.
- Returns:
A nested list of shape (M, N), where each element contains two (H, W) arrays: the horizontal and vertical disparity maps.
- Return type:
List[List[np.ndarray]]
- class miop.PoissonSurfaceReconstruction(depth=5, color=array([[0.5], [0.5], [0.5]]))[source]¶
Bases:
DAGNodeClass for performing surface reconstruction from 3D point clouds using the Poisson Surface Reconstruction method.
This class wraps the Open3D implementation of Poisson reconstruction and includes methods for visualizing and saving the resulting mesh.
- depth¶
Octree depth controlling reconstruction resolution. Higher values give finer meshes at the cost of more computation. Default is 5.
- Type:
int
- color¶
RGB color used to uniformly paint the reconstructed mesh. Default is gray ([[0.5], [0.5], [0.5]]).
- Type:
np.ndarray, shape (3, 1)
- mesh¶
Reconstructed mesh, set after eval() is called.
- Type:
o3d.geometry.TriangleMesh or None
- eval(pcds)[source]¶
Estimates normals and performs Poisson Surface Reconstruction on the first point cloud in the list.
- Parameters:
pcds (list of o3d.geometry.PointCloud) – A list of Open3D point clouds. Only the first point cloud (pcds[0]) is used for reconstruction.
- Returns:
A list containing one element: the reconstructed mesh.
- Return type:
list of o3d.geometry.TriangleMesh
- Raises:
ValueError – If the input list is empty or the input is not a valid point cloud.
- class miop.RaftFlow(device=None)[source]¶
Bases:
DAGNodeComputes dense optical flow (displacement maps) between image pairs using the RAFT model.
- device¶
The device (CPU, CUDA, MPS) used for inference.
- Type:
torch.device
- disp_maps¶
The computed displacement maps for each image pair in each face of the collection. Each displacement map is a 2D flow field of shape (2, H, W), where:
disp[0, :, :] is the x-direction displacement,
disp[1, :, :] is the y-direction displacement.
- Type:
list of list of np.ndarray or None
- compute_flow(image_1, image_2)[source]¶
Computes the dense optical flow (displacement map) between two RGB images using the RAFT model.
- Parameters:
image_1 (np.ndarray) – The first image of shape (H, W, 3) in uint8 or float format.
image_2 (np.ndarray) – The second image of shape (H, W, 3) in uint8 or float format.
- Returns:
A displacement map of shape (2, H, W), where the first channel is x-displacement and the second channel is y-displacement.
- Return type:
np.ndarray
- eval(img_collection)[source]¶
Evaluates the optical flow for each image pair in the image collection.
- Parameters:
img_collection (ImageCollection) – An image collection containing pairs of images organized by faces.
- Returns:
A nested list where each element corresponds to a face, and contains the displacement maps for each image pair in that face.
- Return type:
list of list of np.ndarray
- show(face=0, pair=0, cmap='Spectral')[source]¶
Visualizes the displacement map for a specific image pair using matplotlib.
- Parameters:
face (int, optional) – Index of the face to visualize (default: 0).
pair (int, optional) – Index of the image pair within the face to visualize (default: 0).
cmap (str, optional) – Colormap to use for the visualization (default: “Spectral”).
- Raises:
AttributeError – If self.disp_maps is not available.
- class miop.Reconstruction(by_pair=False)[source]¶
Bases:
DAGNodeReconstructs 3D point clouds from matched 2D image points using camera intrinsics and extrinsics.
- pcds¶
A nested list of reconstructed point clouds. - Outer list corresponds to each face. - Inner list contains one point cloud per image pair.
- Type:
list of list of np.ndarray or None
- by_pair¶
Determines the format of input matches. If True, matches are expected as pairs.
- Type:
bool
- compute_point_cloud(camera_position, dense_matches)[source]¶
Computes 3D point clouds via affine triangulation from dense 2D matches.
- Parameters:
camera_position (list of dict) – List of camera parameters for each face.
dense_matches (list) – List of matched 2D point arrays. Format depends on by_pair.
- Returns:
Nested list of point clouds. One list per face, one cloud per image pair.
- Return type:
list of list of np.ndarray
- eval(camera_position, matches)[source]¶
Computes the 3D point cloud for each image pair using triangulation.
- Parameters:
camera_position (list of dict) –
A list of dictionaries containing camera rotation matrices and intrinsics. Each dictionary represents one face and must contain:
’rotations’: np.ndarray of shape (N, 3, 3)
’intrinsics’: list with two elements:
intrinsics[0]: list of scale factors (k1, k2, …)
intrinsics[1]: list containing [c, s] values (affine scale & shear)
matches (list) – 2D point matches between images. Format depends on by_pair.
- Returns:
Reconstructed 3D point clouds per face and image pair.
- Return type:
list of list of np.ndarray
- inhom_triangulation(b, A)[source]¶
Solves an inhomogeneous linear system Ax = b using the Moore-Penrose pseudo-inverse.
- Parameters:
b (np.ndarray) – Right-hand side matrix (stacked 2D correspondences).
A (np.ndarray) – Projection matrix (camera model matrix).
- Returns:
Triangulated 3D point cloud of shape (N, 3).
- Return type:
np.ndarray
- class miop.RegistrationByCorresp[source]¶
Bases:
DAGNodeClass for performing 3D point cloud registration using Procrustes alignment, based on known point correspondences.
- pcds_tr¶
List of registered point clouds. Set after calling eval().
- Type:
list of open3d.geometry.PointCloud
- compute_point_cloud(camera_position, dense_matches)[source]¶
Triangulates 3D points from matched 2D correspondences using affine camera models.
- Parameters:
camera_position (list of dict) – List containing intrinsic matrices and rotations for each camera/view.
dense_matches (list) – 2D point correspondences across image pairs.
- Returns:
Reconstructed 3D point clouds for each image pair.
- Return type:
list of list of np.ndarray
- eval(pcds, camera_position, corresponding_points)[source]¶
Aligns a set of point clouds using Procrustes registration based on known correspondences.
- Parameters:
pcds (list of list of np.ndarray) – Each element in the outer list corresponds to a face/view. Each inner element is a point cloud (N, 3).
camera_position (list of dict) – Camera intrinsic and rotation parameters for triangulation.
corresponding_points (list) – List of corresponding 2D points across views for triangulation.
- Returns:
Transformed point clouds aligned to a common frame.
- Return type:
list of open3d.geometry.PointCloud
- procrustes_3d(X, Y)[source]¶
Computes the optimal rotation and translation to align Y to X using the Procrustes algorithm.
- Parameters:
X (np.ndarray, shape (N, 3)) – Reference 3D point cloud.
Y (np.ndarray, shape (N, 3)) – Point cloud to align.
- Returns:
R (np.ndarray, shape (3, 3)) – Optimal rotation matrix.
t (np.ndarray, shape (3,)) – Optimal translation vector.
- class miop.RegistrationByICP(threshold=32.0, trans_init=array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]))[source]¶
Bases:
DAGNodeClass for performing 3D point cloud registration using the Iterative Closest Point (ICP) algorithm. This class is a wrapper for Open3D’s ICP implementation.
- threshold¶
Distance threshold for ICP correspondence matching. Default is 32.0.
- Type:
float
- trans_init¶
Initial transformation matrix used to start the ICP process. Default is identity.
- Type:
np.ndarray, shape (4, 4)
- pcds_reg¶
List of registered point clouds after running the eval() method.
- Type:
list of open3d.geometry.PointCloud
- eval(pcds)[source]¶
Registers a list of point clouds using the ICP algorithm.
- Parameters:
pcds (list of list of np.ndarray) – Each sublist represents a group of point clouds for a given face (or view group). Each point cloud is expected to be an (N, 3) NumPy array.
- Returns:
Registered and merged point clouds.
- Return type:
list of open3d.geometry.PointCloud
Modules