Artificial Intelligence and Machine Vision

Recover the camera matrices from estimated fundamental matrix F Use the camera matrices to compute the point correspondences. Without K1 and K2 (internal calibration), recover a projective transforme With K1 and K2, recover the metric reconstruction. The epipolar constraint defines a line, since x2^T F x1 = l1^T x1 = 0 An epipolar line corresponding to the point x2, because l1^T x1 = 0 Thus x2 is on the epipolar line A point e is epipole, when the epipolar lines meet at this point e. Must satisfy: F e1 = 0 or e2^T F = 0

In this blog, I will discuss how to calibrate a suite of sensors used in a L2 autonomous prototype vehicle. Note: - To ensure a dataset generated from a L2 autonomous prototype, calibrate all sensors on board per each trip. In our autonomous vehicle prototypes, we use 6 - Cameras - we use 6 cameras - cropped native resolution from 1600x900 to smaller images - are in native BGR format. - with auto-exposure with a maximum limit of 20 ms. - use Bayer8 format for 1 byte per pixelin encoding - 1/1.8" CMOS sensor for 12 Hz capture frequency. - positions:   - one front center camera   - one front side mirrow camera per side   - one rear center camera   - one rear door centered camera per side 5 - Long Range RADARs - we use 5 sensors of RADAR - @ 13Hz capture frequency at 77 Ghz - measures distance & velocity, independently, in one cycle - positions:   - one front bumper center radar   - one front side mirror radar per side   - one...

Project Clear Sky Note: - This technology was created by me to enhance driving or flying conditions in poor weather. - This project is open sourced under GPL v3. - For the closed source for a commercial product, please contact dparksports at gmail dot com - Copyright 2010

Why RANSAC? Because we want a model of good feature matches. - Inliers:   - Good matches - Outlier:   - Bad matches Interest points (500/image) (640x480) What other algorithms are available? - Exhaustive search   - for each featfure, compare all other feactures in other images - Hashing   - compute a short descriptor from each feature vector - Nearest neighbor techniques:   - k-trees and their variants Putative correspondences (268) (Best match,SSD Outliers (117) (t=1.25 pixel; 43 iterations) How about outlier rejection? - use SSD (patch1, patch2) > threshold - 1-NN: SSD of the closest matches How to handle too many outliers? RANSAC loop: Select four feature pairs (at random) Compute homography H (exact) Compute inliers where  SSD(pi’, H pi) < ε Keep largest set of inliers Re-compute least-squares H estimate on all of the inliers Final inliers (262)

Before we talk about the projection matrix of the depth correspondces, we need to know two things: - Camera extrinsics - Camera intrinsics Camera extrinsics maps the world coorinates to the camera coordinates. For the simplicity of the camera, it is a pinhole camera without lenses.  I'll talk about the lenses, the focal length, the lense aberation, the pixel sensor dimension, etc in Camera intrincs. So, locating an object in two images and projecting in the camera space is not that straight. But, it will be a straight process with the application of Machine Learning. I'll talk about the next part of the series in applying the deep neural network to optimizing the homographic projection and have it robust in low texture settings including low light. Deep Neural Network - Estimating Homography to address: - low texture environment - outside light conditions ( gamma > 2kLs) - robust as or better than SfM or other SLAM techniquese First, we need to locate the ...

Direct Linear Transform (DLT) Using Direct Linear Transformation to find Homography. With the 4 correspondences, we can solve Direct Linear Transform (DLT) for the homography matrix H. However, a correspondence model may contain both inliers and outliers. Random Sample Consensus (RANSAC) In order to filter out the outliers, to reconstruct the model, use RANSAC in an iterative fashion. For example, for every set of 4 correspondences randomly picked, the occurrence of H is calculated. Using this H value, we can determine the outlier correspondences. With only inliers, we can recalculate H for each set of 4 correspondences.

How to Create a holographic display and camcorder In the last part of the series "How to Create a Holographic Display and Camcorder", I talked about what the interest points, descriptors, and features to find the same object in two photos. In this part of the series, I'll talk about how to extract the depth of the object in two photos by calculating the disparity between the photos. In order to that, we need to construct a triangle mesh between correspondences. To construct a mesh, we will use Delaunnay triagulation.  Delaunnay Triagulation - It minimizes angles of all triangles, while the sigma of triangles is maximized. The reason for the triangulation is to do a piece wise affine transformation for each triangle mapped from a projective plane A to a projective plane B. A projective plane A is of a camera projective view at time t, while a projective plane B is of a camera projective view at time t+1. (or, at t-1.  It really doesn't matter)...