1

With FeatureDetector I get features on two images with the same element and match this features with BruteForceMatcher. Then I'm using OpenCv function findHomography to get homography matrix

H = findHomography( src2Dfeatures, dst2Dfeatures, outlierMask, RANSAC, 3);

and getting H matrix, then align image with 

warpPerspective(img1,alignedSrcImage,H,img2.size(),INTER_LINEAR,BORDER_CONSTANT);

I need to know rotation angle, scale, displacement of detected element. Is there any simple way to get this than some big equations? Some evaluated formulas just to put data in?

Samuel Liew
  • 76,741
  • 107
  • 159
  • 260
krzych
  • 2,126
  • 7
  • 31
  • 50

3 Answers3

3

Homography would match projections of your elements lying on a plane or lying arbitrary in 3D if the camera goes through a pure rotation or zoom and no translation. So here are the cases we are talking about with indication of what is the input to our calculations:
- planar target, pure rotation, intra-frame homography
- planar target, rotation and translation, target to frame homography
- 3D target, pure rotation, frame to frame mapping (constrained by a fundamental matrix)

In case of the planar target, a pure rotation is easy to calculate through your frame-to-frame Homography (H12): given intrinsic camera matrix A, plane to image homographies for frame H1, and H2 that can be expressed as H1=A, H2=A*R, H12 = H2*H1-1=ARA-1 and thus R=A-1H12*A

In case of elements lying on a plane, rotation with translation of the camera (up to unknown scale) can be calculated through decomposition of target-to-frame homography. Note that the target can be just one of the views. Assuming you have your original planar target as an image (taken at some reference orientation) your task is to decompose the homography between images H12 which can be done through SVD. The first two columns of H represent the first two columns of the rotation martrix and be be recovered through H=ULVT, [r1 r2] = UDVT where D is 3x2 Identity matrix with the last row being all 0. The third column of a rotation matrix is just a vector product of the first two columns. The last column of the Homography is a translation vector times some constant.

Finally for arbitrary configuration of points in 3D and pure camera rotation, the rotation is calculated using the essential matrix decomposition rather than homography, see this

Community
  • 1
  • 1
Vlad
  • 4,425
  • 1
  • 30
  • 39
  • "[r1 r2] = UDV^T where D is 3x2 Identity matrix with the last row being all 0", U,V are 3x3 matrix, how can D be a 3x2 matrix? – C. Wang Jul 20 '15 at 07:52
  • SVD decomposition acts on non square matrices as opposed to eigenvalue decomposition that requires a square matrix. You got the dimensions of V wrong. They are mxn = mxm * mxn * nxn – Vlad Jul 21 '15 at 15:42
  • Thanks for your answer, but I am still confused, because H is a 3x3 matrix, and applying SVD to H, saying [U,S,V] = svd(H), will produce a 3x3 matrices U, S and V. Could you explain it a little more accurately? Thanks! – C. Wang Jul 22 '15 at 03:29
  • The first two columns of H represent the first two columns of the rotation matrix... – Vlad Jul 23 '15 at 04:19
1

You can try to use levenberg marquardt optimalization, where parameters will be translation and rotation, equations will represent by computed distances between features from two images(use only inliers from ransac homography). Here is C++ implementation of LM http://www.ics.forth.gr/~lourakis/levmar/

VisionC
  • 106
  • 4
1
cv::decomposeProjectionMatrix();

and 

cv::RQDecomp3x3();

are both similar to what you want to achive.

None of them is perfect. The theory behind them and why you cannot extract all params from a 3x3 matrix is a bit cumbersome. But the short answer is that a 3x3 proj matrix is a simplification from the complete 4x4 one, based on the fact that all points stay in the same plane.

Sam
  • 19,708
  • 4
  • 59
  • 82
  • All points in my case stay in the same plane cause I'm using flat elements, there's only rotation, displacement and change of scale. In decomposeProjectionMatrix I can get only displacement vector and rotation matrix, how can I get scale? – krzych Nov 18 '11 at 12:56
  • scale depends on the camera intrinsic parameters (focal distance, field of view), but I do not know any easy or general way to extract it. – Sam Nov 18 '11 at 13:01
  • Sorry but I still don't get it. For H = [225,225,112;154,175,175; 208,63,171] get R = [159,53,91;6,49,63;233,63,112] and Ortho matrix = [243,71,56;60,212,16;,214,63,150] How to get rot angle and disp x,y. How opencv warpPerspective evaluates scale? – krzych Nov 18 '11 at 13:25
  • :) welcome in the 3D world. It's cumbersome and difficult to understand. If you still want to get it here is a nice post. But do not expect to have everything in half a day. It may need weeks of work, math and hacking to understand it. http://stackoverflow.com/questions/6606891/opencv-virtually-camera-rotating-translating-for-birds-eye-view – Sam Nov 18 '11 at 14:11
  • cv::RQDecomp3x3() returns rotation only. If there is translation involved you have to use SVD of Homography to restore the first two columns of a rotation matrix and find the third one as their vector product. The translation up to scale is the last column of your homography. – Vlad Mar 11 '14 at 20:16