### Setup: ##################################### * _______ _______ * * /\ \ / \ / /\ * * / \ \___/ \___/ / \ * * / / CAM2 CAM1 \ \ * * /___/ \___\ * * CAM0 ______ CAM3 * * / \ * * / \ * * \ / * * \______/ * * LiDAR * * * ################################################ CAM0 Intrinsics: - fx = fy = 120 - [u0, v0] = [960.5, 600.5] | 120.0 0.0 960.5 | - K = | 0.0 120.0 600.5 | | 0.0 0.0 1.0 | CAM0 FisheyeParams: - MappingCoefficients: [562.1736 -7.1999e-04 4.3601e-07 -4.9236e-10] - ImageSize: [1200 1920] - DistortionCenter: [953.4174 593.1634] - StretchMatrix: |1 0| |0 1| CAM1 Intrinsics: - fx = 1280.7 - fy = 1281.2 - [u0, v0] = [969.4257, 639.7227] - RadialDistortion: [-0.0020 -0.0015] | 1280.7 0.0 969.4257 | - K = | 0.0 1281.2 639.7227 | | 0.0 0.0 1.0 | CAM2 Intrinsics: - fx = 1276.1 - fy = 1275.4 - [u0, v0] = [965.9650, 618.2222] - RadialDistortion: [0.0157 -0.0104] | 1276.1 0.0 965.9650 | - K = | 0.0 1275.4 618.2222 | | 0.0 0.0 1.0 | CAM3 Intrinsics: - fx = fy = 120 - [u0, v0] = [960.5, 600.5] | 120.0 0.0 960.5 | - K = | 0.0 120.0 600.5 | | 0.0 0.0 1.0 | CAM3 FisheyeParams: - MappingCoefficients: [576.4125 -6.7844e-04 3.5519e-07 -4.2788e-10] - ImageSize: [1200 1920] - DistortionCenter: [972.8745 663.9413] - StretchMatrix: |1 0| |0 1| ### LiDAR to Camera Transform explanation ####################### * - R: rotation, 3x3 * * - t: translation, 1x3 * * - T: transformation matrix * * | | 0 | * * | R | 0 | * * T = |___| 0 | * * | t | 1 | * * - Forward rigid transformation: [x y z 1] = [u v w 1] * T * ################################################################# LiDAR --> CAM0: | -0.0153657793937061 -0.00365119568008550 -0.999875273018455 | - R = | 0.999773464238133 0.0146735043624255 -0.0154177973462293 | | 0.0147279675755847 -0.999885671984468 0.00342489872272477 | - t = [ 0.00365920154169145 -0.101481126401536 -0.183123241266722 ] | -0.0153657793937061 -0.00365119568008550 -0.999875273018455 0 | - T = | 0.999773464238133 0.0146735043624255 -0.0154177973462293 0 | | 0.0147279675755847 -0.999885671984468 0.00342489872272477 0 | | 0.00365920154169145 -0.101481126401536 -0.183123241266722 1 | LiDAR --> CAM1: | 0.938087304116687 -0.000283237125790693 0.346398801429533 | - R = | -0.346205150200322 -0.0342096517324521 0.937534902658625 | | 0.0115846376662857 -0.999414638428453 -0.0321896981201434 | - t = [ 0.0737416004843908 -0.0317341412646937 -0.178044453316751 ] | 0.938087304116687 -0.000283237125790693 0.346398801429533 0 | - T = | -0.346205150200322 -0.0342096517324521 0.937534902658625 0 | | 0.0115846376662857 -0.999414638428453 -0.0321896981201434 0 | | 0.0737416004843908 -0.0317341412646937 -0.178044453316751 1 | LiDAR --> CAM2: | 0.936872686224469 0.00407855516588752 -0.349646872135739 | - R = | 0.349521572195037 -0.0401194110186825 0.936068962966844 | | -0.0102098176700750 -0.999186568288061 -0.0390123233846818 | - t = [ 0.0220629410603821 -0.0314601062659720 -0.196604790031003 ] | 0.936872686224469 0.00407855516588752 -0.349646872135739 0 | - T = | 0.349521572195037 -0.0401194110186825 0.936068962966844 0 | | -0.0102098176700750 -0.999186568288061 -0.0390123233846818 0 | | 0.0220629410603821 -0.0314601062659720 -0.196604790031003 1 | LiDAR --> CAM3: | 0.0196090975554365 -0.0482366508949077 0.998643434266457 | - R = | -0.999590939857959 -0.0217447407207396 0.0185773842311254 | | 0.0208191317528957 -0.998599214781101 -0.0486433139457517 | - t = [ 0.000343481617694225 -0.0362336504224814 -0.137639495670372 ] | 0.0196090975554365 -0.0482366508949077 0.998643434266457 | - T = | -0.999590939857959 -0.0217447407207396 0.0185773842311254 | | 0.0208191317528957 -0.998599214781101 -0.0486433139457517 | | 0.000343481617694225 -0.0362336504224814 -0.137639495670372 | ### Stereo explanation ##################################################################### * * * - P1: point in image1 (camera 1) in pixel coordinates * * - P2: point in image2 (camera 2) in pixel coordinates * * - F: fundamental matrix * * [P1 1] * F * | P2 | = 0 * * | 1 | * * * * - Q1: point in image1 (camera 1) in normalized image coordinates, * * where the origin is at the camera’s optical center. * * The x and y pixel coordinates are normalized by the focal length fx and fy. * * - Q2: point in image1 (camera 1) in normalized image coordinates, * * where the origin is at the camera’s optical center. * * The x and y pixel coordinates are normalized by the focal length fx and fy. * * - E: essential matrix * * [Q1 1] * E * | Q2 | = 0 * * | 1 | * * - R, t: relative pose of the second camera * * orientation1 = orientation2 * R * * location1 = orientation2 * t + location2 * * * ############################################################################################ CAM0 (camera 1) --> CAM2 (camera 2): | 5.55040115548871e-07 0.000718421864392740 -0.430286874505905 | - F = | 0.000717318615105220 1.26807155980575e-05 -0.573607466776144 | | -0.415560569370984 -2.00792963754815 1529.30178062434 | | 0.0849935365081733 110.012255418965 2.11648099522374 | - E = | 109.785651864411 1.94078419090761 156.865887394320 | | 3.41254337981043 -156.734372040631 2.03250758446401 | | 0.340941122147217 0.0178192514304566 -0.939915754473482 | - R = | -0.0294708880197221 0.999531515450624 0.00825932081403361 | | 0.939622593379025 0.0248842098415501 0.341306545662746 | - t: [ -156.744727960585 3.53979871362266 -109.975692237599 ] CAM1 (camera 1) --> CAM2 (camera 2): | -7.15574776578017e-07 2.36366718461812e-05 -0.0133930185607378 | - F = | 1.75203949229116e-05 -3.59607040668389e-07 -0.0909359968898541 | | -0.0111398641028333 0.0483997575140576 22.9336066721754 | | -1.16941141740456 38.6451610793471 1.31972927081527 | - E = | 28.6172668713250 -0.587636736743662 -94.6121299957232 | | -1.28012630574243 90.9799750610368 -1.76478042721460 | | 0.767878652237968 0.00135995194270285 0.640593885365693 | - R = | 0.00272319999182272 0.999981780904770 -0.00538721081162694 | | -0.640589540672501 0.00588118944074345 0.767860958762559 | - t = [ 90.9843386924595 1.98976081920328 -38.6341627985188 ] CAM1 (camera 1) --> CAM3 (camera 2): | -5.56609014244981e-05 0.000601523107664508 -0.344551998376106 | - F = | 0.000898008361551627 3.36822365710805e-05 0.366887304166810 | | -0.478699930927659 -0.740228259370356 189.499493662874 | | -8.55388695045895 92.4830356060761 -1.64437542805360 | - E = | 138.004628180503 5.17857991555810 151.078438986404 | | 9.08213930679450 -182.241636356280 -5.24550618299770 | | 0.351911003829117 0.0159082021153068 -0.935898271442704 | - R = | -0.0397328487714319 0.999208247514709 0.00204421796575303 | | 0.935189791492876 0.0364665216883038 0.352264455606107 | - t = [ -182.468928313348 -5.20622814489955 -92.7463207676057 ]