You are here: / Accueil

- PointsToTrack: given an image, this block must be able to find points and/or features to track. OpenCV has already a lot of traitments, just have to create wrappers...
- PointsMatched: given points and/or features, this block must be able to find correspondances. If only images are known, use first
*PointsToTrack*to find points... OpenCV has a lot of algo too, so not a lot of work. - ExternPosEstim: given points and/or intra parameters of cameras and/or matches and/or known 3D points, this block have to estimate the extern parameters of cameras (position, orientation) and intra parameters if needed. Same as before, a lot of methods exist in OpenCV or using toolbox of Mr Rabaud!
- Points3DEstim: given 2D points and/or correspondances and/or intra parameters and/or extern parameters and/or known 3D points, this block have to estimate the 3D positions of each 2D points sets. Various methods exists (Middle point, iterative least square...), choice of methods has to be done.
- Object: given 3D points from differents field of view and/without correspondances, find commun objects between views. A lot of algorithms exist in PCL, so just need wrappers...

Exchange test data (videos, internals parameters of cameras, sequences fully parameterized...) with mentor.

Exchange informations from studients working on converters for OpenCV and PCL.

Working on a first 3D visualization (simple, just to be able to debug algorithms). Probably just a wrapper of PCLVisualizer

Focus on 2D algorithms: Features and points detections, matching... Compare and choose a good compromise between pcl::Keypoint, cv::Point and a personal new class (

Begin the implementation of

Focus on simple case: the object is planar -> homography is sufficent to model the images relations. I have already coded the most part of this algo, only camera positions are missing (but easy to add).

Try to add a surface variation estimation using

I have already coded a super-resolution algorithm for document image (supposed planar) so the implementation will not be a problem.

Begin with simple SfM algorithms: no missing points match, no errors (matrix factorization). Just an orthographic reconstruction.

Check how to use various infos on structure (internal camera parameters, numbers of cameras...) to improve structure (projective reconstruction?).

Using previous SfM algo, try find meshes and add textures.

Improve the 3D visualization method.

Write tests, improve documentation...

Projective reconstruction.

Dense objects estimation with texture.

Visualization of 3D objects using camera motion (If time, try to improve camera motion using splines).

Write tests, improve documentation...

Focus on 3D estimation:

Let's play with cameras :

//Camera creation :

Camera cam(WEBCAM, 0);//camera from webcam 0

Camera cam1(AVI_FLUX,"video1.avi");//camera from video

Camera cam2(PICT_FLUX,"prefixe",".png",0,100);//camera from images (prefixe0.png -> prefixe100.png)

How will we change or compute things with our nice camera ?

cam2.setInternalParameters(Mat::eye(3,3,CV_64F));//set the internal parameters of cam2...

//If we want to do some steps with particular methods, we can then doing it like that:

//Example: use SURF points detections and BruteForceMatching

SURFDetect algoDetect(params);//See documentation for params details

cam.setDetectAlgo(algoDetect);//Now when points are needed for a FoV from *cam*, SURF method will be used.

BFMatcher

cam.setMatchAlgo(matchsAlgo);//Now if we need matching, brute force will be used.

vector<vector<Point3f>> objectPoints=*???*;//3D points of the calibration pattern in its coordinate system...

cam1.calibrateCamera(objectPoints)//Compute intra and extern parameters using images *cam1* have (using cv::calibrateCamera)...

What can we get from a camera ?

cv::Mat K=cam3.getInternalParameters(CV_PROJECTIVE,CV_FM_RANSAC);//get the internal coefs of cam3... The parameter *CV_PROJECTIVE* says that if internal coefs are not computed then try to estimate the coefs using cv::cvFindFundamentalMat and cv::decomposeprojectionmatrix (We can find projection matrices using decomposition of F)

//Of course, this estimation can be upgraded to perspective or metric recontruction if wanted using different parameters... Details not yet found.

cv::Mat img=cam3.getImage(0);//get the first image of cam3...

pcl::PointCloud<pcl::PointXYZ> world=cam3.getAllObjects();//get all objects found with cam3...

How can we create field of views ?

FieldOfView p1=cam1.getFoV(3);//Get the 3rd field of view from cam1

FieldOfView pNew("pictX.jpg");//Create a new FoV. This will create a new empty camera

pNew.setCam(cam1);//set cam1 as the camera of pNew (equal to p1.getCam())

FieldOfView p2=cam.getCurrentFoV();//Get the current FoV (for video or picture sequences, increment the frame)

OK, but if we have different devices (2 or more differents intra params), like cam1, cam2 and cam3: how can we compute the differents parameters of the two cameras?

Two options :

If we have several images for the differents cameras, we can compute intra and extern parameters, object points for each camera and then merge them:

cam1.compute3DObjects();

cam2.compute3DObjects();

//Now merge the two objects:

Camera::merge({cam1,cam2});//the parameter will be a vector of Camera. Each camera inside the vector will be updated!

Ok, and what for seeing theses points/meshes/textured objects?

//If we want to show all objects of cam1 as viewed by the first FoV:

cam1.showObjects("myWindow",cam1.getFoV(0));

//Or using particular position (position is a 3*3 matrix):

cam1.showObjects("myWindow",position);

//Or with an animation reproducing the camera motion:

PointDeVue::showObjet(world,cam);