dickscheid.net

How to convert Blender camera settings into a projection matrix

I was looking for an easy way to derive a projection matrix corresponding to the camera of images rendered with blender. Given such a 3×4 projection matrix P, I want to define a 3D point in the blender scene, multiply it with the matrix and paint it into the existing image. For example, using Matlab to draw the scene origin as a red plus sign onto the image, I want to write

imshow('blender_img.jpg'); hold on; X = [0 0 0 1]'; x = P*X; x = x(1:2)/x(3); plot(x(1),x(2),'r+'); hold off;

Deriving P turns out to be not documented at all, and besides knowledge on the computer graphics pipelines requires some trial and error. Today I managed to solve the problem, so here's the solution.

First of all, collect the camera position and orientation from the "Transform Properties" dialog, as shown in the image above. In Matlab you get the translation vector and rotation matrix via

X0 = [2 -10 4]'; o = 70*pi/180; p = 15*pi/180; k = -10*pi/180; Ro=[1 0 0; 0 cos(o) -sin(o); 0 sin(o) cos(o)]; Rp=[cos(p) 0 sin(p); 0 1 0; -sin(p) 0 cos(p)]; Rk=[cos(k) -sin(k) 0; sin(k) cos(k) 0; 0 0 1]; R = Rk*Rp*Ro;

Now we need to derive the calibration matrix. Unfortunately, the focal length value in blender is neither documented nor standard. However, it is possible to display the lens angle alpha in degrees, referring to the wider image dimension, as shown in the image below. In the example we have alpha=49.13.

From the image resolution, which is here assumed to be 800×600, we get the focal length in pixel from fl=-400/(49.13*pi/180/2). Observe the minus!

The final projection matrix P is obtained in Matlab using

fl = -400/(49.13*pi/180/2); K = [fl 0 400; 0 fl 300; 0 0 1]; P = K*diag([1 -1 1])*R'*[eye(3) -X0];

Note the flipping of the y axis which results in a left handed system. A
student is planning to write an export script for blender to write
P to file, which I'll post here.

Eigen: A convenient and fast C++ template library for linear algebra

I just discovered eigen, a really nice template library for linear algebra. I was using newmat until now, but was not very satisfied with its API design, and was looking for a single efficient library for both small (3×3) and large matrices. That is were eigen comes in: It has a very intuitive and handy API and can handle both fixed and dynamic-sized matrices in a unified way. Some highlights are

Direct mapping of STL vector and C array memory into eigen's classes
Classes for transformations, including quaternions (!)
Sparse matrix support
Handy typedefs for common structures, i.e. Vector3f for a 3-float-vector and Matrix2d for a 2×2-double matrix
Comma initialization aka Vector3f X = Vector3f::Zero(); X << 1, 2, 3;

Eigen is a pure template library, meaning that you can include it in your project by copying the 1-2 MB of Header files into your source tree.

Really great work!

Don't use underscores in Java class names accessed through JNI

You can use the Java Native Interface to call native code written in C++ directly from Java - and vice versa, which is less often used. When calling Java from C/C++, a virtual machine has to be launched before calling classes, using the so-called "Invocation Interface". A simple example can be found here.

It is important that you do not use underscores in the Java class names when implementing this scenario - JNI won't find your classes then, as happened to me using Sun's Java 6 JRE. Unnecessary to mention that it took Thomas and me about on hour to find the problem...

Debugging and profiling C++ under Linux

Debugging and Profiling C++-Code with the standard GNU tools under Linux may not be as intuitive too many people as it is with bigger IDE's like Visual Studio, XCode or Eclipse. Although I still think that using the GNU debugger gdb and profiler gprof on the command line is feasible, there are more userfriendly choices available:

Nemiver is a stand-alone graphical debugger for Gnome that can use gdb as a backend. I found it intuitive, well-designed and stable. I features easy browsing of sourcecode, setting breakpoints, and especially save and resume for sessions.
cgdb is a curses frontend to gdb, that is, a graphical user interface on the command line. It is also intuitive, although tuned for vim-experienced users.
A comfortable way of profiling your code is to use the KDE call-graph viewer kcachegrind with the callgrind tool of the valgrind suite. Other than using gprof, it does not require compiling your code with special flags. It requires no more than two steps: (1) Calling your binary within the valgrind framework: valgrind --tool=callgrind [program] [arguments]. (2) Running kcachegrind on the output file kcachegrind callgrind.out.[pid]

Note that what you get with callgrind is not an exact analysis of the time your program consumed in every function and line, but rather an analysis of the cache accesses and processor instructions. This is of similar importance for many profiling purposes, but one should be aware of the differences, and read the manuals carefully for more exact usage of profilers.

On ubuntu, you get the whole setup via aptitude install valgrind-callgrind kcachegrind nemiver cgdb.

What if your STL containers get too large?

I recently encountered a problem with a std::vector of objects needing more memory than available on the system. This occurs easily when using the standard allocation procedure, which re-allocates the vector to double size once it needs more memory, and hence suddenly needs triple the amount of memory. On a 32 bit system the limit is at 2GB, so a 600+MB vector is probably too big already.

What is the solution to this problem? First of all, you should pre-allocate the vector and check max_size(), and also catch the allocation exception that vector throws. You may also consider using more memory-efficient datatypes for your problem: Have a look at the boost flyweight library, if you have many items taking on the same values, for example. Second, rethink your algorithm - that's what I did. In most cases there is no need to hold such massive data in memory.

However you might end up searching for a way to stream objects to disk seamlessly "in the background" while still using the familiar STL-like syntax. In such a case have a look at the STXXL library (the "Standard Template Library for Extra Large Data Sets") - it's just that.

Creating high-quality graphics in LaTeX

TikZ

Almost every LaTeX user loves and hates the good old xfig vector graphics editor. I sticked quite long to it, but converted to the incredibly powerful TikZ package of Till Tantau from Lübeck. It enables you to create high-quality vector graphics, node-based diagrams and much more directly from your LaTeX document. You should check the immense amount of examples in the Manual. Great German tutorials for TikZ can be found on www.statistiker-wg.de.

Sketch 3D

If you want to create high-quality sketches in 3D, and not use google's SketchUp for whatever reason, you should have a look Sketch 3D and the very good tutorials on fauskes.net. The idea of Sketch 3D is simple:

You describe a simple 3D scene sketch in a separate textfile
Sketch creates a plain LaTeX file by applying an orthographic projection of the 3D sketch, which is defined by camera viewpoint and orientation.
The LaTeX file is then included directly in your document.

Highlights of the package are the "sweep"-commands, which repeat a 3D drawing under a spatial motion and connect the resulting objects into a closed solid, and the "fspecial" function allowing to insert pstricks or TikZ commands at projections of 3D positions, taking proper care of the z-Buffer. This allows for example to insert LaTeX formulas right into your figures in proper size and rendering without using good old pstricks.

For a first impression have a look at this slide from my talk at ICVS'09 in Liege, Belgium, where I modelled a camera at different 3D positions, projecting a 3D object. Such an image would be immense work in xfig, not to mention what happens if you want to change the viewpoint later on.

Include files missing with gcc 4.4

Switching from gcc 4.3 to 4.3 brought lots of missing include files for standard c functions. Martin Michlmayr has a good list of the necessary #include's here .