Get camera parameters from the OpenGL modelview matrix

OpenGL modelview matrix includes model related operations, therefore, the scene was scaled, translated, or rotated, these effects are also in there. There is no way to separate these operation from the pure camera parameters from the matrix only. Here we assume the model operation is identity. If your renderer separated the model operations and camera parameters, you need to undo the model operation with your implementation dependent way. Our renderer has separation of them, so we needed to undo the effect.

The elements of OpenGL modelview matrix is the following. If you want to know the details of the each element, please refer my other blog (http://shitohichiumaya.blogspot.de/2011/01/what-matrix-glulookat-generates-1.html)
\begin{eqnarray*}
\left[
\begin{array}{cccc}
x_x & y_x & z_x & 0 \\
x_y & y_y & z_y & 0 \\
x_z & y_z & z_z & 0 \\
-(\vec{x} \cdot \vec{e}) &
-(\vec{y} \cdot \vec{e}) &
-(\vec{z} \cdot \vec{e}) & 1 \\
\end{array}
\right]
\end{eqnarray*}
Let rewrite this matrix with replacing the 4th row with $$a,b,c$$.
\begin{eqnarray*}
\left[
\begin{array}{cccc}
x_x & y_x & z_x & 0 \\
x_y & y_y & z_y & 0 \\
x_z & y_z & z_z & 0 \\
a   & b   & c   & 1 \\
\end{array}
\right]
\end{eqnarray*}
This matrix has camera basis, the only problem is the eye position. This is in the 4th row, 1-3 columns as shown in the following. (Here, the matrix and vector are written in the column vectors $$\vec{x}, \vec{y}, \vec{z}, \vec{e}$$)
\begin{eqnarray*}
\left[
\begin{array}{ccc}
& & \\
\vec{x} & \vec{y} & \vec{z}\\
& & \\
\end{array}
\right]
\left[
\begin{array}{c}
\\
\vec{e} \\
\\
\end{array}
\right]
& = &
\left[
\begin{array}{c}
a \\
b \\
c \\
\end{array}
\right]
\end{eqnarray*}
Therefore,
\begin{eqnarray*}
\left[
\begin{array}{c}
\\
\vec{e} \\
\\
\end{array}
\right]
& = &
\left[
\begin{array}{ccc}
& & \\
\vec{x} & \vec{y} & \vec{z}\\
& & \\
\end{array}
\right]^{-1}
\left[
\begin{array}{c}
a \\
b \\
c \\
\end{array}
\right]
\end{eqnarray*}
is the eye position. The implementation example in C++ of those is the
following.

/// get camera parameters from
/// the OpenGL modelview matrix
///
/// \param[out] eyepos  eye position
/// \param[out] viewdir viewing direction
/// \param[out] updir   up direction vector
void gl_get_camera_parameters_from_modelview_matrix(
Vector3d & eyepos,
Vector3d & viewdir,
Vector3d & updir)
{
GLdouble mat[16];
glGetDoublev(GL_MODELVIEW_MATRIX, mat);

Vector3d xdir(0.0, 0.0, 0.0);
Vector3d ydir(0.0, 0.0, 0.0);
Vector3d zdir(0.0, 0.0, 0.0);

xdir[0] = mat[0];  ydir[0] = mat[1];  zdir[0] = mat[2];
xdir[1] = mat[4];  ydir[1] = mat[5];  zdir[1] = mat[6];
xdir[2] = mat[8];  ydir[2] = mat[9];  zdir[2] = mat[10];

// This is a, b, c components.
Vector3d bvec(-mat[12], -mat[13], -mat[14]);

Matrix33d basis_mat(xdir[0], xdir[1], xdir[2],
ydir[0], ydir[1], ydir[2],
zdir[0], zdir[1], zdir[2]);
// This matrix should not be singular.
// invert() gives matrix inverse.
bool const is_inv = basis_mat.invert();
assert(is_inv);

eyepos  = basis_mat * bvec;
viewdir = -zdir;
updir   =  ydir;
}


For simplicity again, I omitted the error checks.

Conclusion
In this blog, I showed
How to get perspective camera parameters from the OpenGL perspective matrix.

How to get other camera parameters (i.e., camera position, view direction, and eye position) from the OpenGL modelview matrix.

There is not so much case that this is needed, but I think some people may find this is interesting.