作业描述
本次需要完成的任务是:
- 修改函数 rasterize_triangle(const Triangle& t) in rasterizer.cpp: 在此处实现与作业 2 类似的插值算法，实现法向量、颜色、纹理颜色的插值。
- 修改函数 get_projection_matrix() in main.cpp: 将之前的实验中实现的投影矩阵填到此处，此时可以运行 ./Rasterizer output.png normal 来观察法向量实现结果。
- 修改函数 phong_fragment_shader() in main.cpp: 实现 Blinn-Phong 模型计算 Fragment Color.
- 修改函数 texture_fragment_shader() in main.cpp: 在实现 Blinn-Phong 的基础上，将纹理颜色视为公式中的 kd，实现 Texture Shading Fragment Shader.
- 修改函数 bump_fragment_shader() in main.cpp: 在实现 Blinn-Phong 的 基础上，仔细阅读该函数中的注释，实现 Bump mapping.
- 修改函数 displacement_fragment_shader() in main.cpp: 在实现 Bump mapping 的基础上，实现 displacement mapping.
附加题：
- 尝试更多模型: 找到其他可用的.obj 文件，提交渲染结果并把模型保存在 /models 目录下。这些模型也应该包含 Vertex Normal 信息。
- 双线性纹理插值: 使用双线性插值进行纹理采样, 在 Texture 类中实现一个新方法 Vector3f getColorBilinear(float u, float v) 并 通过 fragment shader 调用它。为了使双线性插值的效果更加明显，应该考虑选择更小的纹理图。请同时提交纹理插值与双线性纹理插值的结果，并进行比较。
FAQ
http://games-cn.org/forums/topic/frequently-asked-questionskeep-updating/
- bump mapping 部分的 h(u,v)=texture_color(u,v).norm, 其中 u,v 是 tex_coords, w,h 是 texture 的宽度与高度
- rasterizer.cpp 中 v = t.toVector4()
- get_projection_matrix 中的 eye_fov 应该被转化为弧度制
- bump 与 displacement 中修改后的 normal 仍需要 normalize
- 可能用到的 eigen 方法：norm(), normalized(), cwiseProduct()
- 实现 h(u+1/w,v) 的时候要写成 h(u+1.0/w,v)
解题思路
1. 实现法向量、颜色、纹理颜色的插值
基本流程和作业2差不多，因为投影可能改表三角形的重心坐标，需要做一个透视矫正。按照 Barycentric Coordinates 对法向量、颜色、纹理颜色与底纹颜色 (Shading Colors) 进行插值，将它们传递给fragment_shader_payload，调用 fragment shader（片元着色器） 得到计算出的颜色写入 framebuffer。
//Screen space rasterization
void rst::rasterizer::rasterize_triangle(const Triangle& t, const std::array<Eigen::Vector3f, 3>& view_pos) {
// DONE: From your HW3, get the triangle rasterization code.
auto v = t.toVector4();
// Find out the bounding box of current triangle.
float x_min = std::min(v[0][0], std::min(v[1][0], v[2][0]));
float x_max = std::max(v[0][0], std::max(v[1][0], v[2][0]));
float y_min = std::min(v[0][1], std::min(v[1][1], v[2][1]));
float y_max = std::max(v[0][1], std::max(v[1][1], v[2][1]));
// 取整，保证三角形在包围盒内
int left = std::floor(x_min);
int right = std::ceil(x_max);
int bottom = std::floor(y_min);
int top = std::ceil(y_max);
// iterate through the pixel and find if the current pixel is inside the triangle
for (int x = left; x <= right; ++x) {
for (int y = bottom; y <= top;++y) {
if (insideTriangle(x, y, t.v)) {
// DONE: Inside your rasterization loop:
// 获取重心坐标分量
auto [alpha, beta, gamma] = computeBarycentric2D(x, y, t.v);
// 投影时三角形重心会变，所以要使用透视校正插值
// * v[i].w() is the vertex view space depth value z.
// * Z is interpolated view space depth for the current pixel
float Z = 1.0 / (alpha / v[0].w() + beta / v[1].w() + gamma / v[2].w());
// * zp is depth between zNear and zFar, used for z-buffer
float zp = alpha * v[0].z() / v[0].w() + beta * v[1].z() / v[1].w() + gamma * v[2].z() / v[2].w();
zp *= Z;
if (zp < depth_buf[get_index(x, y)]) {
// DONE: Interpolate the attributes:
// auto interpolated_color 对颜色进行线性插值
auto interpolated_color = interpolate(alpha, beta, gamma, t.color[0], t.color[1], t.color[2], 1.0);
// auto interpolated_normal 对法向量进行线性插值
auto interpolated_normal = interpolate(alpha, beta, gamma, t.normal[0], t.normal[1], t.normal[2], 1.0);
// auto interpolated_texcoords 对纹理坐标uv进行线性插值
auto interpolated_texcoords = interpolate(alpha, beta, gamma, t.tex_coords[0], t.tex_coords[1], t.tex_coords[2], 1.0);
// auto interpolated_shadingcoords 视图空间坐标进行线性插值
//view_pos[]是三角形顶点在view space中的坐标,插值是为了还原在camera space中的坐标，详见http://games-cn.org/forums/topic/zuoye3-interpolated_shadingcoords/
auto interpolated_shadingcoords = interpolate(alpha, beta, gamma, view_pos[0], view_pos[1], view_pos[2], 1.0);
fragment_shader_payload payload(interpolated_color, interpolated_normal.normalized(), interpolated_texcoords, texture ? &*texture : nullptr);
payload.view_pos = interpolated_shadingcoords;
// Use: Instead of passing the triangle's color directly to the frame buffer, pass the color to the shaders first to get the final color;
auto pixel_color = fragment_shader(payload);
depth_buf[get_index(x, y)] = zp;
set_pixel(Eigen::Vector2i(x, y), pixel_color);
}
}
}
}
}
2. 修改投影矩阵
因为作业框架与老师授课所用的坐标系不一样（左手系和右手系的差别），在之前的作业里为了避免图形上下颠倒人为把Z轴方向反转，直接使用作业2的投影矩阵输出结果发现渲染有坏点。
将两个变换矩阵合并为一个的情况下图像显示正常，推测坏点是浮点数运算导致的，计算出投影矩阵以后需要使用旋转矩阵将图形逆时针旋转180度。
Eigen::Matrix4f get_projection_matrix(float eye_fov, float aspect_ratio,
float zNear, float zFar) {
// Students will implement this function
Eigen::Matrix4f projection = Eigen::Matrix4f::Identity();
// DONE: Implement this function
// Create the projection matrix for the given parameters.
// Then return it.
float n = -zNear; // Z轴反向
float f = -zFar; // Z轴反向
float t = std::tan(eye_fov / 180 * MY_PI / 2) * std::abs(n);
float r = aspect_ratio * t;
float b = -t;
float l = -r;
Eigen::Matrix4f temp;
temp <<
2 * n / (r - l), 0, (l + r) / (l - r), 0,
0, 2 * n / (t - b), (b + t) / (b - t), 0,
0, 0, (n + f) / (n - f), 2 * f * n / (f - n),
0, 0, 1, 0;
projection = temp * projection;
// 坐标系变换
Eigen::Matrix4f tran;
tran <<
1, 0, 0, 0,
0, 1, 0, 0,
0, 0, -1, 0,
0, 0, 0, 1;
projection = tran * projection;
return projection;
}
3. Blinn-phong 反射模型
按照公式实现即可：
Eigen::Vector3f phong_fragment_shader(const fragment_shader_payload& payload) {
Eigen::Vector3f ka = Eigen::Vector3f(0.005, 0.005, 0.005);
Eigen::Vector3f kd = payload.color;
Eigen::Vector3f ks = Eigen::Vector3f(0.7937, 0.7937, 0.7937);
auto l1 = light{ {20, 20, 20}, {500, 500, 500} };
auto l2 = light{ {-20, 20, 0}, {500, 500, 500} };
std::vector<light> lights = { l1, l2 };
Eigen::Vector3f amb_light_intensity{ 10, 10, 10 };
Eigen::Vector3f eye_pos{ 0, 0, 10 };
float p = 150;
Eigen::Vector3f color = payload.color;
Eigen::Vector3f point = payload.view_pos;
Eigen::Vector3f normal = payload.normal;
Eigen::Vector3f result_color = { 0, 0, 0 };
for (auto& light : lights) {
// DONE: For each light source in the code, calculate what the *ambient*, *diffuse*, and *specular*
// components are. Then, accumulate that result on the *result_color* object.
// cwiseProduct()：矩阵点对点相乘
// 衰减半径
float r = (light.position - point).norm();
// 入射光
Eigen::Vector3f l = (light.position - point).normalized();
// 视线
Eigen::Vector3f e = (eye_pos - point).normalized();
// 半程向量
Eigen::Vector3f h = (l + e).normalized();
// 环境光
Eigen::Vector3f la = ka.cwiseProduct(amb_light_intensity);
// 漫反射
Eigen::Vector3f ld = kd.cwiseProduct(light.intensity / pow(r, 2)) * std::max(0.0f, normal.dot(l));
// 高光
Eigen::Vector3f ls = ks.cwiseProduct(light.intensity / pow(r, 2)) * pow(std::max(0.0f, normal.dot(h)), p);
result_color = result_color + la + ld + ls;
}
return result_color * 255.f;
}
4. 纹理
在实现 Blinn-Phong 的基础上，将纹理颜色视为公式中的 kd，将 phong_fragment_shader 的代码拷贝到 texture_fragment_shader, 在此基础上正确实现 Texture Mapping。作业不需要考虑双线性插值和 mipmap ，直接把纹理坐标插值算出来调用提供的接口即可。
Eigen::Vector3f texture_fragment_shader(const fragment_shader_payload& payload) {
Eigen::Vector3f return_color = { 0, 0, 0 };
if (payload.texture) {
// DONE: Get the texture value at the texture coordinates of the current fragment
return_color = payload.texture->getColor(payload.tex_coords.x(), payload.tex_coords.y());
}
Eigen::Vector3f texture_color;
texture_color << return_color.x(), return_color.y(), return_color.z();
Eigen::Vector3f ka = Eigen::Vector3f(0.005, 0.005, 0.005);
Eigen::Vector3f kd = texture_color / 255.f;
Eigen::Vector3f ks = Eigen::Vector3f(0.7937, 0.7937, 0.7937);
auto l1 = light{ {20, 20, 20}, {500, 500, 500} };
auto l2 = light{ {-20, 20, 0}, {500, 500, 500} };
std::vector<light> lights = { l1, l2 };
Eigen::Vector3f amb_light_intensity{ 10, 10, 10 };
Eigen::Vector3f eye_pos{ 0, 0, 10 };
float p = 150;
Eigen::Vector3f color = texture_color;
Eigen::Vector3f point = payload.view_pos;
Eigen::Vector3f normal = payload.normal;
Eigen::Vector3f result_color = { 0, 0, 0 };
for (auto& light : lights) {
// DONE: For each light source in the code, calculate what the *ambient*, *diffuse*, and *specular*
// components are. Then, accumulate that result on the *result_color* object.
// 衰减半径
float r = (light.position - point).norm();
// 入射光
Eigen::Vector3f l = (light.position - point).normalized();
// 视线
Eigen::Vector3f e = (eye_pos - point).normalized();
// 半程向量
Eigen::Vector3f h = (l + e).normalized();
// 环境光
Eigen::Vector3f la = ka.cwiseProduct(amb_light_intensity);
// 漫反射
Eigen::Vector3f ld = kd.cwiseProduct(light.intensity / pow(r, 2)) * std::max(0.0f, normal.dot(l));
// 高光
Eigen::Vector3f ls = ks.cwiseProduct(light.intensity / pow(r, 2)) * pow(std::max(0.0f, normal.dot(h)), p);
result_color = result_color + la + ld + ls;
}
return result_color * 255.f;
}
5. Bump Mapping
按照注释提示实现即可，注意h(u, v) = texture_color(u, v).norm()，w和h是纹理的宽度和高度。
Eigen::Vector3f bump_fragment_shader(const fragment_shader_payload& payload) {
Eigen::Vector3f ka = Eigen::Vector3f(0.005, 0.005, 0.005);
Eigen::Vector3f kd = payload.color;
Eigen::Vector3f ks = Eigen::Vector3f(0.7937, 0.7937, 0.7937);
auto l1 = light{ {20, 20, 20}, {500, 500, 500} };
auto l2 = light{ {-20, 20, 0}, {500, 500, 500} };
std::vector<light> lights = { l1, l2 };
Eigen::Vector3f amb_light_intensity{ 10, 10, 10 };
Eigen::Vector3f eye_pos{ 0, 0, 10 };
float p = 150;
Eigen::Vector3f color = payload.color;
Eigen::Vector3f point = payload.view_pos;
Eigen::Vector3f normal = payload.normal;
float kh = 0.2, kn = 0.1;
// DONE: Implement bump mapping here
// Let n = normal = (x, y, z)
// Vector t = (x*y/sqrt(x*x+z*z),sqrt(x*x+z*z),z*y/sqrt(x*x+z*z))
// Vector b = n cross product t
// Matrix TBN = [t b n]
// dU = kh * kn * (h(u+1/w,v)-h(u,v))
// dV = kh * kn * (h(u,v+1/h)-h(u,v))
// Vector ln = (-dU, -dV, 1)
// Normal n = normalize(TBN * ln)
float x = normal.x();
float y = normal.y();
float z = normal.z();
Eigen::Vector3f t{ x * y / std::sqrt(x * x + z * z), std::sqrt(x * x + z * z), z * y / std::sqrt(x * x + z * z) };
Eigen::Vector3f b = normal.cross(t);
Eigen::Matrix3f TBN;
TBN << t, b, normal;
float u = payload.tex_coords.x();
float v = payload.tex_coords.y();
int w = payload.texture->width;
int h = payload.texture->height;
auto dU = kh * kn * (payload.texture->getColor(u + 1.0 / w, v).norm() - payload.texture->getColor(u, v).norm());
auto dV = kh * kn * (payload.texture->getColor(u, v + 1.0 / h).norm() - payload.texture->getColor(u, v).norm());
Eigen::Vector3f ln{ -dU,-dV,1.0 };
normal = (TBN * ln).normalized();
Eigen::Vector3f result_color = { 0, 0, 0 };
result_color = normal;
return result_color * 255.f;
}
6. Displacement Mapping
按照注释提示实现即可。
Eigen::Vector3f displacement_fragment_shader(const fragment_shader_payload& payload) {
Eigen::Vector3f ka = Eigen::Vector3f(0.005, 0.005, 0.005);
Eigen::Vector3f kd = payload.color;
Eigen::Vector3f ks = Eigen::Vector3f(0.7937, 0.7937, 0.7937);
auto l1 = light{ {20, 20, 20}, {500, 500, 500} };
auto l2 = light{ {-20, 20, 0}, {500, 500, 500} };
std::vector<light> lights = { l1, l2 };
Eigen::Vector3f amb_light_intensity{ 10, 10, 10 };
Eigen::Vector3f eye_pos{ 0, 0, 10 };
float p = 150;
Eigen::Vector3f color = payload.color;
Eigen::Vector3f point = payload.view_pos;
Eigen::Vector3f normal = payload.normal;
float kh = 0.2, kn = 0.1;
// DONE: Implement displacement mapping here
// Let n = normal = (x, y, z)
// Vector t = (x*y/sqrt(x*x+z*z),sqrt(x*x+z*z),z*y/sqrt(x*x+z*z))
// Vector b = n cross product t
// Matrix TBN = [t b n]
// dU = kh * kn * (h(u+1/w,v)-h(u,v))
// dV = kh * kn * (h(u,v+1/h)-h(u,v))
// Vector ln = (-dU, -dV, 1)
// Position p = p + kn * n * h(u,v)
// Normal n = normalize(TBN * ln)
float x = normal.x();
float y = normal.y();
float z = normal.z();
Eigen::Vector3f t{ x * y / std::sqrt(x * x + z * z), std::sqrt(x * x + z * z), z * y / std::sqrt(x * x + z * z) };
Eigen::Vector3f b = normal.cross(t);
Eigen::Matrix3f TBN;
TBN << t, b, normal;
float u = payload.tex_coords.x();
float v = payload.tex_coords.y();
int w = payload.texture->width;
int h = payload.texture->height;
auto dU = kh * kn * (payload.texture->getColor(u + 1.0 / w, v).norm() - payload.texture->getColor(u, v).norm());
auto dV = kh * kn * (payload.texture->getColor(u, v + 1.0 / h).norm() - payload.texture->getColor(u, v).norm());
Eigen::Vector3f ln{ -dU,-dV,1.0 };
point = point + kn * normal * payload.texture->getColor(u, v).norm();
normal = (TBN * ln).normalized();
Eigen::Vector3f result_color = { 0, 0, 0 };
for (auto& light : lights) {
// DONE: For each light source in the code, calculate what the *ambient*, *diffuse*, and *specular*
// components are. Then, accumulate that result on the *result_color* object.
// 衰减半径
float r = (light.position - point).norm();
// 入射光
Eigen::Vector3f l = (light.position - point).normalized();
// 视线
Eigen::Vector3f e = (eye_pos - point).normalized();
// 半程向量
Eigen::Vector3f h = (l + e).normalized();
// 环境光
Eigen::Vector3f la = ka.cwiseProduct(amb_light_intensity);
// 漫反射
Eigen::Vector3f ld = kd.cwiseProduct(light.intensity / pow(r, 2)) * std::max(0.0f, normal.dot(l));
// 高光
Eigen::Vector3f ls = ks.cwiseProduct(light.intensity / pow(r, 2)) * pow(std::max(0.0f, normal.dot(h)), p);
result_color = result_color + la + ld + ls;
}
return result_color * 255.f;
}
7. 尝试更多模型
试了一下工程里的其他模型，替换文件路径即可。
8. 双线性纹理插值
使用双线性插值进行纹理采样, 在 Texture 类中实现一个新方法 Vector3f getColorBilinear(float u, float v) 并通过 fragment shader 调用。需要注意 uv map 和 image 在 v 方向相反。
Eigen::Vector3f getColorBilinear(float u, float v) {
auto u_img = u * (width - 1);
auto v_img = (1 - v) * (height - 1);
// 中心点坐标
int x = std::floor(u_img);
int y = std::floor(v_img);
x = (u_img - x) > 0.5 ? std::ceil(u_img) : std::floor(u_img);
y = (v_img - y) > 0.5 ? std::ceil(v_img) : std::floor(v_img);
// 周围四个点(注意参考系方向)
auto u00 = image_data.at<cv::Vec3b>(y + 0.5, x - 0.5);
auto u01 = image_data.at<cv::Vec3b>(y - 0.5, x - 0.5);
auto u10 = image_data.at<cv::Vec3b>(y + 0.5, x + 0.5);
auto u11 = image_data.at<cv::Vec3b>(y - 0.5, x + 0.5);
float s = u_img - x + 0.5;
float t = y - v_img + 0.5;
auto u0 = u00 + s * (u10 - u00);
auto u1 = u01 + s * (u11 - u01);
auto color = u0 + t * (u1 - u0);
return Eigen::Vector3f(color[0], color[1], color[2]);
}
为了便于检查效果，将奶牛纹理贴图大小压缩至原来的25%。以下是前后对比，可见使用双线性插值后纹理着色更加平滑。
