Vulkan right handed coordinate system become Left handed - graphics

Question

Problem:
Vulkan right handed coordinate system became left handed coordinate system after applying projection matrix. How can I make it consistent with Vulkan coordinate system?
Details:
I know that Vulkan is right handed coordinate system where
X+ points toward right
Y+ points toward down
Z+ points toward inside the screen
I've this line in the vertex shader: https://github.com/AndreaCatania/HelloVulkan/blob/master/shaders/shader.vert#L23
gl_Position = scene.cameraProjection * scene.cameraView * meshUBO.model * vec4(vertexPosition, 1.0);
At this point: https://github.com/AndreaCatania/HelloVulkan/blob/master/main.cpp#L62-L68 I'm defining the position of camera at center of scene and the position of box at (4, 4, -10) World space
The result is this:
As you can see in the picture above I'm getting Z- that point inside the screen but it should be positive.
Is it expected and I need to add something more or I did something wrong?
Useful part of code:
Projection calculation: https://github.com/AndreaCatania/HelloVulkan/blob/master/VisualServer.cpp#L88-L98
void Camera::reloadProjection(){
projection = glm::perspectiveRH_ZO(FOV, aspect, near, far);
isProjectionDirty = false;
}
Camera UBO fill: https://github.com/AndreaCatania/HelloVulkan/blob/master/VisualServer.cpp#L403-L414
SceneUniformBufferObject sceneUBO = {};
sceneUBO.cameraView = camera.transform;
sceneUBO.cameraProjection = camera.getProjection();

Answer 1

I do not use or know Vulcan but perspective projection matrix (at lest in OpenGL) is looking in the Z- direction which inverts one axis of your coordinate system. That inverts the winding rule of the coordinate system.
If you want to preserve original winding than just invert Z axis vector in the matrix for more info see:
Understanding 4x4 homogenous transform matrices
So just scale the Z axis by -1 either by some analogy to glScale(1.0,1.0,-1.0); or by direct matrix cells access.

Answer 2

All the OpenGL left coordinate system vs Vulkan right coordinate system happens during the fragment shader in NDC space, it means your view matrix doesn't care.
If you are using glm, everything you do in world space or view space is done via a right handed coordinate system.
GLM, a very popular math library that every beginner uses, uses right-handed coordinate system by default.
Your view matrix must be set accordingly, the only way to get a right handed system with x from left to right and y from bottom to top is if to set your z looking direction looking down at the negative values. If you don't provide a right handed system to your glm::lookat call, glm will convert it with one of your axis getting flipped via a series of glm::cross see glm source code
the proper way:
glm::vec3 eye = glm::vec3(0, 0, 10);
glm::vec3 up = glm::vec3(0, 1, 0);
glm::vec3 center = glm::vec3(0, 0, 0);
// looking in the negative z direction
glm::mat4 viewMat = glm::lookAt(eye, up, center);
Personnaly I store all information for coordinate system conversion in the projection matrix because by default glm doest it for you for the z coordinate
from songho: http://www.songho.ca/opengl/gl_projectionmatrix.html
Note that the eye coordinates are defined in the right-handed coordinate system, but NDC uses the left-handed coordinate system. That is, the camera at the origin is looking along -Z axis in eye space, but it is looking along +Z axis in NDC. Since glFrustum() accepts only positive values of near and far distances, we need to NEGATE them during the construction of GL_PROJECTION matrix.
Because we are looking at the negative z direction glm by default negate the sign.
It turns out that the y coordinate is flipped between vulkan and openGL so everything will get turned upside down. One way to resolve the problem is to negate the y values aswell:
glm::mat4 projection = glm::perspective(glm::radians(verticalFov), screenDimension.x / screenDimension.y, near, far);
// Vulkan NDC space points downward by default everything will get flipped
projection[1][1] \*= -1.0f;
If you follow the above step you must end up with something very similar to old openGL applications and with the up vector of your camera with the same sign than most 3D models.