I think your approach is not right, you just don’t check on X and Y axes, but the axes are formed by the perpendiculars of the edges in the polygon. These are all the axes that you must test whether it is a separating axis.
Vector2 tmpNormal = Vector2.REUSABLE_STACK.pop();
for (int i = 0; i < a.vertexCount(); i++)
{
Vector2 e1 = a.getVertex(i);
Vector2 e2 = a.getVertex((i + 1) % a.vertexCount());
Vector2 edge = tmpNormal.set(e2).subtractSelf(e1);
Vector2 normal = edge.perpendicularSelf().normalizeSelf();
if (isSeparatingAxis(a, b, normal, response))
{
Vector2.REUSABLE_STACK.push(tmpNormal);
return false;
}
}
In the same way, you test the polygons with the axes from polygon B too. Only thing is that keep in mind that these vertices doesn’t contain the position of the polygon. The position is kept separate from the vertices, and is tested by the [icode]isSeparatingAxis[/icode] method.
According to the statement of separating axis theorem, “an AXIS is said to be a separating axis, if that axis, when extended in both directions, does not pass through the polygons”. To do this, we project the vertices on to that axis, and measure the projection, not the distance between the centers.
private static Vector2 flattenPoints(List<Vector2> vertices, Vector2 normal, Vector2 projection)
{
float min = Float.MAX_VALUE;
float max = -min;
for (Vector2 vertex : vertices)
{
float dot = vertex.dot(normal);
if (dot < min) min = dot;
if (dot > max) max = dot;
}
return projection.set(min, max);
}
The above method [icode]flattenPoints[/icode] does exactly the same. It takes a list of vertices, a normal (that is, the axis), and another [icode]Vector2[/icode] to store the resulting projection. Now we know on what axes we need to test, and we also know the projection. What we need to find is whether the axis is a separating axis, or whether it intersects.
Sorry if this image looked shit, I drew it in MSPaint. If you look at the projection, the [icode]min[/icode] is the [icode]x[/icode] component, and [icode]max[/icode] is the [icode]y[/icode] component of the projection vector. So we can clearly say if they are separated by checking the line intersection of the projections.
if (rangeA.x > rangeB.y || rangeB.x > rangeA.y)
return true;
This still doesn’t solve the issue of collision detection, it works fine if both polygons are positioned in the origin (top-left corner is the position, not the center). To solve this, we calculate an offset to the [icode]rangeB[/icode] and we add that offset to it.
Vector2 offset = tmpOffset.set(b.getPosition()).subtractSelf(a.getPosition());
float projectedOffset = offset.dot(axis);
rangeB.addSelf(projectedOffset, projectedOffset);
Now the intersection test works fine, we still need to calculate the collision response. Before we do that, I will show my [icode]Response[/icode] class to you, so that you understand what is what. It is actually somewhat large, so here is a link that points to this class in my SilenceEngine repository. Now, the idea is, we calculate the overlap distance, only if the axis is not a separating axis. The collision response calculation is rather long to explain here, so here is the code.
float overlap;
if (rangeA.x < rangeB.x)
{
response.aInB = false;
if (rangeA.y < rangeB.y)
{
overlap = rangeA.y - rangeB.x;
response.bInA = false;
}
else
{
float option1 = rangeA.y - rangeB.x;
float option2 = rangeB.y - rangeA.x;
overlap = option1 < option2 ? option1 : -option2;
}
}
else
{
response.bInA = false;
if (rangeA.y > rangeB.y)
{
overlap = rangeA.y - rangeB.x;
response.aInB = false;
}
else
{
float option1 = rangeA.y - rangeB.x;
float option2 = rangeB.y - rangeA.x;
overlap = option1 < option2 ? option1 : -option2;
}
}
overlap = Math.abs(overlap);
if (overlap < response.overlap)
{
response.overlap = overlap;
response.overlapN.set(axis.normalizeSelf());
if (overlap < 0)
response.overlapN.negateSelf();
}
In this piece of code, [icode]overlapN[/icode] is just the axis on which the overlap happened. To get the MTV, you need to scale the axis with the overlap, like [icode]response.overlapV.set(response.overlapN).scaleSelf(response.overlap);[/icode]. This is the MTV value.
Hope this helps.