Take as your simple closed curve an isosceles triangle with the angle between the legs measuring greater than 90 degrees. It isn't hard to see that any inscribed square would have to have a corner on each of the three sides (since opposite sides of the square are parallel but no two sides of the triangle are parallel). But then since the angle between the legs is greater than 90 degrees, neither of the legs can have two corners of an inscribed square (for if it did, the other leg would contain no corner of the square).

This means any inscribed square must have two corners on the base and one corner on each of the legs. This square must then be symmetric with the altitude of the isosceles triangle, and while there are many rectangles so inscribed in the curve, clearly only one of them is a square.