Why is determining the exact status of an observed star difficult based purely on apparent size?

Because accurate distance measurement is required first.

A primary challenge in observational astronomy when classifying stellar giants is distinguishing accurately between a red giant and a red supergiant based solely on visual observation from Earth. While red supergiants are intrinsically much larger and more luminous, how bright a star appears (its apparent brightness) depends heavily on its distance from the observer. If a moderately sized red giant is relatively close, it might appear as bright as a much larger, but extremely distant, red supergiant. Conversely, a very large but distant supergiant might be mistaken for a closer, smaller giant. Therefore, without knowing the precise distance to the star, astronomers cannot accurately gauge its true physical radius or total luminosity, necessitating detailed stellar modeling incorporating luminosity and temperature data to correctly place the object on diagrams like the Hertzsprung-Russell diagram.