Speaking for myself, I know the first time I took the ACT I went in not even sure what the four subjects were. Did well, took a study class at my high school, bought a review book, did practice tests, etc. Took it again two months afterwards, exact same composite score.
I realize the test prep helps some people tremendously, but I think the ACT at least does a decent job of testing what you actually know (or at least how good you are at taking tests) than how well you prepared.
I had a similar experience where I took the SAT three separate times and scored exactly the same on the verbal section each time. The math section improved between tests one and two, but was the same for test three. And they weren't "good" scores at all. 560 on verbal and 590 to 620 on math.
But I also took the SAT 2 subject area tests which I felt were much more accurate assessments of my abilities where I scored 670 in writing and 680 in math without studying at all and only one exposure to the test.
I never really cared about those scores so much, but it definitely makes me question the value of the SAT for much of anything.
I realize the test prep helps some people tremendously, but I think the ACT at least does a decent job of testing what you actually know (or at least how good you are at taking tests) than how well you prepared.