Your setup sounds similar to mine.
This is what I have:
1996 2520 MVSC (14 degree deadrise).
Stainless Marine bracket. Full transom.
1997 Yamaha OX66 (fuel injected).
End of model year hull, first year injected motor
Mercury Marine 17p stainless 3-blade prop
24x9 Bennett trim tabs (upgraded from 12x12 OEM).
Prior to raising the motor, max engine RPM's were only 5000, and that was on a 'spring' clean bottom.
Later in the season, the motor would only turn ~4900.
Photo #1 is the factory position of the motor, and yes, it would blow out in a turn if I wasn't careful and throttle down.
Photo #2 is the position of the motor now, and if I had it to do over, I would have raised it to that last hole.
Photo #3 and #4 are of the prop that I am wearing.
Boat will now turn 5100 RPM at WOT even late in the season.
Max speed now is ~33 to 34 knots depending on trim.
This motor should be turning 5500 RPM at WOT, so I'm still a little low.
However, I bought this prop from another person who had added a bit of cup to it. It is not a stock off-the-shelf prop.
Photo #5 is the overall view of the transom.
That swim platform was fabricated by the previous owner, and is not an OEM piece.
Since adding the larger tabs and raising the motor, the boat is much more responsive, quicker on plane, and exhibits no bad qualities.
Hope this helps.