I have a 2005 2520SL, with an F250, bow rail, windlass, swim ladder/platform, and a few smaller optiions, that I bought slighly used in late 2005. Without trailer and electronics. Since it was a slighly used leftover I got a good deal.
The same boat brand new with an F225, instead of an F250, and without the windlass, was 62k last year in Pont Pleasant NJ, and the dealer would not budge off the price, and the 2520XL (bracket) was about 7k more.
The 2320 has a 21 degree deadrise, and it has a a slighly narrower beam than the 2520SL; I'd expect it to run a tad bit better in chop and into a head sea. The 2520SL and the 2520XL are both 16 degree deadrise and 9.5 ft beam. I would imagine that as they are a tad flatter and wider, that she would be more stable at rest.
That answer is theoretical; you need to sea trial both for your own real world perceptions! Even if you could perceive that the 2320SL was a smoother ride in chop and heading offshore, you might decide that the 2520 is fine, and that the extra room is worth the tradeoff... for you! Both are great boats.
For 2007 if you have the money, you can also have the 2520SL and 2520XL in twin engine confiurations as DV models! I think I would personally find a 2520SLD with twin F150's, or 2520XLD with twin F200's, my perfect boat. I have not trialed either of them, but I would imagine the extra weight of twins on a bracketed 2520XLD might smoothen out the ride, which many will say is naturally a bit bumpy in chop if you go too fast, because of the forward location of the helm/pilothouse. Again seatrial them!