My vote is definitely for the Xterra. Having just traded in my 2006 WRX for a 2011 Xterra Pro-4X, I am obviously pretty biased. But, I did a lot of research and a lot of test driving before purchasing. For me, it was the only SUV that came with a manual transmission and a locking differential right off the lot. I have owned a 2003 Toyota Tacoma in the past and I have to say that the Nissan SUV's/Trucks drive much better. The 4.0 liter engine in the Xterra is extremely powerful and full of torque, especially when mated with the 6 speed manual that my Xterra has. I had also considered getting one of the newer Tacomas, or finding an older 4Runner with a manual transmission. But after reading all of the issues that the Tacomas are having with both the manual and automatic transmissions I had to pass. Not to mention they have seriously watered those trucks down tremendously...seriously, a plastic bed??
If you are looking at Xterras, my recommendation would be 2005+ if at all possible. That was the beginning of the 2nd generation, and came with the 4.0 liter V6 instead of the 3.4 liter.
Here is my truck, that I have now had for exactly 2 weeks. I hit 500 miles on my way home today!