The four major uses of the PSP, in order are: Games, Movies, Photo, MP3. If you want to play near PS2 quality games on-the-go then this is pretty much your only option (unless the Gizzmondo ever hit's NZ).
The reason I say this is because if Photo's and MP3's are an attractive feature of this device you would be better off with something like an IPod Photo 30G. Expanding on this, if you are mainly attracted to Movies, Photo's and MP3 features you should definitely consider the Creative Zen which offer's 7hrs of Video playback and 22Hrs of Music playback and photo support for just under double the cost of PSP (although the Zen only has a 3.8 inch 4:3 display).
Perhaps the main reason you'll be wanting the PSP device for games however is because realistically you would be wanting a minimum of a 2Gb memory stick to get the best from the device, and a 2Gb memory stick costs slightly more than the cost of the PSP itself. Okay, so maybe about $150 dollars more, BUT if you get the 2Gb stick you will be totally setup. With 2Gb you have room for 1 full length movie and enough MP3's to get you through the day.
My rating is heavily slanted against Sony only including a 32mb stick in this package. My rating for the PSP as a piece of technology would be 5 out of 5. IF and WHEN Sony releases writeable UMD's, and in my opinion they will need to, then the PSP will become a must own device. As it stands it's only a must own if you really must play near PS2 games on-the-go.