To an extend it also depends on where you are going. The GPS alone (without maps & data) is practically worthless. It gives you two numbers (lat/long) which at best you can write into your diary. Ok, maybe you have a map with a lat/long grid, and if you're lucky it's the same format as your GPS spits out, but that's more academic than useful in most situations.

So, first of all, if you're going to places where there is no (or only bad quality) mapping data, then there's no point in a GPS whatsoever. Now it also happens to be that in many undeveloped places the mapping data is sparse, but so are the major road networks. E.g. there aren't many roads traversing Kazahkstanz, there is only one Karakoram Highway, and you'll find it without any form of navigational device anyways, so for me there the GPS is also next to worthless.

The big advantage in my opinion are builtup or densely populated areas IF you have detailed mapping data. Naturally this is the case in Western Europe and North America, other parts of the world vary. You might want to find a sight, a hotel or a mechanic in a place where you can't read the signs and are busy enough with traffic to not even have time to read them. Even in your home country, it can save you a lot of time getting to a friend's place in a large city that you don't know your way around.

This, plus maybe finding the one and only water well/petrol station on your solo Sahara crossing on which your life depends, for me are the only scenarios in which the cost of a GPS is really justified, and it really makes a difference. Sure it may be nice/gadgety/whatnot in other situations, but so is having a foldable chair if you see what I mean.

So ask yourself where you are going, if you can get good mapping data (and factor in the cost/time of getting in) and what the use will be for you. Don't get it because everyone else seems to have one.

One last but not least: a GPS does not under any circumstances replace a good quality paper map. It's more of an added bonus.