But your point does stand for websites. Politely asking people to review your site on HN will allow for some great feedback, but you really only get one shot at it.
Anyone know if a service exists (-MTurk) where you can pay people to constantly review your site?
I think that'd be an awesome way to evolve a website. Just apply constant feedback by paying people to launching fast and iterating.
Concise answer: MTurk was an obvious, yet too simple solution.
Long answer: My imagination was something designed specifically for pushing websites to some degree of success. So without me having to think of all of the different dimensions that bring a site to success (clearly state what it is, beautiful design, something people want, easy to use) -- the site I have in mind would walk you through that process taking your money at iterations of each step. In the end, you should have a site that meets the standards of sites which are already winners with most if not all of the dimensions in question.
There's FiveSecondTest (http://www.fivesecondtest.com/) that shows an image of your website for five seconds and asks users for their feedback. Not enough time for users to read all the text, but it's good at seeing if your general point is getting across. It's free too.
that's what the non-tech guy on our team is for (among others things). constant user studies.
you have to be hitting the right crowd. asking advice from HN or random people only works if your site is meant to have universal appeal. HN in particular is a heavily skewed audience.
Once he's been on the team for a short time though, and if he's used the website a lot, then he's not going to provide the newcomers or non-technologists perspective. This is a situation in which increased knowledge of the user decreases their worth to the process.
It's very hard to pretend to yourself that you haven't seen something before and imagine your reaction - was that logo clear in its affordance?, etc..
