Development Blog

[...] full post on Hacker News If you enjoyed this article, please consider sharing it! Tagged with: Guerrilla [...]

[...] Kath­rin Pas­sig hat heute eine Usability-Test-Empfehlung rum­ge­schickt, näm­lich die [hier], deren zen­tra­ler Bestand­teil betrun­kene Bar­be­su­cher sind, weil die am bes­ten [...]

Well it may seem like $5 test but actually if you count the time you spend for your beer and your time in a pub it may prove to be a bit more expensive But nevertheless can be fun

You can also use some free tools available to test your UI – e.g. http://www.usabilitytest.com

Well it may seem like $5 test but actually if you count the time you spend for your beer and your time in a pub it may prove to be a bit more expensive But nevertheless can be fun

You can also use some free tools available to test your UI – e.g. http://www.usabilitytest.com

I've been doing this for years, only on enterprise apps, using participants of office parties. It really does work wonders for your interface.

genius.

genius.

Brilliant, thanks for sharing. I was going to say, maybe the $5 beer reward should be given up front to increase the subject's intoxication level .. haha

Freakin brilliant. I can't wait to do this.

This would work for some projects, but breaks the number 1 rule of usability studies. Make sure you carefully qualify the people you are testing it with to make sure they are in your target audience/demographic.

I'll have to try this out at some point when I'm testing something rather generic, like a blog design.

This would work for some projects, but breaks the number 1 rule of usability studies. Make sure you carefully qualify the people you are testing it with to make sure they are in your target audience/demographic.

I'll have to try this out at some point when I'm testing something rather generic, like a blog design.

This would work for some projects, but breaks the number 1 rule of usability studies. Make sure you carefully qualify the people you are testing it with to make sure they are in your target audience/demographic.

I'll have to try this out at some point when I'm testing something rather generic, like a blog design.

“Users aren’t so much unintelligent as they are distracted and indifferent”

I've seen a bunch of articles white knighting for “the users” like this recently, but I'm not buying it. They are unintelligent. They are idiots. To actually get confused by that page requires a staggering lack of awareness and ability to think.

In the opening paragraph you say it is condescending and wrong to “think stupid” to make your UI more accessible. How different is it if you “think like a drunk, distracted, and/or indifferent person?” It's the same exact thing, you've just renamed it something less condescending.

Hmm, a user sitting there with glazed over eyes, so distracted and indifferent to the screen he is using that he can't even identify one of 800 clues that this isn't the facebook login page, somehow bypassing all of that to find the comments section and hammer out some AOL-speak…sounds like an idiot to me. Nothing against the author of this post, I just think we should accept the fact that many people are not smart (not entirely their fault, no doubt, but that's how it is).

This is a great approach thanks for sharing it.

James: Of course there are unintelligent users, I'm not denying that. The point I'm trying to make is there are also many smart people who end up doing stupid looking things on your application because they're distracted & indifferent.

The difference between thinking of users as stupid vs thinking of them as distracted & indifferent comes down to one of respect. Too often, I'm talking to developers who have a subtle vein of condescension towards any suggestion I have simplify an interface because their mental image of the users who are affected by this issue is that they're drooling idiots.

Once I hammer it into their heads that instead who they're dealing with is capable people, just like them, who simply have other things going on in their lives, the tone of the conversation changes dramatically.

Thanks for this — I've always been uncomfortable with the artificial environment of lab-based usability evaluations and this is an interesting idea to try out

[...] The $5 Guerrilla User Test – BumbleBeeLabs [...]

[...] The $5 Guerrilla User Test [...]

It's unscientific. It's creative. It's scrappy. I think I like it.

(Though Josh Walsh has a good point about selecting the proper test group)

[...] The $5 Guerrilla User Test « Bumblebee Labs Blog (tags: testing ux psychology) [...]

[...] $5 Guerilla User Test A recent blog post from BumbleBee Labs proposed an intriguing method for user testing – bring your laptop into a bar, find someone a [...]

love it!

[...] quote sums up Bumblebee Labs $5 Guerrilla User Test nicely: Drunk people are a pretty accurate mimic of distracted, indifferent [...]

[...] The $5 Guerilla User Test http://blog.bumblebeelabs.com/the-5-guerrilla-user-test/ [...]

This is such a great point, Hang. Not only does it change the tone of the conversation, but it re-orients the culture of your whole team away from one of 'techie condescension' towards a culture of genuine care for the user. It's so weird how many people spend their lives building software for people who they think are idiots…

[...] The $5 guerrilla user test — Bumblebee Labs (via @joelflom) [...]

[...] The $5 Guerrilla User Test [...]

I like this idea, but can I borrow your laptop as I don't want any of my drunk friends using mine.

[...] User testing in bars 15 03 2010 The $5 Guerrilla User Test « Bumblebee Labs Blog. [...]

I liked your article very useful to me. I will come back and read the article again.
More info on the web site long nightgown

Your website is beautiful and useful. How do I access your site often. Come visit my website at addlink addurl and Linking to websites

Type your comment here.Oh this topic… It's unscientific. It's creative. It's scrappy. I think I like it.

(Although Josh Walsh has a good point about choosing the proper group test) Nice to visit your site. Wish you post another good topic like this. I will wait to read it.

thank you! That looks like a great resource

[...] The $5 Guerrilla User Test -what a crock! – I couldn’t resist commenting « User Experience for Start-ups [...]

There are so many things I disagree with here. but most of all, this seems utterly ridiculous: “You want someone who’s pretty drunk but not completely trashed. ” and “Match the bar you’re going to with the user population you want to target. “.

And how about not pre-judging your participants: “Focus on people who don’t normally get talked to at bars, middle aged people, homely girls, the guy sitting in the corner.”

I especially like: it “gives you insights you never would have gotten hiring a professional usability consulting firm”. ah that's probably because usability professionals interview people who actually fit within the target audience of the product.

I'm all for lean usability principles, but they have to be implemented correctly or else much of your time is wasted. Let's say you interviewed 8 people over 2 hours for $40 bucks. and none of them actually fit the typical user profile for your site. You would be better interviewing 3 people who fit the profile for 20 minutes each and giving them $10… or a movie voucher. whatever seems of value to them.

Then again, I might have completely missed the mark. perhaps this is meant to be a humerous article…

There are so many things I disagree with here. but most of all, this seems utterly ridiculous: “You want someone who’s pretty drunk but not completely trashed. ” and “Match the bar you’re going to with the user population you want to target. “.

And how about not pre-judging your participants: “Focus on people who don’t normally get talked to at bars, middle aged people, homely girls, the guy sitting in the corner.”

I especially like: it “gives you insights you never would have gotten hiring a professional usability consulting firm”. ah that's probably because usability professionals interview people who actually fit within the target audience of the product.

I'm all for lean usability principles, but they have to be implemented correctly or else much of your time is wasted. Let's say you interviewed 8 people over 2 hours for $40 bucks. and none of them actually fit the typical user profile for your site. You would be better interviewing 3 people who fit the profile for 20 minutes each and giving them $10… or a movie voucher. whatever seems of value to them.

Then again, I might have completely missed the mark. perhaps this is meant to be a humerous article…

“The first time you do it, you’re probably going to be suffering from approach anxiety. Start off approaching a group of the same sex as you so that the encounter isn’t sexualized.”

Erm, that may not *always* work.

Thank you so much for this informative response. I hope it is helpful to many others as well.

Hang, if only it was as easy as this…

I think this is a really fun article. This method seems like it would be a lot of fun, but I think that's all it is. It may reveal some usability issues that you had not seen in other diagnostics, but anything uncovered in this method would certainly need to be examined further. I would love to know the context in which the UX Specialist said his “drunk user” quote. Like everything in usability, it depends.
If I were designing an electronic game or music system to be used in a bar, this would be the perfect setting. Couple of beers, loud music, lots of flashing lights, sit them in front of the interaction design, see what happens. Perfect, I have no argument. I'm sure that the user group would be extremely honest too given inhibitions would be lowered.

But, you obviously just can't take any design and apply this method. Cockpits, medical equipment etc. If you just want to “…watch your application crash & burn as people do all sorts of ridiculous ass shit they would never do in a lab but constantly do in real life” then getting drunk people in front of a computer may not be the right way to go. Drunk people are going to show you “all sorts of ridiculous ass shit they would never do in a lab but constantly do when *drunk*”. If you want to test distraction and interruption then there are other much better ways to do that. If I was testing a new car dashboard design, I would give them a simulator (this would be your laptop in your example) and a cellphone. I'd keep calling it when they were driving the simulator. I don't know if I would get them drunk, because sure, they may take the car “off-roading” in the simulator when drunk, but they wouldn't do this in real life – not for the most part anyway.

Motivation is a huge factor when people work on applications or websites. For someone to do anything, they have to be motivated to do it. This is either because they will be fired, or they want to book a flight, or a social network application because they want to communicate etc etc. Yes, they may be indifferent about the design per se, but they aren't going to use something *because* of indifference – there is always a motivation. Within the first few seconds of the person using the site or application, you need to make them understand the foundation, the layout, the concept. The discovery stage is the time of least distraction, high motivation and highest concentration. Depending on the application or motivation, it may only last a few seconds but this is the time that is the most important to test. If your participant is under the influence (Under the Influence [UI] too funny) from the onset, you are going to lose this critical mental model building time. You'll almost certainly get a lot of false negatives.

Next, if your design is already on a laptop, you're probably too late. To me it would look like you already have your concepts and architecture rightly or wrongly figured out, and then you set someone down to evaluate the lower level user-interaction. Usability testing should be about iterative improvement, and not about about a final diagnostic to see if the product works. Testing at that stage may be, and is often too late.

Remember that UX specialists are looking for flaws in the design. They are like quality engineers. Yes, I know this happens in the real world, but the person designing the thing shouldn't be the person testing the thing. As a former developer, I never saw anything wrong with my code, and if there was a problem, I'd be making excuses for it, and somehow deem it acceptable. The same thing happens with design. If someone uncovers a legitimate issue in a bar, what's to stop the designer and tester saying “OK, yeah, but this person was extremely trashed…” and you throw out the data. It all just seems too risky.

Finally, the challenge of design isn't just the design – it's the politics, it's the buy-in from the development team, it's about convincing the management team that a different approach maybe necessary and that they need to invest more $$$ to make this thing work. I'm a UX specialist, and I can't imagine going to a VP with data collected from Paddy O'Shea's on Friday night with a show stopping usability issue and actually being taken seriously.

Yes, this is an extremely cheap method, but it's like everything – in the end you get what you pay for. UX Specialists, for the most part, know what they are doing, trust them.

Hi Lisa,

The cost of formal usability testing is closer to $100 than $10 (http://www.useit.com/alertbox/20030120.html) and it costs even more when you take into account all the time required to organize (1 hour per participant!), record & analyze the results. The certainly have their place and are an invaluable tool but I think they are best used synergistically with other, cheaper tools.

While your target audience can be helpful uncovering more of the domain specific aspects of usability, their time is valuable and you want to spend it uncovering deep issues and not superficial usability concerns.

This is why testing people in your non-target audience is useful. They can still uncover a wide range of issues that should be fixed before you move onto formal usability testing. That way, you're not paying someone $171 for them to come in and say they can't find the button to buy the good because the language was unclear.

I'm not saying this is the be all and end all of user testing but I think it's a valuable tool to have in your arsenal as long as you understand the context of it's use.

Mark, I think everything you've said makes a lot of sense and I generally agree with it. This is not the right tool for highly specialized equipment, companies with a staff of dedicated UX professionals on board or highly conservative management structures (this sadly rules out a lot of graduate students who would never be able to get this past an IRB). I should have made more clear in my article that I was primarily writing this for the type of software companies I had typically been dealing with, small, scrappy startups with tight deadlines building mainstream consumer web apps and comprised mostly of engineers with no full time UX person on board.

As for the laptop thing, I'm actually a huge fan of paper based prototyping but I didn't want to have to explain how to run a paper based user test in this article so I used the example of a laptop to simplify.

Thanks for your comment, it's always great to receive this kind of reasoned & insightful feedback.

Cheers
Hang

[...] [from steve_portigal] The $5 Guerrilla User Test [Bumblebee Labs Blog] – [While we're obviously big advocates of getting input about designs from people as frequently as possible and at various levels of fidelity, it's a bit dissonant when informal methods get distilled (so to speak) into formal-seeming methods without any of the purposefulness and planfulness of established methods. Challenging to my assumptions and thus helpfully provocative] Drunk people are a pretty accurate mimic of distracted, indifferent people. This insight has lead to a wonderful technique I’ve been refining over the years that I call “The $5 Guerrilla User Test”. Here’s the 5 second version: 1. Bring a laptop to a bar, 2. Offer to buy someone a beer in exchange for participating in a user study, 3. Watch your application crash & burn as people do all sorts of ridiculous ass shit they would never do in a lab but constantly do in real life, 4. Go back, apply the lessons you have learnt, repeat until you have an app that is 100% drunk person proof Share this post [...]

That is really a very useful and also focused piece of writing. Though my opinion differs from that of the author, I realize that this is excellent site. I will certainly subscribe to Rss of this fabulous as well as different web log.

Does user research get the respect it deserves in the Valley? Why or why not?…

The timeline thing I think is a bigger deal than it should be. I’ve been a huge proponent of scrappy, unorthodox user testing. Back in February, I wrote about the $5 Guerilla User Test: http://blog.bumblebeelabs.com/the-5-guerrilla-user-test/ that I’…

To what extent is it still valid to assume that your web app users are stupid?…

It is not. You should assume that they are drunk. http://blog.bumblebeelabs.com/the-5-guerrilla-user-test/…

This is surely a especially good as well as aimed piece of content. Although my estimation differs from that of the author, I find that this is wonderful site. I’ll surely sign up for Rss on this great as well as special blog site.

I have to try this sometime.. Sounds like a way to make a tedius task pretty fun mixing beer into the picture

very nice center

Top post. I look forward to reading more. Thank you.

Why would you post publicly on Google+ if you do not want a non-friend to comment on the post? Using Circles correctly would resolve your concern for privacy….

Because that happens to be the default setting, and most users will use whatever the default setting is. I did not use the service “incorrectly.” I knew, prior to making my posts as “Public,” exactly how the features worked because the concepts are…

This is a great approach thanks for sharing it.

This is innovative concept to read.. But nice..  You have mentioned very useful and profitable message for us..
Web Designer Riyadh | Riyadh website design company

I like very much your way of presentation Some great information to be absorbed in this post.Thanks a lot for sharing.
hire web developers | hire a website developer

I enjoyed this post of yours and I must say that every time I come back to your blog, I always have something new and informative to read. Thanks for keeping me updated.
website developers for hire | hire a website developer

ok, let me read it/
http://www.medionakku.de/Medion-MD98150-akku-2150mAh.htm

i truthfully enjoy your own writing kind, very remarkable,
don’t give up as well as keep writing due to the fact that it simply just worth to follow it.
looking forward to see a whole lot more of your current well written articles, enjoy your day
SEO Services Company
Dominican Republic Real Estate
Guaranteed SEO Services

Excellent information , please keep it up , we  are a team of seo consultants and a best seo services company by seo expertMr Altaf Gohar

Everyone’s
talking about this man. Give us something more, something that we can get
behind so we can feel as passionately about it as you do. 

I couldn’t say it anybetter:The Atlantis of the internet!

I couldn’t say it anybetter:The Atlantis of the internet!

Our business is mostly functioning ugg boots sale.We have a Kids Infants Erin UGG Boots range Classic Cardy UGG Boots of ugg classic boots.They have numerous kings of models.For example,ugg ugg boots sale 5825,ugg5803 and ugg 5815.Our ugg boots shop is the most popular internet store on the web.

This is a really good read for me, Must admit that you are one of the best bloggers I ever saw.Thanks for posting this informative post.

Ugg boots outlet
ugg boots cheap
Ugg outlet
cheap ugg boots
Ugg boots sale uk Whatever,
uk Ugg Boots chances are that you end
up with boots that appear very similar ugg looking but bear no same
flexibility,
Uggs On Sale
durability, lightness as well as thermostatic benefits as the genuine UGG.
ugg boots outlet If you still desire
to get an authentic pair,
Ugg
outlet your local department stores and branded shoe shops are the best bet.
If you can’ t find what exactly you want in the usual stores,
uggs outlet online go shopping
online.
ugg online You can
even find excellent sheepskin boots at a fraction of
ugg boots outlet their steep
original sales price.
Tory Burch
Outlet
tory burch
uk
tory burch outlet
store
tory burch sale.

Thanks a lot for share such a great info material on your blog. i like to share this with my friends soon.

I regularly visit your site as you have very useful and interesting resource on your blog.. I appreciate your interest towards sharing the useful stuffs. Great work you guys are doing, keep it up! 

This is a great site. Good new interface and nice informative blogs. I will be coming back soon, thanks for the great blog.