SitePoint Sponsor

User Tag List

Page 1 of 3 123 LastLast
Results 1 to 25 of 57
  1. #1
    SitePoint Guru mattymcg's Avatar
    Join Date
    Oct 2005
    Location
    Melbourne, Australia
    Posts
    574
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Arrow Does User Testing Take The Fun Out Of Design?

    I found the write-up that Google's lead visual designer, Doug Bowman, posted about the reasons he left Google last week an interesting read:

    When a company is filled with engineers, it turns to engineering to solve problems. Reduce each decision to a simple logic problem. Remove all subjectivity and just look at the data. Data in your favor? Ok, launch it. Data shows negative effects? Back to the drawing board. And that data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions.
    What do you guys think? How much weight to you give to data from user testing and usage?
    I design beautiful, usable interfaces. Oh, and I wrote a kids' book.
    Follow me on Twitter.
    Read my blog.
    Buy my book, Charlie Weatherburn and the Flying Machine.

  2. #2
    SitePoint Zealot Bannaz's Avatar
    Join Date
    Apr 2002
    Location
    United Kingdom
    Posts
    146
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Standards do get in the way of an attractive design, but 'good webdesign' is a working/succesful website that allows users of all ages/abilities to visit with ease.
    Bannaz - Flash Banner Design
    █ Leading designers in online advertising.
    www.bannaz.com

  3. #3
    SitePoint Guru mattymcg's Avatar
    Join Date
    Oct 2005
    Location
    Melbourne, Australia
    Posts
    574
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    @bannaz This has nothing to do with standards. What Doug is talking about is trialling two or more differently designed pages, viewing which of the two gets the most traffic (or which performs better during user testing), and using that as the basis for all decisions.
    I design beautiful, usable interfaces. Oh, and I wrote a kids' book.
    Follow me on Twitter.
    Read my blog.
    Buy my book, Charlie Weatherburn and the Flying Machine.

  4. #4
    SitePoint Addict
    Join Date
    Jan 2008
    Posts
    203
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by mattymcg View Post
    @bannaz This has nothing to do with standards. What Doug is talking about is trialling two or more differently designed pages, viewing which of the two gets the most traffic (or which performs better during user testing), and using that as the basis for all decisions.
    urm i agree with google on this

    they care more about us (the users) than some stuck up his own artsy fartsy designer

    they should have fired someone like that long ago

    if he cant work in a team and handle disagreements and compromises, he should go work on his own

    i would not hire people like that

  5. #5
    SitePoint Enthusiast robertss's Avatar
    Join Date
    Mar 2008
    Posts
    32
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It's all about managing risk. In today's economy I think Google is making the right move in relying on data to make the best decision rather than making a random choice that may or may not work out. This makes them more likely to be sucessful and make more money. Since when was making money not fun?

  6. #6
    phpLD Fanatic bronze trophy dvduval's Avatar
    Join Date
    Mar 2002
    Location
    Silicon Valley
    Posts
    3,626
    Mentioned
    3 Post(s)
    Tagged
    0 Thread(s)
    Maybe google doesn't really need a top notch designer right now, especially with regard to things like search results.

  7. #7
    SitePoint Addict
    Join Date
    Jan 2008
    Posts
    203
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by dvduval View Post
    Maybe google doesn't really need a top notch designer right now, especially with regard to things like search results.
    no what they need are less ads its no longer neat and clean the interface

  8. #8
    SitePoint Author silver trophybronze trophy

    Join Date
    Nov 2004
    Location
    Ankh-Morpork
    Posts
    12,158
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by Bannaz View Post
    Standards do get in the way of an attractive design
    Would you care to explain how and why standards 'get in the way' of attractive design? If not, I'll call that statement B.S., pure and simple.

    Quote Originally Posted by mattymcg View Post
    What do you guys think? How much weight to you give to data from user testing and usage?
    If someone wants to stick with a design that their users clearly don't like and/or find unusable, they should look for a new job. Dictator, perhaps? I'm very surprised to hear something like that from Doug Bowman.

    Form should follow function. You first make sure that something works, then you apply a design that suits the function – whether that's a website, a car or a toaster – and makes it appealing for buyers and users.
    Birnam wood is come to Dunsinane

  9. #9
    Brevity is greatly overrated brandaggio's Avatar
    Join Date
    Dec 2005
    Posts
    1,424
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think for people like Doug (and I for better or worse), that design by committee at a certain scale of business can be really soul crushing. It borders on emasculating for a designer to consistently be trumped by an engineer dude/dudette with no style that up until their Google gig, lived purely off of Red Bull and clothes from Kmart. Data schmata !

    (Disclaimer: I am overstating it a bit for effect - testing is important, but classical design constructs are quite old and near immutable. I am not against testing - I am against engineers making purely cosmetic design decisions. They can help provide supporting data if available, but design is not their purview, so they should leave it to people like Doug - and most importantly - be happy to do it! If it involves the server performance or back end architecture, I would similarly expect the designer to defer, gladly, to the engineer. Just like how front end devs don't like it when server side devs think they know how to code the front end (a rare handful can, yes) and all the while are re/introducing age old, deprecated methods and mucking things up.

    I just went through a somewhat similar experience with too many chefs in the kitchen and there becomes a point where the pay and name dropping where you work is no longer enough compensation against not being able to do things the way you envision them.

    People don't ask Joni Mitchell to rewrite her songs and as such Doug does not want to jerk around about border widths he just want his designs to stand on their own - all any artists wants. Yeah we have iterations, but at some point we have to take a snapshot and say this is the targeted, desired design and we meant it as cohesive statement.

    In these tough times people are really taking a hard look at quality of life issues and this to me, is just one more example of that.

    Life is to short (to sell out).

  10. #10
    SitePoint Member
    Join Date
    Mar 2009
    Posts
    12
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I don't think Doug Bowman's complaint really has anything to do with the data. It's about not having last say on how his designs are going to be implemented.

    As Marissa Mayer says in that article he links to, if she didn't go by the data, then it would become about who or what is her favorite. Because even if the data doesn't have the last word, Mayer still has the last word. And that's what irks Bowman. If it wasn't "I can't work for an organization who lives by the sword of the data", it would be "I can't work for an organization who has a Computer Science graduate questioning my artistic vision."

    This is really a common thing for big corporations. I'm sure some big businesses do give complete and final judgment on visual design to their artist, but on the whole I think most managements will choose to keep a tight rein on how their image ultimately looks or doesn't look, whatever metric they choose to measure by - be it hard data or personal favoritism.

    I don't think there's anything wrong with either Bowman or Mayer's position. They are both perfectly valid and valuable points of view. They are simply not compatible with each other.

    If I would criticize one thing about Bowman's article it's that on a couple of comments he makes, you get the impression that when he signed on to Google he secretly had the hope (fantasy?) that he, and he alone, would get to shape and mold the visual image of Google to the world from that point on. Given that Google was already hugely established by the time he came into it, I think that was an entirely unrealistic expectation. (I'm aware that I could be totally wrong in this, it's just the impression I got.)

    To address the original question of the thread title, I fully and unreservedly agree with Mayer that the user experience trumps anything else, and that any data you have on it should be followed, even if it means letting go of our original vision.

    But I can sympathize with the Bowman's point of view as well.

  11. #11
    Brevity is greatly overrated brandaggio's Avatar
    Join Date
    Dec 2005
    Posts
    1,424
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If the data says, "Jump off the Brooklyn bridge" I would still consider it awfully hard before doing it. I would not just follow it blindly as if it were my God.

    People seem to be incredible ignorant of the great (and is some cases, timeless) designs throughout history that were not at all created by data/committee/computer model - even for complex, highly usable UI's.

    In the industrial design world, much of it comes from clay and people's hands first (e.g. some of the coolest west coast car designs). This idea that data should govern every decision frightens me (however, it should always available for consideration) - seems people think they can default to it (seems kinda lazy actually) because it must be right. Guess what? Sometimes the odds makers (data analyzers) get it wrong and the dark horse wins! Design has an organic element to it that is qualitative, not quantitative.

  12. #12
    SitePoint Member
    Join Date
    Mar 2009
    Posts
    12
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by brandaggio View Post
    If the data says, "Jump off the Brooklyn bridge" I would still consider it awfully hard before doing it. I would not just follow it blindly as if it were my God.
    Ah yes of course, but if you read both articles, Mayer makes it clear that the reason they had testing going on to see which one of the 41 shades of blue produced the best result among the user base was because there was originally a disagreement about which shade of blue to use. One team wanted one shade of blue while the other team wanted another shade of blue. She compromised by selecting the shade that was in the middle between the two original shades and by starting a data-collection study to see which of the two and the other 39 shades in between them is actually the best to use in regards to user reaction.

    That's very far from making the data your God. In fact, it's the opposite.

    Edit -> I would actually like an example of how collecting data of your user's preferences, and about the little things that generate the best user reaction, can in any way, shape or form lead you to make a wrong or hurtful decision to your website. I am not aware of any that exist, so I'm sure I would become a better designer if someone could point that out to me.

  13. #13
    Brevity is greatly overrated brandaggio's Avatar
    Join Date
    Dec 2005
    Posts
    1,424
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    ^Right - an exhausting, soul crushing circle jerk about a shade of blue (for crying out loud!) that was one of the catalysts for a talented person to quit.

    Now that is wasteful !

    Further, regardless of what Marissa says in any particular case, there is a culture at Google where design is a stepchild to data and as a designer that could be tiresome - guess it was to a talented, experienced guy like Doug. It is definitely not the "opposite" - read between the lines.

  14. #14
    SitePoint Member
    Join Date
    Mar 2009
    Posts
    12
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by brandaggio View Post
    ^Right - an exhausting, soul crushing circle jerk about a shade of blue.

    Now that is wasteful !

    Further, regardless of what Marissa says in any particular case, there is a culture at Google where design is a stepchild to data and as a designer that could be tiresome - guess it was to a talented, experienced guy like Doug. It is definitely not the "opposite" - read between the lines.
    It is actually the opposite. "read between the lines" doesn't really mean anything to me or to anybody, if you want to say something, well... say it.

    Waiting on the example.

    Edit -> Please don't think I'm being combative or anything. If there is any situation in which following your user's preferences can be harmful to the final product of my website, I want to know about it.

  15. #15
    Brevity is greatly overrated brandaggio's Avatar
    Join Date
    Dec 2005
    Posts
    1,424
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    ^Apple seems to be doing very well being driven primarily by design and the "taste" of their designers.

    You don't have to do it this way, or have faith that you can, but people do.


    Lots of people tell me how well Gmail is designed and so on. Personally I think it works well (I use it) but is not at all sexy - frankly it looks like it wasn't designed at all (did they pay someone to design it? My goodness !).

    Some people like real artistic design in what they use, not something watered down by committee.

    It a difference of method and preference.

  16. #16
    SitePoint Member
    Join Date
    Mar 2009
    Posts
    12
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hm, ok, while I think that Apple's #1 value is creating the best user experience possible, of which having a great design (and they are pretty great ) is only a part of, I do agree that they are two different ways of doing things, both of which can work when the people inside are compatible as a team. These two people just weren't... not the end of the world.

  17. #17
    Brevity is greatly overrated brandaggio's Avatar
    Join Date
    Dec 2005
    Posts
    1,424
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    if you work for Google you sort of have to let go to the Borg - resistance is futile - designers, being an irascible, opinionated lot, don't lend themselves naturally to this environment - it is sort of intrinsically an anathema to them - or at least in this case, to Doug.

    --------

    Jobs says, "I want no buttons on the thing"...

    User testing does not back this up - it later becomes a defacto industry design standard (people didn't know they would like it because nothing like it had existed before and they were untrained and unfamiliar)...

    In design, sometimes you have to go to where people will be, not where they tell you they think they should be. Sometimes that takes intuition and reliance on classical design in lieu/irrespective of data.

    To me, what Google does to design as akin to politicians working solely off of exit poll data, without any real platform of their own - it is soulless and uninspiring.

  18. #18
    SitePoint Wizard
    Join Date
    Dec 2003
    Location
    USA
    Posts
    2,582
    Mentioned
    29 Post(s)
    Tagged
    0 Thread(s)
    You also have to keep in mind the size of your data set as well. If you get 100 people judging a site that will be used by a million, you can't very accurately base much off of what those 100 people. When it comes to things as flexible as web design, if you ask 1000 people what they think, you'll get 1000 responses.

    On that note, you definitely should try to find some trends in that data and make sure you cover that area as well.

  19. #19
    SitePoint Guru mattymcg's Avatar
    Join Date
    Oct 2005
    Location
    Melbourne, Australia
    Posts
    574
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    This is a great discussion folks. Thanks for your valuable comments everyone.

    I like the point Buzz Anderson raised about design establishing an emotional connection with your product and your brand. Usage data might be able to tell you what people prefer when they use the thing, but all the data in the world won't tell you how it makes them feel. That emotional response that truly bold, visionary design accomplishes won't come from data, and I think that's what Doug is getting at.

    I don't know if this is the example you're looking for, Aracuan, but off the top of my head if someone is so impressed by a product that is well designed, they might be in awe and just spend time looking at it. If it was a purely functional design, then perhaps they would launch into using it straight away. The data from those two responses don't necessarily reflect how successful each was. Like I said, this may not be a reasonable example, and I think that brandaggio hit the nail on the head when he said that debating miniscule design decisions can be emasculating for a designer. If the difference between two shades of blue is a tiny %, then was it really worth the exercise?
    I design beautiful, usable interfaces. Oh, and I wrote a kids' book.
    Follow me on Twitter.
    Read my blog.
    Buy my book, Charlie Weatherburn and the Flying Machine.

  20. #20
    SitePoint Member
    Join Date
    Feb 2009
    Posts
    11
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I read that article a few days ago, actually made me feel like I wouldn't mind working at Google myself because I feel I'm more of a programmer by nature(+happen to be an engineering major) and I like what I've read about how they operate.

    On the other hand, I wouldn't give user testing/usage as much weight as Google does on my own projects because I've never worked with as large a user base as Google. In Google's case though, their being such a large company makes me understand why they work the way they do, and you can't really argue with their success.

    Apple is a good example of a large company working thanks to innovative designers rather than user testing alone, and I'm sure the designers over there enjoy life more than true designers at Google.

    Does it take the fun out of design? In Google's case, probably yes. How much weight would I give to user testing? Without a huge number of testers, I'd still give the designer room for creativity and weigh their input heavily if they're experienced and are a talented designer. With a very large number of people, personally I'd give plenty of weight to user testing.
    Need a vBulletin style implemented to the latest version of vB?
    Get it done at http://ashrafabed.com.

  21. #21
    SitePoint Addict Fre420's Avatar
    Join Date
    Nov 2005
    Location
    Leuven, Belgium
    Posts
    211
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think a lot of people here misunderstand the data thing.

    Google makes software, software that achieves goals of the users.

    For example, one interface Doug Bowman made is the Google Calendar interface.
    What Google does, & what we can do to with Google Website Optimizer, is to check which design variations make people achieve their goals faster ... what the success rates are of each design.
    Interface design is not about the fancy arty shizzl, it's about making the user interface as intuitive as possible, to make users achieve their goal, & the company achieve theirs.

    When someone asks you to design a website, he actually wants to make more money. For an e-shop that means that as many people should buy a product on your website (preferable one with a big margin). Every design decision you make, can influence the decision of the visitor. A huge product picture vs a small one. A red buy button vs a blue one etc, a shiny apple-like interface vs standard web 2.0 style, etc ...

    People that argue against data, may have forgotten why they make the website. A website is something that has to be used, not something you just have to look at.

    It has nothing to do with emotions, as you can test 5 different emotional designs, & test which one performs best to achieve the main goal of the page.

    I actually really like data to base decisions on. Now I don't have to argue for hours with clients because they want an ugly design feature. I design my version & design their version. If their design request performs better, they don't have to pay for the extra time. If mine performs better, they pay the spent time + an extra. Since I work this way, these discussions only take minutes instead of hours/days.

    Aracuan. There are no 100% graphic design rules for this. A Blue button might work better on website A, & worse on website B.
    The only rules that I'm aware of that work 99.9% on every website are usability heuristics. General contrast/color rules & design guidelines do also work, but you're never 100% sure. An A/B or multivariate test might give you totally different results as expected.

    If you want to know more about these website optimizations I highly recommend you to read the following books:
    - Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions
    - Always Be Testing: The Complete Guide to Google Website Optimizer
    - Call to Action: Secret Formulas to Improve Online Results
    - Web Design for ROI: Turning Browsers into Buyers & Prospects into Leads

  22. #22
    SitePoint Addict Fre420's Avatar
    Join Date
    Nov 2005
    Location
    Leuven, Belgium
    Posts
    211
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    If I read the article correctly, the thing Dough Bowman hated was not that Google was data-driven, he understands that, because it's used by millions of users. He hated that they tested every minuscule thing, which took the fun out of it for him.

    I can understand his opinion, & I think that leaving Google was the best thing to do for him. If you really like more artistic freedom, you shouldn't work on a strictly user interface design like the ones Google has.
    If he wants more artistic freedom, he's better of designing websites for products & services, like 99% of us do. You can still rely on a/b & multivariate tests, but you have more artistic freedom to choose what design elements you want to test & most of all you can experiment more with emotional elements. (for example 5 different branding images on the homepage).

    I just wished Facebook followed the Google attitude, if they did, they wouldn't have that much angry visitors like they have now since their latest redesign that sucks (IMO). If their platform was 100% open, they'd be out of business in no time.

    Youtube is a great example of the way Google works. Have you noticed how many different versions they tried of the high quality video button (latest version I saw was HQ icon embedded in the video, first was a normal link under the video).
    The Youtube interface constantly changes, there hasn't passed a month that I haven't seen a new feature, placement, design, etc ... some disappear forever, some change, some stay. Sometimes it takes only a few hours, sometimes days, sometimes weeks.
    I really like their step by step improvements, which hardly anyone notices (but after a year you get a totally different interface).

  23. #23
    SitePoint Addict tuxus's Avatar
    Join Date
    Feb 2009
    Posts
    254
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think a fine line exists here, data is great, without it we would be shooting in the dark but over analyzing can be just as bad as a lack of data. Where does it stop?

    Ms. Mayer split the difference by choosing a shade halfway between those of the two camps.

    Her decision was diplomatic, but it also amounted to relying on her gut rather than research. Since then, she said, she has asked her team to test the 41 gradations between the competing blues to see which ones consumers might prefer.
    This seems odd to me, how are they finding the best color if their only testing shades of blue? I would think contrast and text content would be better areas to research as testing 41 shades of one color doesn't seem like it would yield the best ROI, also with a test like this they assume it is a matter of a nonlinear dynamic, I wouldn't be so quick to jump to such a conclusion and instead use the time to invest in features (or UI improvements on existing features) which will almost surely generate interest, media attention and therefore income. A different shade of blue for links would surely not produce such attention nor would their be such a difference between 41 shades that 1 would emerge with a clear margin over all others.

    I guess what it comes down to is that I believe the time and effort put into something like this would be better spent elsewhere.

    A designer should not have to justify every detail as control groups are often conflicting, over the little things the result is similar to rolling dice sometimes. Usability is a whole other topic than testing 41 shades of blue or a 1 or 2 px border difference though.

  24. #24
    SitePoint Guru
    Join Date
    Oct 2006
    Location
    Queensland, Australia
    Posts
    852
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I think it's about finding that perfect balance. Something I've learnt and believe very strongly, is that most people don't know what they want and what they like - in other words, we don't always know what's best for ourselves. It's part of the human condition. Fashion is an example of that. With fashion, what we like changes year by year, and what we think we like, is normally decided by looking at what others like. If you looked at what you wore to a party 10 or 20 years ago, you'd probably wonder how you ever liked the outfit, and vice versa. Most of us are followers/sheep, because following is safe.

    So if you design purely on getting users first impressions, then you're pretty much shaping/choosing your designs based on what people "think" is best for them. If you've made a completely new, innovative interface unlike anything the user has ever seen or used, then naturally they're going to dislike it, even if the interface would in the long term, double their level of efficiency. People are uncomfortable with change, because change is unsafe, and most of our psychological conditions are the result of trying to keep ourselves safe and out of danger.

    At the same time, the designer can also suffer from the same problem. He may not necessarily know what's best for himself or others, even though he may think it. Although designers who specialise in interfaces, are more likely to know what's best for the user.

  25. #25
    Resident curmudgeon bronze trophy gary.turner's Avatar
    Join Date
    Jan 2009
    Location
    Dallas
    Posts
    990
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I find it hard to sympathize with Bowman on this. The very things he complains about are the very things he, as a lead designer, should have been instigating. That color thing, for example; why didn't the graphics department set up the user tests? It speaks poorly of the graphics people if a poor schlub, a lowly, artistically impaired engineer, can suggest a better color for its use, and prove it.

    Most of what graphic designers, especially those with formal training, think of as their purview actually belongs to the information architect. Google is nothing if not a company built on gathering and disseminating information. I wonder how many of those 'engineers' are actually degreed library scientists? In the web world, the graphic designer's purpose is to support the information architect; not the other way 'round.

    The single most elegantly designed page on the web can be found at google.com. From the NYT article, apparently Ms Mayer deserves major credit for the design.

    cheers,

    gary
    Anyone can build a usable website. It takes a graphic
    designer to make it slow, confusing, and painful to use.

    Simple minded html & css demos and tutorials


Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •