[ODE] <Survey>: ODE - Present and Direction.
gl
gl at ntlworld.com
Thu May 13 10:08:27 MST 2004
I'm not being defensive, I just disagree - the answers are there, they make
sense, and they're valid. There's scope for probing deeper and nuancing,
but as an overview, it works just fine.
But if you feel that strongly about it, stop criticising and design another.
--
gl
----- Original Message -----
From: "Mike Reinstein" <web_fella at hotmail.com>
To: <gl at ntlworld.com>; <ode at q12.org>
Sent: Wednesday, May 12, 2004 10:53 PM
Subject: Re: [ODE] <Survey>: ODE - Present and Direction.
> gl:
>
> Yes, its true that you have to go through more work to actually READ the
> results of polls. But thats where polls and surveys are most helpfull. By
> adhering to an automated poll survey collection system you lose a lot of
> data. These automated polls/surveys are the hottest thing right now as
> demonstrated by slashdot, along with every portal/ web content management
> based web site. But that doesnt mean that the polls are accurate or
> representative.
>
>
> If you want to take a defensive stance on your poll, thats your choice.
It's
> just a warning to everyone that the results of the poll are only as good
as
> the design of the questions and the format in which you allow people to
> answer them. I worry that major contributors on this project will use the
> survey for considering the near/long term project roadmap; if they doit
will
> be based on unstructured questions be based on poorly accumulated
> statistics.
>
> I'm not speaking as a casual observer of the phenomenon.At work,I'm
> responsible for managing our support system which includes several user
> forums, content management systems, weekly technical articles and
> phone/email based support. Over 1,000 developers frequent the site and our
> company has experienced the down side of relying on poorly planned
> polls/surveys too heavily on a first hand basis.
>
> I implore ALL people who might use the results of these surveys to
consider
> what other questions need to be asked, and spend a little time with
> this...In much the same way adequate design needs to be done to design
> software before coding it,you need to spend adequate time designing polls
> (especially the ones that are factored into important decisions) There are
> entire semester long courses dedicated to poll/survey question creation,
> design and analysis, and there's a reason for it. Its not a joke and its
not
> trivial. This isn't to say it requires a lot of formal training to do it
> adequately, but it does seem to indicate that throwing together an auto
> poller that just adds numbers isnt going to get you very far.
>
>
> kind regards,
>
> Neko
>
>
> >From: "gl" <gl at ntlworld.com>
> >To: <ode at q12.org>
> >Subject: Re: [ODE] <Survey>: ODE - Present and Direction.
> >Date: Wed, 12 May 2004 20:54:38 +0100
> >
> >
> > > Question 1: The middle ranges are too qualitative; While
> "perfect" and
> > > useless are clear indicators of how someone feels about ode, the
> middle
> > > ranges are ambiguous and meaningless.. for example, how do you
> quantify
> >the
> > > difference between "needs some work" and "needs
> plentyof work"?
> > > suggestions for improvement:
> >
> >I disagree - the differentiation is clear to me. Looking at the
> results, it
> >also seems clear to the participants. 'Needs some work' means it's
> mostly
> >living up to expectation, 'needs plenty of work' means it isn't.
> >
> > > Add a short answer section that allows people to comment on why
> and how
> >ode
> > > is good/bad. While this does not allow you to tally results in a
> nice
> > > looking progress bar % meter, it lets people generate
meaningful,
> tangible
> > > feedback that can be quantified. (eg "Needs Some work,
> capped cylinder
> > > supoort is blah blah blah [insert comment here]")
> >
> >Sure - if you're willing to read through all the replies manually, and
> write
> >up a summary : ) - I'm not.
> >
> > > Question 2: This is probably ok, but I might add a section
> allowing
> >people
> > > to specify which specific platform they are using for the last
> option.
> >
> >See above : ). Seriously, text input would be useful is someone were
> >willing to study it all. But then we might aswell pose questions on
the
> >mailing list, and study all the answers. The whole point was to cut
> through
> >the long-winded answers, and often resulting discussions, and just
> quickly
> >find out what the deal is in general. If that then generates
> discussion, at
> >least we have some basics out of the way.
> >
> > > Question 3: The wording here is misleading..."Where does
ODE
> failthe
> >most?"
> > > implies that a single answer response is expected. Limit this to
> allowing
> > > only 1 entry to be chosen.
> >
> >Sure, but then you only get a single failure. I could have done the
> 'top 3
> >failures' thing, but I wanted to keep it short. Again the answers
seem
> >pretty clear to me.
> >
> > > As with question 1, allowing people to provide information
> describing WHY
> > > that particular item fails the most is important because it
allows
> people
> >to
> > > quantify and explain their choice.
> >
> >Above.
> >
> > > Question 4: This question should be eliminated, it is redundant
> w.r.t.
> > > questions 5-7
> >
> >Kinda true, but I wanted an easy way to see if certain things weren't
of
> >interest at all, no matter what the priority.
> >
> > > Question 5: Change the wording to "which future development
> is top
> >priority > for you?"
> > > Questions 6,7: See q5
> >
> >I don't think anybody got confused.
> >
> > > Question 8: Split this up into 2 questions; q8 contains list of
> existing
> > > primitives that are important, q9 includes list of future
> primitives that
> > > are important (be sure to provide a brief explanation of future
> primitives
> > > so more people will know what they are selecting)
> >
> >Why the seperation? The questions is basically saying 'in an ideal
> world,
> >what primtives would you use?' Whether they exist or not isn't the
> point.
> >
> >Re. description, the text limit is pretty harsh - I actually had to
cut
> a
> >few lines down.
> >
> > > Question 9:(will be question 10) this is fine, although I dont
see
> this
> > > providing much insight into anything...
> >
> >It shows that plenty of people would like to see 'beyond rigid-body
> core'
> >features. Again, it's just an overview - there's plenty of scope for
> >getting into detail in future surveys or on the list. Hey, I wasn't
> even
> >sure anybody would bother filling this one out : ).
> >
> > > This may seem like nitpicking, but I assure you it's not.
> Carefully
> > > developing a survey is critical because otherwise you accumulate
> feedback
> >on
> > > an issue, attempt to satisfy the needs derived from the survey
> results,
> >and
> > > that in turn inevitably leads to converging on the wrong
> solution(s).
> > > If the survey system you are using does not allow for these
types
> of
> >survey
> > > collection controls,I would argue that we should go with a
another
> survey
> > > system that does allow information to be collected in this
> fashion. (There
> > > are dozens of free systems to choose from, or we could write
> one...we are
> > > talking about to mysql tables and a handfullof php scripts)
> >
> >Look, if you want to design an exhaustive survey, carefully tuned (and
I
> >agree you can tune this very carefully), then please do. What I
wanted
> to
> >do is make a very quick, easy to use, survey that (often busy) people
> would
> >actually fill out, covering some of the basics - keeping it simple was
> >deliberate.
> >--
> >gl
> >
> >_______________________________________________
> >ODE mailing list
> >ODE at q12.org
> >http://q12.org/mailman/listinfo/ode
>
> _________________________________________________________________
> Stop worrying about overloading your inbox - get MSN Hotmail Extra
Storage!
>
http://join.msn.com/?pgmarket=en-us&page=hotmail/es2&ST=1/go/onm00200362ave/direct/01/
>
>
More information about the ODE
mailing list