

> I had some nice and long story explaining everything I wanted to see.
> But I accidentally deleted it. So now I make it short.
>
> First of all. Your questionnaire is not that bad as long as it is not
> seen as representing anything meaningful. It can be a starting point
> for another questionnaire which goes more deeper into the matter. In
> the beginning you do not have to be very extensive. However some
> things can already be predicted beforehand and additional questions
> or adjustments may already be very helpful.
>
> - Proper research question and presentation of the questionnaire.
> -
>
> You must be aware of the fact that certain quantities are not very
> well suitable to be measured by means of a questionnaire and also
> that certain questions are not suitable to represent a certain
> quantity. You make it yourself very difficult by trying to determine
> the influence of drinks by means of a questionnaire and by asking
> people: 'Just tick off your favorites".
>
> People may have very different ideas about the most influential
> drinks. It would be better to guide people a little bit more. Ask
> more different types of questions. You should also not want to grasp
> influence by a single quantity. I would ask
>
> 1) "Which drinks do you feel are currently having the most influence"
>
> 2) "Which drinks do you feel have had the most influential in the
> past"
> 3) "Which drink or drinks do you think others would see as currently
> the most influential"
> 4) "Which drinks do you think others would see as the most
> influential in the past".
>
> I am not sure about the 'do you feel' part in question one and two.
> They tend to be more representative for 'popularity' or 'perceived
> influence'. Therefore I've added questions three and four. If the
> results of the answers are too different you may see the
> questionnaire as invalid since too many other factors like popularity
> might have played a role. The division between 1 present and 2 past
> is because general influence is difficult to grasp. Questioning
> people about their idea of the most influential drinks is not
> necessarily coinciding with the most influential drinks. If you
> define the question a little bit more specific and less abstract than
> the quantity the result is representing may be shifting a little bit
> more from the quantity 'perceived influence' towards the quantity
> 'true influence'.
>
> Some examples of more specific questions:
>
> 5) Drinks which have influenced me personally the most
> 6) Drinks which have influenced and the people around me the most
> 7) Drinks which have had the most influence on the histroy of
> drinks.
>
> - Control questions -
>
> Example: If you would like to do a research about the influence of
> alcohol consumption and the life expectancy you can not simply
> measure the alcohol consumption and the age of dead and use the
> correlation between these two. People who often drink could, for
> instance,also be smoking more often. In order to rule out the effect
> of the correlation to be due to the smoking one has to measure the
> smoking habits as well.
>
>
> A single question can not be enough to represent a proper quantity.
> From your questionnaire it becomes not clear what the average
> bartender thinks because you don't know whether the people who have
> been filling in your questionnaire are your average bartender.
>
> For your research I would measure age (younger people might be less
> acquainted with some drinks), location (not every drinks is equally
> important from place to place), bartending experience.
>
> Using those variables you might check if they have influence and you
> should compensate if the distribution of your voters among the groups
> is not representing the average bartender.
>
> An interesting extra question could be which drinks people know and
> which not. While famousness of a drink might indicate influence, some
> influential cocktails may have been slightly forgotten. (because of
> this also the difference between question 1 and 2). You might want to
> increase the score of a drink which is less well known as this 'being
> less well know' has influence on it's score but not necessarily
> because of a lower influence. You could present this as a separate
> result or take into account when making conclusions about the result.
> Again, you should not want to grasp influence by means of a single
> question and a single value.
>
> - question form -
>
> The way people can fill in a question form can have influence on the
> result. For instance the number of points on a scale. If this is an
> even number of points people can not fill in a middle option and are
> forced to decide between left or right.
>
> It is a good thing that you have abandoned the one vote only thing
> (and also changed alphabetic order to random order). Drinks which
> have much resemblance with other drinks have a disadvantage. People
> need to make a choice between those drinks while people who are
> voting for a sort of drink of which there are few with a lot
> resemblance do not have to make such choice.
>
> Still, also if you have people using more votes there is still an
> effect. There are for instance many martini type of drinks. When
> people are ticking the options they might think something like 'I
> already have a lot of these type of drinks' and vote less for the
> drinks which resemble each other a lot. (I predict loners like
> Bramble, Bloody Mary and Kamikaze will score high. A drink like
> French 75 will score low because of several other champagne drinks on
> the list)
>
> It is best to have a voting system in which the determination of the
> influence of a particular drink is not influenced by other drinks on
> the list. Such a decoupled system would be when for each drink people
> had to fill in a scale from 'very influential' to 'very not
> influential' (you could also add the option "don't know").
>
> Such system would also introduce graduality in the answers. Now it is
> only yes and no. People are forced to make a strong point. There is
> no measurement of the finer graduation. If you would be searching for
> a top 5 or top 10 this wouldn't be a big problem. But since you are
> looking for a top 25, in which the difference between the drinks in
> the lower regions are fine, you should be having a finer scale in
> order to measure peoples opinion about drinks.
>
> - Statistical tests -
>
> An almost final remark: You can not automatically see your results as
> a rating which is a one to one relation with the top 25 ranking. Your
> questionnaire is subject to chance. Your results could be
> coincidental and do not need to reflect an underlying ranking. I
> would calculate whether the differences are by any reasonable chance
> neglectible. If they are than the difference should not be regarded
> as a true difference. I personally always detest those rankings in
> which the differences aren't of any meaning but the ranking is still
> being made according to those meaningless differences.
>
> Also very important: if you would be using a sort of scoring system
> like I described, it is good to check if the votes represent any
> natural distribution. For a natural distribution, e.g. the Gaussian
> distribution, the average has a clear meaning. However if the
> distribution is not natural (e.g. only low and high scores) than it
> is very difficult to give any clear meaning to the average and it
> should actually not be used. It is often seen in internet polls that
> you have a lot people giving high votes and a lot people giving low
> votes. The average of those votes isn't really an average of some
> value but, instead, it represents more something like the rate
> between people voting 0 and people voting 10. You should doubt the
> validity of your questionnaire in this case.
>
> Final point: Imagine that the influence of a drink can not be grasped
> by a single value but should more extensively be explained from
> different viewpoints with more different possible values playing a
> role. If we create a certain questionnaire whose result is a single
> number (so the creator defines how the extensive body of complexity
> must be reduced into a single number) than we might have more
> influence on the final result than the opinions of the people filling
> in that questionnaire. It happens very very often that you read in a
> magazine or a paper about some questionnaire results while the it is
> practically the structure and the design of the questionnaire which
> influences the results instead of the people filling in the
> questionnaire.
WOW! While I like your explaining of it all I think that it would lose the
majority of bartenders. And while I think that yes, this would probably be
more efficient I am not quie sure if you would have the same amount of people
voting or doing the survey if it seemed that complicated.
Maybe we could pu our heads together in the next few months and do a full on
survey like you are talking and come up with a definitive answer to the
question. At the end of the day, I am happy with the results I am getting and
think that I am getting more grass roots bartenders voting for it and enjoying
the fact that they are.
Once I have the results I will be putting together a spread in Chilled
Magazine, you should pick up a copy in the new year and see how it all pans
out.
Thank you so much for your input, it means alot to see people as passionate as
I am. My email address is mixology[at]chilledmagazine.com, drop me a line some
time and we can talk further about this.
Current thread (46 messages):
|
Home · Drink Recipes · Bookstore · Barstore · Handbook · Web Index · Feedback
Copyright © The Webtender.
About | Disclaimer | Privacy policy