The Webtender
Forum and Chat

 Message 36568 of 39187 in General Discussion
 Share on Facebook |  Save to del.icio.us  
Subject: Re: The 25 Most Influential Cocktails in History
From: apetail
Posted: Mon Oct 20. 2008, 21:48 UTC
Followup to: "Re: The 25 Most Influential Cocktails in History"  by LiquidRevolutionist  (Mon Oct 20. 2008, 01:52 UTC)
I had some nice and long story explaining everything I wanted to see. But I 
accidentally deleted it. So now I make it short.

First of all. Your questionnaire is not that bad as long as it is not seen as 
representing anything meaningful. It can be a starting point for another 
questionnaire which goes more deeper into the matter. In the beginning you do 
not have to be very extensive. However some things can already be predicted 
beforehand and additional questions or adjustments may already be very helpful.

- Proper research question and presentation of the questionnaire. -

You must be aware of the fact that certain quantities are not very well 
suitable to be measured by means of a questionnaire and also that certain 
questions are not suitable to represent a certain quantity. You make it 
yourself very difficult by trying to determine the influence of drinks by 
means of a questionnaire and by asking people: 'Just tick off your favorites".

People may have very different ideas about the most influential drinks. It 
would be better to guide people a little bit more. Ask more different types of 
questions. You should also not want to grasp influence by a single quantity. I 
would ask

1) "Which drinks do you feel are currently having the most influence" 
2) "Which drinks do you feel have had the most influential in the past" 
3) "Which drink or drinks do you think others would see as currently the most 
influential" 
4) "Which drinks do you think others would see as the most influential in the 
past".

I am not sure about the 'do you feel' part in question one and two. They tend 
to be more representative for 'popularity' or 'perceived influence'. Therefore 
I've added questions three and four. If the results of the answers are too 
different you may see the questionnaire as invalid since too many other 
factors like popularity might have played a role. The division between 1 
present and 2 past is because general influence is difficult to grasp. 
Questioning people about their idea of the most influential drinks is not 
necessarily coinciding with the most influential drinks. If you define the 
question a little bit more specific and less abstract than the quantity the 
result is representing may be shifting a little bit more from the quantity 
'perceived influence' towards the quantity 'true influence'.

Some examples of more specific questions:

5) Drinks which have influenced me personally the most
6) Drinks which have influenced and the people around me the most
7) Drinks which have had the most influence on the histroy of drinks.

- Control questions -

Example: If you would like to do a research about the influence of alcohol 
consumption and the life expectancy you can not simply measure the alcohol 
consumption and the age of dead and use the correlation between these two. 
People who often drink could, for instance,also be smoking more often. In 
order to rule out the effect of the correlation to be due to the smoking one 
has to measure the smoking habits as well.


A single question can not be enough to represent a proper quantity. From your 
questionnaire it becomes not clear what the average bartender thinks because 
you don't know whether the people who have been filling in your questionnaire 
are your average bartender.

For your research I would measure age (younger people might be less acquainted 
with some drinks), location (not every drinks is equally important from place 
to place), bartending experience.

Using those variables you might check if they have influence and you should 
compensate if the distribution of your voters among the groups is not 
representing the average bartender.

An interesting extra question could be which drinks people know and which not. 
While famousness of a drink might indicate influence, some influential 
cocktails may have been slightly forgotten. (because of this also the 
difference between question 1 and 2). You might want to increase the score of 
a drink which is less well known as this 'being less well know' has influence 
on it's score but not necessarily because of a lower influence. You could 
present this as a separate result or take into account when making conclusions 
about the result. Again, you should not want to grasp influence by means of a 
single question and a single value.

- question form -

The way people can fill in a question form can have influence on the result. 
For instance the number of points on a scale. If this is an even number of 
points people can not fill in a middle option and are forced to decide between 
left or right.

It is a good thing that you have abandoned the one vote only thing (and also 
changed alphabetic order to random order). Drinks which have much resemblance 
with other drinks have a disadvantage. People need to make a choice between 
those drinks while people who are voting for a sort of drink of which there 
are few with a lot resemblance do not have to make such choice. 

Still, also if you have people using more votes there is still an effect. 
There are for instance many martini type of drinks. When people are ticking 
the options they might think something like 'I already have a lot of these 
type of drinks' and vote less for the drinks which resemble each other a lot. 
(I predict loners like Bramble, Bloody Mary and Kamikaze will score high. A 
drink like French 75 will score low because of several other champagne drinks 
on the list)

It is best to have a voting system in which the determination of the influence 
of a particular drink is not influenced by other drinks on the list. Such a 
decoupled system would be when for each drink people had to fill in a scale 
from 'very influential' to 'very not influential' (you could also add the 
option "don't know").

Such system would also introduce graduality in the answers. Now it is only yes 
and no. People are forced to make a strong point. There is no measurement of 
the finer graduation. If you would be searching for a top 5 or top 10 this 
wouldn't be a big problem. But since you are looking for a top 25, in which 
the difference between the drinks in the lower regions are fine, you should be 
having a finer scale in order to measure peoples opinion about drinks.

- Statistical tests -

An almost final remark: You can not automatically see your results as a rating 
which is a one to one relation with the top 25 ranking. Your questionnaire is 
subject to chance. Your results could be coincidental and do not need to 
reflect an underlying ranking. I would calculate whether the differences are 
by any reasonable chance neglectible. If they are than the difference should 
not be regarded as a true difference. I personally always detest those 
rankings in which the differences aren't of any meaning but the ranking is 
still being made according to those meaningless differences.

Also very important: if you would be using a sort of scoring system like I 
described, it is good to check if the votes represent any natural 
distribution. For a natural distribution, e.g. the Gaussian distribution, the 
average has a clear meaning. However if the distribution is not natural (e.g. 
only low and high scores) than it is very difficult to give any clear meaning 
to the average and it should actually not be used. It is often seen in 
internet polls that you have a lot people giving high votes and a lot people 
giving low votes. The average of those votes isn't really an average of some 
value but, instead, it represents more something like the rate between people 
voting 0 and people voting 10. You should doubt the validity of your 
questionnaire in this case.

Final point: Imagine that the influence of a drink can not be grasped by a 
single value but should more extensively be explained from different 
viewpoints with more different possible values playing a role. If we create a 
certain questionnaire whose result is a single number (so the creator defines 
how the extensive body of complexity must be reduced into a single number) 
than we might have more influence on the final result than the opinions of the 
people filling in that questionnaire. It happens very very often that you read 
in a magazine or a paper about some questionnaire results while the it is 
practically the structure and the design of the questionnaire which influences 
the results instead of the people filling in the questionnaire.


 Current thread (46 messages):
 Message options:
 

Get everything you need for your bar from
The Webtender's BarStore.

Home · Drink Recipes · Bookstore · Barstore · Handbook · Web Index · Feedback

Copyright © The Webtender.
About | Disclaimer | Privacy policy