Commentary - Citizens are 4' 8" tall with red hair (or, Ms. Longrie - all skewed up). [Updated]

At last night's special budget hearing (formerly known as Truth in Taxation) Ms. Longrie presented her "alternative budget". She prefaced her remarks with a description of the survey she has been touting. She then used the results to declare what it is that citizens see as priorities.

What survey is that? During her failed campaign, Ms. Longrie mailed some questions to some residents, and asked those questions of others as she door knocked. Taking her word for it, she mailed or doorknocked 2500 copies, and received 250 responses. She then used her summary of those 250 responses to make her declarations about citizen priorities.

Let's look at how an actual survey works. Having decided on a research objective, defined questions to meet that objective, and tested the questions for non-bias, we put those questions to use. First, we define a population - say, Maplewood citizens. We then devise a mechanism to contact a random sample drawn from that population, a sample sufficiently large to produce enough respondents to meet our research requirements. Then we send out the survey and gather responses, keeping track of who from the sample responded and who did not. After allowing time to respond, we contact enough non-respondents to gather information to determine whether the non-respondents are different from the respondents, along any variables we care about (for example: did more men respond than women?).

If we determine that our respondents are representative of the sample, and the sample was randomly drawn, we can then discuss the results as if they represented the population, within some range of probabilities (a confidence interval) that the sample was actually representative of the population. (Purely by chance you might draw two red marbles from a bag containing an equal number of blue and red ones - a random but not representative sample. A confidence interval defines how likely it is that your sample is not one of these unrepresentative ones).

It is only through the randomness of the respondents that we gain any confidence at all that the responses we get are somehow representative of the population we wish to describe.

Now, ask yourself these questions:

• If, during the campaign, you received a set of questions from candidate Longrie and another set from candidate Rossbach, what would you have done with either set?
• Is a candidate likely to contact known supporters in general agreement of his or her policies, or those who support the opponent?

Armed with your answers, do you think the responses Ms. Longrie touts as representative of citizen priorities are remotely representative of the citizens of Maplewood? Or, are they just representative of those fans willing to tell Ms. Longrie what they thought.

By her logic and use of language, if Ms. Longrie met a 4'8" redheaded citizen, she would claim all citizens are so endowed.


Stephan

[See this earlier post for a related argument, with graphics].

Update:
Lest you think some information is always better than none, consider how well off you are if all you know about grizzly bears is from one encounter with a comatose bear. Unbeknownst to you it got that way by gorging on the last backpacker that happened by.