Tuesday, July 31, 2007

Oilers survey to season ticketholders reeks bias


























With a week left before the EIG take a vote on whether to sell the team to local multi-billionaire, Daryl Katz, Edmonton Oilers season ticket holders received an email survey, administered by an organization called Banister Research. The timing was coincidental - one could logically assume

- The EIG have made the decision to retain ownership, and will use the survey results as a defense on announcement day

OR

- The EIG is genuinely concerned about public perception and wish to evaluate fan perception before making a collective decision.

Looking at my Psychology 104 notes, under the topic of survey bias, the following commentary is provided:

Factors that can threaten the validity of your results fall into three broad categories:

  1. Survey Construction
  2. Survey Administration
  3. Analysis and Interpretation

Let's examine number 1 (using these excellent class notes):

ITEM DIRECTION

Should a survey item be worded positively? Negatively? Or Neutrally?

Example:
I was satisfied with the customer support I received.
I was not satisfied with the customer support I received.
How satisfied were you with the customer support you received?

Ideally, you should use neutral statements so you aren't leading an individual to respond a certain way. However positively and negatively worded items are frequently used. The most important consideration is balance. If there are negatively worded items then there should be an equal number of positively worded items as well.

___
- Following that criteria, Which of the following indicate a hint of objectivity?

a) It is important that the Edmonton Oilers Hockey Club is locally owned.
NO
Neutral statement corrected: Do you feel that it's important the Edmonton Oilers Hockey Club is locally owned?

b) The Oilers ownership group fosters the Edmonton Oilers Hockey Club as a community team.
NO
Neutral statement corrected: Do you feel the Oilers ownership group fosters the Edmonton Oilers Hockey Club as a community team?

c) I think the Edmonton Oilers should be owned by a group of individuals, rather than a single owner.
NO
Neutral statement corrected (a little tougher with numerical answer surveys. This should have been an open answer question): What type of ownership would you prefer? Group or an individual owner?

d) The Oilers ownership group does a good job of communicating with Oilers fans.
NO
Neutral statement corrected: Do you feel the Oilers ownership group does a good job of communicating with Oilers fans?

e) I think the Oilers ownership group manages the Oilers organization well.
NO
Neutral statement corrected: Do you feel the the Oilers ownership group manages the Oilers organization well?

or possibly

Do you feel that Cal Nichols was suggesting sound advice to Pronger regarding a possible divorce?

f) The Oilers ownership group is working hard to build a competitive team.
NO
Neutral statement corrected: Do you feel the the Oilers ownership is working hard and using all it's resources towards building a competitive team?

g) The Oilers ownership group members are strategic thinkers
NO
Neutral statement corrected: Do you feel the the Oilers ownership group members are strategic thinkers?

____

Now, let's examine number 2:

- Seems the Oilers did bring an independent party to administer the survey.
- The failure is that the Oilers delivered this survey directly to the season ticketholders.
- More people are likely to display their true opinions if directly asked by a neutral/independent party
- The chance to win $100.00 of Oilers merchandise isn't a gesture of actively trying to minimize experiment bias
____

Number 3 of the criteria has yet to be determined, but it makes you wonder why they only emailed season ticket holders, and not every member of the ''Oilers Pipeline.'' Will the survey results be more positively slanted towards the response the Oilers want? Are the Oiler Pipeline members that do not purchase tickets more likely to be a threat to the desired survey results? Will the Oilers analyze and intentionally interpret these results as the voice of the entire Oiler nation? The Fuck happened to democracy. Isn't my opinion on the Oilers ownership group (taking into account of the emotional investment and time) worth just as much as the big corporation season ticket holders whose primary purpose of the ST is a tax write-off rather than the score shown on the scoreboard?

Why only season ticket holders questioned? That's one legitimate question that only the Oilers know.

To be fair:
- Some may say this blog entry reeks bias. Perhaps. I can't possibly objectively evaluate myself on whether I have bias.
- Surveys are difficult to conduct, without leaking bias on at least a few occasions
- The blame should fall entirely fall on Banister Research for conducting a poor survey, if they were under no influence by the Oilers to generate specific results
- The above questions analyzed are not unusual in the form itself. Typically however, they are repeated again using an opposite statement

Examples:
1) The Oilers ownership group does a good job of communicating with Oilers fans.

This would be followed up in the survey later on, with a question like this:

20)
The Oilers ownership group does a poor job of communicating with Oilers fans.


2)
I think the Edmonton Oilers should be owned by a group of individuals, rather than a single owner.

Followed up later in the survey with:

19)
I think the Edmonton Oilers should be owned by a single owner, rather than a ownership group.

Colour me not impressed.

Which leads me to believe the EIG have made the decision to retain ownership, and will use the survey results as a defense on announcement day.

1 comment:

Andrew said...

Good call. First time I've seen the blog... good stuff. I'll be back.