Well, neither have I. But I have done the next most insulting thing, which is to cast aspersions on the credibility of another man’s research. What started out as a mildly raised eyebrow in last Summer’s Ski Area Management has grown into an industry tempest, and it all has to do with how many skiers there are, what they buy and how often they ski.
Let me bring everyone up to speed on this fast-developing statistical brushfire. In 1976, the highly-regarded, Princeton-based research firm of Opinion Research Corporation (ORC) conducted a massive study of the market for Skiing Magazine. The methodology called for sophisticated sampling procedures and over 30,000 telephone calls (see Summer SAM, “At 14 million, We’re Bigger Than We Thought” by Al Greenbert). I won’t go into the intricate details of the methodology, but suffice it to say that they reflect the accepted “state-of-the-art” techniques for this sort of research.
That study determined that in the 1976-77 ski season there were 9 million adults (over 18) who skied alpine and/or cross country at least once, plus 4 million teens (12-18). This figure surprised everyone, including its Skiing Magazine sponsors, because it was substantially in excess of all previous estimates and also of the findings of the major syndicated research firms of A.C. Nielson, TGI and Simmons. The surprise was heightened because the season that was studied was the infamous year of the western snow drought when skier participation could be expected to be under-stated in this type of research.
The trouble that most of us literal-minded, unskilled dabblers in research had with the figures was that certain projections would not fit our prior market (knowledge) (estimates) (guesses) (information) (prejudices)—choose one, depending where your’re from.
For instance, it appeared that a projection of the alpine skier population to the number of times they said they skied resulted in over 150 million skier visits—or some three times the hitherto accepted industry estimates of the national total. And they weren’t even counting the not-inconsiderable skier-visits of the under twelves.
“Crazy,” some of us said.
“Not so!” came the emphatic response. “Your conclusions stem from inappropriate and incorrect comparisons based on misinterpretations or misuses of the data.” Translated, that meant, “You’re full of horse feathers.”
And they were right. They had asked the skier activity questions in terms of ski trips, not days skied, so an adjustment had to be made for travel time. And then there were skier-visits to Canada and Europe to be subtracted. And besides, the study had not asked teenage skiers how often they skied, so no projection of the adult figure to them was appropriate. Finally, the research was clearly designed to study the skier household and it was a distortion to view the results only as the profile of the individual skier.
Ok, we were comparing apples and oranges; it was back to the numbers game and nothing was proved either way. But neither were many people convinced.
Same thing with the equipment purchase figures. By projecting what the adult skiers said they had purchased and how much they had spent, the totals came through more than double the findings of the National Sporting Goods Association, and that was without any purchases by teenagers, why buy largely adult equipment.
But again, we are told we are not comparing similar things. What about the equipment purchased second-hand? What about equipment purchased outside the country?
Yeah, comes the snarled response, but does that explain that big a difference?
Then the skeptics zeroed in on what they thought was a known fact: the audited circulation of Skiing Magazine at the time of the study showed 349,786 subscribers. But the study showed that 1,210,500 claimed to be subscribers! That’s about 3½ times what it is known to be. How come?
Well, the answer comes, some of that is explained by the weighting within the skier household. If there is a subscriber in a skier household, say the father, and the teenage skier son is the household member who is interviewed, then the son is likely to answer “yes” when asked if he reads Skiing Magazine, and to say he is a subscriber rather than a reader of someone else’s copy. Furthermore, in the weighting procedures used, the responses of one member of a skier household were weighted (multiplied) by the number of adult skiers in that household.
At this point the marketing executive looking for a simple fact to grab hold of is likely to say, “Screw it!” because every simple projection made or conclusion drawn has to be modified by a “Yes, but…”
Then, as if we were not churned up enough, another massive study is undertaken by the Forest Service, using ORC again and a methodology virtually identical to their earlier study for Skiing (see Winter ’79 SAM, “Skiers: Past, Present and Potential” by Stacy Standley and Wilbur LaPage).
Again, the discovery of a large skier population: 12.8 million over 16 years old actively skied in the 1977-78 season. Considering that there is no figure given for the under 16’s, this is a whopping big figure.
The project directors for this study, Stacy Standley, of Sno-engineering, and Wilbur LaPage, a market research specialist for the Forest Service, have defused much of the initial skepticism by genuinely agreeing that a significant upward bias is present in a skier’s recall of how often he skied, so they disown any projections of that sort. LaPage even suspects that there may be people who, when asked out of the blue in a telephone interview whether they ski, will say “Yes” even though they don’t. A harmless Walter Mitty type of thing, one supposes. By questioning the accuracy of a skier’s recall they avoided the catcalls directed at Skiing, which had dismissed the possibility of response error as “neither systematic nor consequential in nature.” However, Standley and LaPage do stick by the skier population figure they found.
Standley and Sno-engineering are also of the opinion that the ski area industry substantially underestimates the total skier-visits by making its projections based on relatively few NSAA respondents to the NSAA Economic Study. There are lots of high-volume areas that don’t report, they say; lots that don’t belong to NSAA, lots that simply don’t know what their skier-visit total is, lots that understate for reasons of tax avoidance.
So where do we stand? We’ve got the benefit of the most sophisticated market research ever done on the ski market presented to us on a silver platter, and we see only the turd in the punchbowl.
The answer, I think, is for all of us to come out from behind our previously-prepared positions, stop stonewalling and sniping (how dull that’ll be!) and try to identify what we really do know from the studies; what we don’t know and cannot accurately get from them; what is legitimately projectable; and what, if projected, results in conjecture rather than supportable fact.
I am happy, then, that NSAA has scheduled a meeting of the NSAA marketing committee at the time of the the convention, expanded for the occasion to include some of the originators of the research in question and some statistical mavens like Ted Farwell, Jim Spring, Bob Enzel and me. The purpose will be to sort out the questions listed above. Perhaps even more importantly, the group will attempt to pin down what we really don’t know but should, and what sort of research might provide the answers. For instance, we have statistics on skiers when each study was made, but we have no way to monitor the growth rate.
Personally, I hope the committee will be able to agree on some specific facts. For instance, I am tired, as publisher of this magazine, of being told how many ski areas there are. It’s my business to know, give or take a very few, and I do. Even more so, it is Bob Enzel’s stock in trade to know, for he is the indefatigable co-author of The White Book of Ski Areas. He knows how many ski areas there are. He makes a point of running down every one and every rumor of one. Bob and I are in complete agreement on the total, and it isn’t the 1,200 and 1,300 figures that are sometimes bandied about.* Let’s agree on a figure!
I also hope the committee will agree to study the unstudied. We must stop speculating on “all of these small rope tow areas that don’t belong to NSAA, whose figures are not included in the totals and which may account for a lot more of the market than we have given them credit for…etc.” This may or may not be the case (frankly, I’m skeptical), but let’s find out.
If this ad hoc committee has any left-over time I hope it will consider establishing a sort of permanent, independent Ski Research Council which would help coordinate all research done in the ski industry. It would be available to preview a research project and make suggestions on how to improve it. The sort of suggestions it might make could involve changing the working of a question to make it conform with other research already completed or under way, which in turn would provide a better basis for comparison.
For instance, the two studies in question gave us a fix on two distinct seasons: 1976-77 and 1977-78. But even though the research methodology–sampling and stratification techniques, etc., were very similar, it is impossible to compare the two. Mind you, the respective sponsors had different objectives, so it is understandable that questions would be asked differently, but I still feel that a couple of benchmark questions could have been asked in identical terms that would have provided invaluable data at no extra cost. And this sort of thing could come out of a non-political, non-commercial, objective Ski Research Council.
I feel we are on the brink of some major new thresholds of knowledge about our industry. For instance, the Forest Service study, in its regional breakouts, may suggest a lot of new marketing strategies. I am less sanguine about the usefulness of the potential skier data they have generated, but certainly the “fix” they give us on the identified skier is invaluable.
But, to summarize, we have got to make our conclusions logical. Right now, the sum of the parts is more than the whole, which might be acceptable if making an aesthetic judgment in a beauty contest, but it will never do for a marketing data base.
*I thought you would ask. To answer I’ll quote Bob Enzel: “As my mother-in-law likes to say, and I translate from her mother-tongue, ‘Put a gun to my head and I still can’t give you an exact number.’ If I had to pick the exact number of ski areas I couldn’t do it either, and I believe I can respond to this better than all but a few people, one of whom is my partner. Now a round number I can respond to more decisively: There are about 600 active ski areas that open every season, plus another 150 rope tow areas that open more-or-less every season, and another 150 that are marginal operations, dependent on natural snowfall, money or both. In The White Book we document 727 areas for public consumption; for ski industry use we may toss in another 160 that are small, private or marginal.”
