Thirty-five years ago, a Speak-Out by SAM founding publisher David Rowan declared, “The ski industry appears to be in a state of statistical chaos.” In “Our Statistical Disarray,” he cited conflicting projections of the size of the industry and other data presented in four different industry studies, including the Kottke Report and the Nielsen sports participation study. How far have we come since then?

The “disarray” article acknowledged the value of consistent and reliable data and recommended six steps, many based on standardizing research across the various studies. It argued for agreement on what aspects of business and the market to measure and how to measure them, called for standardized definitions, and urged heightened focus on beginners and drop-outs. “What prompts a potential skier to become an active participant, what reactivates inactives, and what causes actives to drop out?” Rowan asked.

The article noted the importance of understanding the participation level of skiers, and especially the need to monitor the junior skier: “A lot of the outside research we have struggled with in recent years measure only adult skiers. … But kids represent skier days, they are a part of capacity—used and potential. Their butts sit on our chairlifts. They must be counted.”

So: How much progress have we made? Is there greater consistency across the available reports? Does the industry have the statistical data it needs to drive growth? To find out, we evaluated three NSAA reports—the National Demographic Study, the Kottke Report, and the Conversion Study—and the SIA Participation Study.

*NSAA defines children as under age 18; SIA, ages 6-17. **Occasionals are those in the NSAA study who skipped at least one season in the past five; and in the SIA study are those who skipped the previous season.

Among the greatest differences between the NSAA Demographic Study and the SIA Participation Study are the reported ethnicities of skiers and riders. The Demographic Study doesn’t separate skiers and riders; SIA does.

These are valuable studies all, but aligned, they are not. Each has its own objectives, unique respondent bases, and methodologies. These produce different, sometimes wildly conflicting, findings. To use them, it is important to understand how the data was collected, when it was collected and from whom, the key limitations, and the reliability of the data. This article highlights all this and suggests ways to improve the studies. Several of the recommendations are similar to Rowan’s from 35 years ago.

NSAA NATIONAL DEMOGRAPHIC STUDY

The National Demographic Study derives from data collected in guest surveys. However, the surveys vary from resort to resort. Plus, they are completed either on-site during guests’ stays, or post departure; site and timing impact the results. The different survey instruments include RRC’s dedicated survey, in which resort staff ask guests onsite about that day’s experience. But the majority of resorts use their own, proprietary surveys. Guest Research’s online survey, conducted post-departure, was the source for half of all reporting resorts. This survey measures the entire visit, not the one day when guests were intercepted on-site and surveyed. All that variability leads to different types of answers, to a different range of questions.

Sample size is another factor. Of the 481 operating resorts in the U.S., a total of 87 (18 percent) provided data for the 2016-17 report. “Data from each participating ski area has been weighted in proportion to its number of downhill snowsports visits,” the Study says. “Additionally, the respective NSAA regions have been weighted in proportion to their share of total national visits.”

The value of the study is limited by two other methodology factors. First, children under 18 are omitted, since children are generally not surveyed without the permission of their parents. While this is understandable, the Study underrepresents children, and much of the report’s demographic information regarding the overall market—e.g., age, generation, and ability level, as well as lessons, skiing vs. snowboarding, age when first tried snowsports, and terrain park usage—is skewed.

For example, the report indicates that only 10.4 percent of total skier/snowboarder visits were from children under 18. This is in sharp contrast to the Kottke Report, which indicates 29.4 percent of the visits were from children. That’s nearly three times as many visits. The SIA Participation Study largely echoes Kottke.

In addition, the report states that 13.8 percent of skiers/snowboarders are first timers or beginners. Would that number be higher or lower if children—and non-lesson-taking beginners—were included? Impossible to say, given the data available. The Kottke and SIA reports do not measure ability level.

Second, the incidence of rentals and lessons is exaggerated, because the overall sample skews toward renters and lesson takers. Why? Because these groups are over-represented in post-departure surveys. The contact information for these surveys often comes from point-of-sale exports, which tend to over represent lesson takers and equipment renters. This effect is somewhat offset by RRC’s survey, which asks only about lessons taken the day of the survey. This likely underreports lessons for those who took a lesson during a multi-day visit, but not on the day they were surveyed.

Can we estimate this combined effect? The National Demographic Study reports that 13.7 percent of skiers and snowboarders took a lesson. Kottke reports 7.5 percent. We believe the Kottke number, with data provided by the ski areas, is more reliable.

The NSAA Model for Growth (above) shows a relatively large core audience, with a combination of newcomers and “revivals” comprising the remaining portion of the audience.

The SIA Participation Study finds a smaller core audience, with a much larger group of occasional visitors who often skip a season, compared to the Model for Growth.

Kottke Report

As the 2016-17 Kottke Report states, it “is designed to provide ongoing tracking of several key barometers of interest and importance to the ski industry.” Participation is quite broad: 231 of the 481 operating ski areas provided data, including 223 whose 46.8 million visits comprise 85.5 percent of total projected visits. Much of the data is based on measured operating statistics, though some is based on estimates provided by resorts. With more and more ski areas tracking visits with electronic systems, these estimates should be increasingly accurate.

The greatest difference between the Demographic Study and Kottke is the source of the data: guests in the Demographic Study, resorts in Kottke. And that leads to different results, of which the percentage of kids on the slopes is most obvious. Kottke is also NSAA’s only source for snowsports visits, one of the most-cited of all industry statistics.

A second key industry statistic is the number of “active domestic participants.” This is also one of the most disputed, as the figures cited by NSAA and SIA are widely divergent—as they were 35 years ago.

Surprisingly, this number is not included in the Kottke Report or the Demographic Study, although both contribute to NSAA’s calculation. RRC derives the yearly estimate by dividing the total visits by the average visits per active participant. For 2016-17, dividing the 54.8 million visits by the approximately six visits per participant yields 9.2 million active domestic participants.

This number is significantly lower than that reported in the SIA study. That’s in part because RRC unweights the reported number of days per season, per person, and SIA doesn’t. In the Demographic Study, individuals report they expect to ski/ride 15.6 days for the season. Through experience, RRC has learned to unweight these self-reported estimates, leading to the average of approximately six days.

Overall, we consider the Kottke Report the most reliable report of the bunch, due to the broader resort participation—and despite its reliance, in some instances, on estimates rather than hard data regarding participation.

NSAA Conversion Study

The Conversion Study’s research methodology has varied over its short life. During the first three years, it tracked the behavior of only first-time and beginner adults who took lessons. This is an important limitation; children, according to Kottke, make up slightly more than one-half of those who take learn-to-ski and -ride lessons, and were not included. Neither were first timers and beginners who did not take a lesson.

For those adults who took a learn-to, RRC followed up by email, using contact information provided by the participating resorts, to measure guests’ ongoing improvement and interest in continuing to ski/snowboard. The Study defines “conversion” as when lesson takers indicate they have progressed in ability level (up to a low intermediate or higher) and express a high likelihood to continue with the sport. More specifically, it is calculated by multiplying the percentage that reported a self-assessed improvement in their ability level times the percentage indicating they were highly likely to continue skiing and riding.

For the 2017-18 season, RRC made two significant enhancements to the methodology that increase the value of this research. First, data on children and non-lesson takers are included, providing a more complete measure of conversion. According to Dave Belin of RRC, surveys are sent to those “who are tagged in ski area databases as first timers/beginners, either through taking a Level 1 lesson product, or by renting equipment and classifying themselves as Level 1. There are multiple waves to the same people over time to determine longer-term patterns of behavior—sticking with it, visiting new ski areas, purchasing equipment, etc.”

While expanding the respondent base is a step forward, keep in mind that it limits the value of comparisons to the prior three years.

Will the survey changes raise or lower the conversion rate? We may know soon. On one hand, children are thought to convert at a higher percentage than adults, due to the support they receive from their families to continue. On the other hand, non-lesson takers are thought to convert at a lower percentage than lesson takers.

Another limiting factor: resort participation declined this past season, to perhaps 30 resorts, compared to 35 in 2016-17. Through early April the study had about 3,900 participants, compared to more than 8,000 skiers and riders for all of 2016-17.

While the 2017-18 conversion rate was not available at press time, the 2016-17 conversion rate was 19.1 percent, up from 17 percent the prior year. That rate is largely in line with the national conversion percentage, estimated at 17 percent; the slightly higher conversion rate in the 2016-17 study might be due to greater effort by the participating resorts than the average resort, rather than an actual increase in the national conversion rate.

If the Conversion Study spreads to a much larger number of areas, it could become a highly useful tool.

Percent of skier/snowboarder visits by state of residence: 2016/17 “enhanced Data”

Alpine Skiing by Region

Where do skiers and riders live? While there are several points of similarity between the NSAA Demographic Study (left) and the SIA Participation Study (center and right), differences are significant. NSAA counts a greater percentage of participants in Colorado than does SIA (though the SIA map shows skiers only, the snowboard map is similar), and a lesser percentage in Texas and Florida. Different methodologies lead to different results.

SIA Participation Study

The Snowsports Industries America (SIA) Participation Study for 2017 takes the broadest view of the market. It derives from “data produced by the Physical Activity Council (PAC), a partnership of eight of the major trade associations in U.S. sports, fitness, and leisure industries.” It aims “to establish levels of activity and identify key trends in sports, fitness and recreation participation.”

The 2017 report reflects results from 24,134 online interviews “with a nationwide sample of individuals and households from the US Online Panel of over one million people operated by IPSOS/Synovate. ... The total panel is maintained to be representative of the U.S. population for people ages six and older. Oversampling of ethnic groups took place to boost responses from typically under-responding groups.” The report breaks the panel participants into seven snowsports categories—alpine skiing, freeskiing, snowboarding, tele skiing, XC skiing, backcountry skiing/riding (both lift-accessed and hike-to), and sledding. Sports Marketing Surveys USA performed the analysis.

Despite the large overall sample size, it includes only 1,500 or so downhillers. Skiing and snowboarding represented 4.0 percent and 2.55 percent of the panel participants, respectively. Accounting for participants who do more than one type of downhillling and applying the resulting percentages to the 296,251,344 U.S. citizens ages six and older results in an estimate of roughly 18 million active participants. That’s nearly double the RRC estimate of 9.2 million.

The SIA study also reports much greater participation by non-white guests than the NSAA studies. The SIA report shows that 75 percent of alpine skiers are white, as are 61 percent of snowboarders. Asians and Hispanics constitute roughly 15 percent each of the snowboard population, and combined account for more than 15 percent of alpine skiers. The Demographic Study pegs the white downhilling population at 85 percent. Some, but not all, of the difference between the two studies may stem from oversampling of minorities.

And finally, the SIA study reveals a greater number and percentage of occasional participants, who may or may not participate in a given year. The study tallies 5.5 million lapsed skiers and 4.9 million lapsed snowboarders who “decided to take the year off.” That means 62 percent of skiers and riders report that they did not ski or ride in the previous year.

That’s quite different from the Demographic Study, in which just 31 percent say they dropped out for at least one year in the past five.

Are these sometime participants overreported in the SIA survey, or underreported in the Demographic Survey? Or is either one right on the money? The answers are important, because it takes different marketing tools and messages to connect with the different audiences.

The different methodologies in the two studies may explain, in part, the disparity between them on this point. The Demographic Study asks, “In the past 5 years, has there been a period of one or more seasons when you dropped out of skiing or snowboarding?” In contrast, the SIA study tracks whether those who participated during 2015-16 also participated in 2016-17.

Differences between the studies are also due to timing and the wording of questions. For example, the SIA study asks how many times the respondent has skied and/or snowboarded in the past year, while the National Demographic Study asks respondents how many ski/snowboard trips they will (i.e., plan to) take “this winter.” In the Demographic Study, that figure is 15.6 days. The SIA study reports 7.9 days for alpine skiers and 7.5 for snowboarders.

While there’s reason to quibble with SIA’s overall numbers, there are several advantages of its methodology. Most importantly, the study includes children 6-17 (whose information is provided by parents or guardians). The report indicates 24 percent of alpine skiers and 34 percent of snowboarders were ages 6-17. Combining the two would yield a percentage close to the Kottke’s estimate of 29.4 percent of under-18 downhillers.

Offsetting the SIA study’s methodological advantages, however, is its relatively small sample size, which could make it less precise.

Recommendations

To enhance the reliability and value of these studies, researchers could take several steps:

1. Increase resort participation in the Conversion Study. Why have so few resorts joined, and why have others dropped out? Can the current methodology be improved? A closer look at these questions and continuing refinements of the methodology might then yield the desired results.

2. Design and coordinate any research that relies on guest input with interested resorts, resort companies, and their research partners. This will serve three purposes: enhance methodologies and improve research design; encourage resorts to adopt these changes and follow a more uniform methodology; and encourage those resorts that contribute to the design process to participate in the study.

3. Collect accurate data regarding the ages and ability levels of all participants. This applies to all current surveys. The RRC Demographic survey, for example, asks, “Who are you here with today?” but does not reveal details (e.g., friends, children 12 or younger, children 13-17, etc.). This information will narrow differences with Kottke and provide a better estimate of the population of skiers and snowboarders by ability level.

4. Where possible, standardize the terminology, definitions, and questions across all studies.

5. While conversion is important, tackle all of the industry’s growth opportunities, such as attracting new users, increasing visit frequencies, and reengaging lapsed skiers and snowboarders. These goals require a deeper understanding of the guest experience than we currently have. Addressing these groups can help attract new users, increase conversion, increase retention, and reengage lapsed skiers and snowboarders.

6. Adopt post-departure surveys as the standard means of measuring the guest experience. These surveys provide the most accurate NPS and satisfaction scores.