California High Speed Rail Authority “Messaging Survey” – CARRD Analysis

“High-Speed Rail Messaging Survey” commissioned by Ogilvy on behalf of the California High Speed Rail Authority

Background

In May 2010 a telephone survey was conducted with 1,206 Californians about their views of the state’s high-speed rail project. The survey was commissioned for the California High-Speed Rail Authority’s (CHSRA) by their communications and outreach firm, Ogilvy + Mather, at a cost of $60,000. According to CHSRA, the survey was carried out as part of the project’s outreach efforts by two California research firms, Fairbank, Maslin, Maullin, Metz & Associates and Public Opinion Strategies.

The CHSRA’s press release[1] and interviews with the media shared a few statistics from the survey. CARRD received the raw data of the survey through a Public Records Request and has produced this report to fully understand the scope of the survey and the types of questions answered.

Download Executive Summary

Full Analysis

The California High Speed Rail Authority’s survey not only determined support for the project, but also evaluated how various messages succeeded in swaying a person’s opinion on the project (hence the title of the study itself: “High Speed Rail Messaging Survey”).

General format of the survey

The survey started off with gathering some basic information: Are respondents aware of the project? What are their opinions of various transportation organizations? Do they know much about the project and where do they get their information?

They were then asked about their level of support for the project and whether they felt it was a priority. An informational paragraph was read, followed by a re-ask of the support question to determine whether people’s opinions were swayed by the information (message) they received.

The next section and bulk of the survey covered people’s opinions of high-speed rail’s benefits and negative impacts. These were asked in a variety of ways; many of the questions were split-tested where half the respondents were given one set of questions or statement pairs, while the other half were given a different set.

Near the end of the survey people were given a final opportunity to change their opinion of the project, with a third re-read of the support question. And finally, demographic information was collected regarding union membership, party affiliation, education level, racial or ethnic group, age, income and gender.

Targets

Before we get into the results, there are some important things to note to better understand the results. The survey made use of “targets” which were used to tease out which people are persuaded by the information paragraph[2] at the beginning of the survey and again at the end of the survey after hearing the questions and messaging pairs. The HSRA or its PR consultant can then take these different categories of people and see which messages are most effective for converting them.

Targets monitor the response to the support question, (build it now, have some concerns, opposed, don’t know) which was repeated three times (Q6, Q8 and Q18).  Targets were categorized by “Vote Consistency” – “Consistent supporters” replied build it now on all 3 questions; “ever opposed” were those that stayed opposed through out; finally, “swing” indicate a change in support but not from or to the “opposed” response. Targets were further categorized as “Movers” – people who moved into the support position during the course of the survey. “Movers” were based on whether they were swayed by the description of the project – “information movers”, or at the end of the survey – “positive movers”.

Messaging pairs

Participants heard a set of message pairs and were asked to decide which one was most representative of their opinion. Most of these were split-tested: respondents were split into two groups and asked two different sets of questions. This effectively reduced the sample size in half. (See the results section below under Sample Size.)

Targets in combination with the messaging pairs (Q13) were used as a tool to see which messages resonated with people. The survey tracked how people’s opinions changed after hearing a paragraph on the project, and again at the end of the survey after hearing a variety of messaging statements.

As the survey name “California high speed rail messaging survey” suggests, this survey was organized such that the CHSRA and its communications team could understand what messages resonate with people, and who the best messengers might be.

Survey Results

The California High Speed Rail Authority announced the partial survey results in their press release on July 27, 2010.  As mentioned previously, limited information from the survey was made available to the public. The following is a summary of the survey as conducted.

Support:

Support for the project was measured by three choices:

1)    I support the project and would like it to move forward as quickly as possible.

2)    I would like to see the train built, but have some concerns about the timing and cost of the project.

3)    I oppose the project, and would prefer that the train not be built.

Note: People who were generally supportive of the concept of HSR, but opposed to the project in its current form (build it right or not at all) were difficult to track from the choices offered.

• First time asked (Q6):  34% support; 42% with concerns; 13% oppose; 10% don’t know.
• After info was read (Q8): 42% support; 42% with concerns; 15% oppose; 2% don’t know.
• At end of survey (Q18):  45% support; 37% with concerns; 15% oppose; 3% don’t know.

How informed were the survey participants?

• 28% have not seen or heard anything about a plan to build HSR.
• 19% who support building the project right away never heard of it.
• 15% of consistent supporters don’t know about plans to build HSR.
• The Bay Area is most informed; the Los Angeles Area is least informed.
• People who live outside of counties in which the project will pass through are somewhat better informed than those who live within the HST counties. However, those surveyed who live within outnumber those who live further away by nearly 3 to 1.

Priority:

Is Building HSR high, medium or low priority?

• 21% rated the project as high, 37% rated it medium, and 40% surveyed consider HSR a low priority.
• 38% of respondents who supported the project at the end of the survey rated it as a high priority.
• Only 8% who “support with concerns” rated it as a high priority.
• For those who say their preferred method of long distance travel will be HSR when it becomes available, 30% said it was a high priority.
• 90% of those who are opposed said it was a low priority.

Favorability ratings of transportation agencies and organizations:

The choices were CHSRA, Amtrak, Caltrain, and Caltrans.

• Amtrak was the clear favorite, followed by Caltrans.
• CHSRA was least favorable.

Budget, taxes and surplus:

• 38% think HSR will have a negative impact on the state budget, while 46% think it will have a negative impact on the amount of taxes we pay.
• 41% think HSR will increase taxes; 45% believe it will pay for itself. Split B: 48% believe it will pay for itself and also generate an operating surplus.

Note: This question was asked after the “information paragraph” was read which stated that the project would pay for itself.

Cars  & trains:

• People view HSR as a replacement for car travel more than they do for air travel. Building HSR in order to reduce our dependence on foreign oil and reduce reliance on freeways and cars resonated with 69%.

Note: The costs cited in the survey compared travel by HST to air travel and to single occupancy automobile travel.  Fuel efficiencies were based on current car and airplane measures.

Greatest negative effect on Californians:

• Taxes
• State budget

Greatest positive effect: (Major, minor, total positive effect)

• Jobs (49%, 32%, 81%)
• Unemployment (42%, 33%, 75%)
• Traffic congestion (42%, 31%, 73%)
• Economy (37%, 31%, 68%)
• Safety & security (33%, 35%, 68%)

Most convincing arguments for supporting HSR:

• Energy/oil (reduce dependence on foreign oil)
• You’ll use it (because you’ll save time & money – tickets are “much less” than plane)
• Jobs

“Best” reason for supporting:

• Jobs
• Cheaper, faster mode of transportation
• 44% of those opposed said none of the reasons given were the best for building the project; 57% said “none” were the second best.

Least convincing arguments for supporting high speed rail:

• Pride.  “High speed rail once again puts California at the forefront of pioneering a new technology for the entire United States.”
• Taxpayer protections. “Once built, California’s high-speed rail system will not require any funding from taxpayers. Operations, maintenance, and even some construction will be paid for by riders.”

Who to believe?

Survey participants were asked to determine if a person or organization was “believable, somewhat believable, not too believable, or not believable at all”.  The results are listed with decreasing levels of believability:

• HSR engineers 76%
• CA Small businesses 73%
• CA university transportation economists* 71%
• Public health organizations 70%
• AAA  67%
• CHSRA Board members 55%
• CHSRA 55%
• Labor unions 48%

*Note: CA University transportation economists were likely included in the survey given that experts from UC Berkeley and UC Irvine, under orders from legislators, recently peer reviewed the ridership model for the project.

Concerns:

• Those in support of the project are least concerned about its negative impacts; those opposed are most concerned about impacts.

Convincing arguments in favor of the project:

• Economic impacts convince everyone to varying degrees, including those who feel it’s a low priority.
• The most convincing arguments for people in both “low priority” and “opposed” groups were “decrease dependence on foreign oil” and “save time & money”.

Sample size:

This survey used split-messaging on many questions – so the full sample size is NOT representative. For split questions, the sample size was:

• 18 people – San Francisco
• 18 people – San Jose
• 6 people – Anaheim
• 6 people – Bakersfield
• 6 people – Fresno
• 3 people – Palo Alto, Menlo Park, Atherton combined

Note: Results for the full sample have a margin of sampling error of +/- 3.5%, however margins of sampling error for subgroups within the sample is higher.

Odds & ends:

• The explanation of movement from support prior and after hearing info on HSR and again at the end of the survey may be found in the Targets Table 56 on page 212, Q6, Q8 and Q18. (Contact Rita Wespi at rwespi@carrdnet.org for more information.)
• Crosstabs were missing for the following questions: Q9 – are list of impacts positive or negative, and Q19 – believability of various transportation organizations.
• The Top Lines and Cross Tabs raw data are available here.
• A copy of the CHSRA press release and the Research Memo from the consultant are available here.
• CARRD’s calls and emails to the firm that performed the survey were not returned.

Conclusion

The “Messaging Survey” measured which messages are most effective in building people’s support of the project and which messengers are most believable. In their press release, the CHSRA shared limited information about the level of support Californians have for the project. By reviewing the rest of the data, it’s clear there is more to be learned from this survey.

CARRD encourages public engagement on this project in order to build the best project possible. Unfortunately, the limited release of the survey results (paid for by CA tax payers for a public project) could actually exacerbate the Authority’s believability problem. Cherry picking survey results does not engender trust and misses the opportunity to learn what the concerns are and what can be improved.

This report was compiled by:

Rita Wespi
Co-founder – CARRD

Phone: 650-269-1781
Email: rwespi@carrdnet.org

Download a copy of the Executive Summary and the Full Analysis

[2] The information paragraph read to respondents was as follows:

“Now let me tell you a little bit more about this project. California voters passed Prop 1a in 2008, which provided nine billion dollars to plan and begin construction on the high-speed train. The high-speed rail system will consist of 800 miles of Track connecting major population centers from San Francisco and Sacramento to Los Angeles, Anaheim and San Diego, including stops in the central valley cities of Fresno and Bakersfield. Trains will reach speeds of 220 miles per hour in the central valley, cutting travel time from San Francisco to Los Angeles to about 2 hours and 40 minutes – instead of six hours by car. It will be the first high speed rail system in the nation. Once it is fully built, estimates are that it will carry 42 million passengers a year. The first phase of the project is expected to cost $43 billion dollars. Funding for this first phase, which will run from San Francisco to Los Angeles and Anaheim through the Central Valley, is coming from the voter-approved bonds; two billion dollars in funds from the federal government; and other public/private partnerships, local agencies and private investors. By law, the system must operate without a taxpayer subsidy. It is expected to generate a surplus after construction is complete.”