Pollfail: Bizarre WaPo poll UPDATED w/reponse

Your tax dollars at work.

speaking in Taipei on #tiananmen anniversary Shanghai venture capitalist and Party ideologue Eric Li tells me: ‘Taiwan independence will be finished within 8 years, by force if necessary. Young people here must pay attention to earning a living, not politics.’ so much for june 4. -- John Keane on Twitter

UPDATE: My latest at ACT is also inspired by this.

WaPo's Monkey Cage published a very strange poll that avoided any serious questions, comes to upside down conclusions, and couches everything in loaded ideological terms. Pretty much what you'd expect from an establishment survey discussed in WaPo, in other words.

From the opening it is a conventional and deeply confused piece of thinking...

One motive may be Beijing’s concern that it is losing the battle for the “hearts and minds” of the Taiwanese people: If more and more people in Taiwan self-identify as Taiwanese instead of Chinese, this could lead to greater support in Taiwan for a declaration of formal independence. This is an outcome the PRC is determined to prevent.

One problem that people writing in the US still don't get is that (a) Beijing is well aware it has lost the battle for hearts and minds and did years ago and (b) the "hearts and minds" propaganda isn't aimed at Taiwan but at its own people, to convince them that it has done everything it can and they must now get their sons killed to annex Taiwan for the CCP and its wealthy cronies. Beijing knows perfectly well that Taiwan isn't going to declare independence. Tiresomely conventional view...

After noting that familiar NCCU surveys finding rising Taiwanese identity, the authors note that....

The conclusions about Taiwanese identity, however, depend on how identity is measured. To test this, we carried out a unique survey in Taiwan earlier this year. Our results suggest Beijing’s concerns about Taiwanese identity trends may be excessive.

Their conclusions are completely at odds with their findings -- in fact they have made a classic error in interpretation, likely deliberately (below). The old, familiar NCCU tracking poll cuts people up into tiny groups of "unification now" to "independence now" because its blindingly clear political purpose is to obscure the simple fact that most everyone in Taiwan supports independence. Instead, we are told they support the status quo -- which is independence. That is why surveys that force people into an independence or not choice find strong support for it.

More on that below...

The survey used random digital dialing to reach landlines, and randomized respondents into two groups. One group heard the conventional fixed choice between three categories (“Taiwanese,” “Chinese” or “Both”). The other group was asked to allocate 10 points across two identity categories.

.....

The figure below shows the results. The fixed-choice method’s findings were similar to other surveys — a majority of respondents (64 percent) chose “Taiwanese only.” In contrast, the allocation method found that “Taiwanese only” are a minority (46 percent), while a majority of respondents (52 percent) believe they are “both” Taiwanese and Chinese.

The survey asked "how Taiwanese do you feel?" It didn't ask what people meant by Chinese or Taiwanese -- that key question was scrupulously avoided. It replaced one set of fixed choices with another set, recapitulating the problem it claims it was intended to solve.

Everyone in Taiwan knows people like my wife, deep green Taiwanese who will show up at the invasion beach kitchen knife in hand and eviscerate the incoming PRC troops out their ass while cursing them in colorful Taiwanese (the PRC is lucky the ROC generals are giving them a fighting chance by using an army composed of men. If the Taiwan army were composed of Taiwanese women, no PRC troops would reach the beach alive).

Nevertheless, such people have no trouble referring to themselves as "Chinese" if you ask them why they are making a sacrifice to the kitchen god or giving red envelopes at new years. They don't see a problem with those identities, because they don't conflict.

But never mind that, look at who they surveyed: people with landlines. So they got mostly people over 40,no? And that group, as recent work by Taiwanese academics has shown, has experienced a slight bump in their identification as "Chinese" (would have been interesting to see the geographical and age breakdown of respondents).

Moving on...

This matters because in our survey the “Both” category has systematically different preferences and attitudes compared with the “Taiwanese only” group. For example, only 36 percent of the “Both” respondents prioritized defending Taiwan’s sovereignty over developing cross-strait economic relations, compared with 65 percent of the “Taiwanese only” respondents.

Compared with the “Taiwanese only” group, those who identified as “Both” also had significantly more positive views of the people of China — and more moderate views of cross-strait relations — than those who identify as Taiwanese-only.

Note the ideologically loaded description... you're a "moderate" if you like China a bit more (I guess people who want to live in an independent democracy are immoderate).

Note also the little trick they played. First they tell us that identity changes when you measure it and the fixed choices are misleading. No problem -- if you give more mixed options, you get more mixed identities, a well known fact. Then they provide two fixed choices, prioritize Taiwanese sovereignty or develop cross strait economic relations -- after explaining that fixed choices can result in misleading findings (!).

This really tells us nothing because these two choices are not at odds with each other in the real world (the same problem with the Taiwanese/Chinese identity frame they use) -- and note: the "choices" they give us are deeply ideological: they are a conventional pro-KMT framing of the debate over how to relate to China.

GIGO.

It's like the real world doesn't even exist. You remember that world, right? Trade and exports to China exploded during the Chen Administration which was way more concerned about Taiwan's sovereignty than the succeeding Ma Administration, whose policies screwed Taiwan's cross-strait trade position (deliberately, most likely). More: we have massive trade with China right now, busy cross strait flights, and millions of tourists each year criss-crossing the Strait, and we have a president really interested in guarding Taiwan's sovereignty.

There's no conflict between these two fixed choices that the pollsters gave this group of respondents. What they really did was force people to make a false choice so they could come up with misleading results which appear to suit some ideological goal of their own. *shrug*

Which brings me back to their findings:

In contrast, the allocation method found that “Taiwanese only” are a minority (46 percent), while a majority of respondents (52 percent) believe they are “both” Taiwanese and Chinese.

Yes, that's right: only in WaPo could you find that 98% of respondents see themselves as Taiwanese to some extent, and then conclude that Beijing should relax.

It's exactly the same as NCCU's tracking poll: they cut the population up into little pieces to obscure the fact that identification with Taiwaneseness is very high.

But Tai Li-an, one of the poll's authors, has done this polling before. Here is 2013 TISR poll reported in TT. What did he find? He very sensibly asked the respondents what they meant by Chinese.

In a tracking poll about identity, wherein respondents were allowed to make multiple choices, 96.5 percent of respondents identified themselves as Taiwanese, an increase of 0.6 percent from a similar poll conducted in September 2008, the survey showed.

In answer to the same question, 85.3 percent of respondents also identified themselves as “citizens of the Republic of China,” 74.1 percent checked Zhonghua minzu (中華民族, Chinese ethnic group), 72.3 percent chose “Asians” and 69.8 percent huaren (華人, ethnic Chinese).

Meanwhile, the percentage of those who identified themselves as zhongguo ren (中國人, Chinese) dropped to 43.5 percent from 46.6 percent in the 2008 poll, and only 7.5 percent said they were “citizens of the People’s Republic of China [PRC],” down 1.9 percent.

Oh yeah, what other question didn't they ask in the WaPo piece?

What's your position on Taiwan independence?

I think we all know why they avoided that question (the answer would have totally undermined their argument), and I think we all know what that answer would be.

Sadly, the really terrifying problem with this piece is not its conventional misunderstandings of Taiwan or its ideologically loaded framings, but the fact that it is going to be quoted again and again by the media and "knowledgeable" people about Taiwan. Can't wait to have people use it to explain Taiwan to me.

UPDATE: A net-friend asks: For “Chinese,” I wonder if they used 「華人」or 「中國人」.

UPDATE 2: The latter. 中國人

UPDATE 3: Taiwan News has a very cogent report which correctly represents the study's findings and also calls them on their bogus attempt to turn "cultural" Chineseness into "political" Chineseness.

+++++++++++++++++++

I asked Johston a very short question about which term they used for "Chinese". His very long response is below:

Dear Mr. Turton,

Thank you for your question.

We used 中國人 in the identity question, the same language as standard surveys, for comparability purposes. If you’re interested in some of the questions we used you can find them linked to the WaPo piece at: http://www.my-formosa.com/DOC_133088.htm

That said, we think people were probably thinking more about ‘cultural’ Chinese, than about political Chinese. We also asked later in the survey about ethnicity (族群身分). People were asked to allocate 10 points across five ethnic identity choices: 閩南人, 客家人, 原住民, 中華民族, 其他人. It turns out that all three of the original identity groups (Taiwanese, Chinese, both) allocated the most points to 閩南人 and then to 中華民族, and there was no significant difference across these three groups in the average number of points allocated to these two ethnicities. In other words, even those who say they are “Chinese only” give more points to 閩南人 than to 中華民族.

I read your blog post about our study. I have a few of responses (I can’t speak for my colleague George Yin, though I’ve cc’d him so that he can disagree with me if he wants to).

We didn’t ask about unification v. independence because we were focused on identity in this, the first of a series of polls we hope to do. We were also limited in terms of the length of the survey. We had a lot of questions about social media use that we wanted to ask. In future polls we can easily include a question about independence, and link this to another series of questions about identity. I worry a bit, however, about the reliability of the answers. Because Taiwanese are under the threat of Chinese military force it may be that support for something that is not de jure independence is compelled to some degree by knowledge of this threat of force. So I am not sure how reliable the typical support for independence questions are. Perhaps one could poll Taiwanese people who are not currently living in Taiwan to see if physical distance from the PLA threat makes a difference. In any event, if you have any suggestions about how to control for threat perceptions as a confounder for support for independence, we’d be happy to hear them.

The question about prioritizing sovereignty or economic development was designed to see what trade offs people make because this may reveal something about their own ideological leanings. In this case, asking people to make policy trade-offs provides information about what they value in cross-strait relations. For example, in Beijing surveys I’ve done we’ve asked whether people support increasing military spending or support keeping it the same or support reducing it. Huge majorities want to increase military spending, small minorities wants to decrease military spending. But when we give them a guns v butter choice — do they support reduce military spending in order to improve social welfare — the proportion of those who want to reduce milex goes up. A forced choice on a policy issue can reveal priorities. In addition, this question about x-strait policy trade-offs seemed to us to be more connected to concrete categories of day-to-day cross-strait policies than the unification v. independent question.

Concerning the use of the term moderate/moderating. The dictionary definition of moderate is someone/something not at the extremes. If there is a single policy dimension anchored by position A at one end and position B at the other, any position between A and B is by definition more moderate, since A and B are the extremes. We didn’t see “moderate” as an ideological term, but as descriptive of a position in a policy space that is not at the extremes (this notion of moderate comes from spatial models of politics). But I take your point that non-social science readers might intepret “moderate” differently.

We tries to unpack the “Both” category precisely because it includes people who see themselves as mostly Taiwanese and people who see themselves as mostly Chinese. But this problem exists for the standard method too. There is likely to be some proportion of people who choose “Taiwanese only” in the standard surveys who would allocate up to 4 points to Chineseness if they had the option. So the standard method may under-count people who would acknowledge some Chinese identity. This then creates an amorphousness about “Taiwanese only” using the standard method. At least the allocation method allows respondents (and scholars) to unpack better what people mean when they say they are “Both”.

Our goal in this piece was to show the problems that wording and question choice raise for measuring identity, and perhaps to encourage others to try using the allocation method as well to see whether our finds are replicable. Surveys are not, nor should be, the only tool for measuring identity. Surveys should be combined with anthropological observation, and very labor intensive analysis of popular culture content (see for example the huge project developed by Ted Hopf: https://nationalidentitydatabase.wordpress.com/).

If you have any other questions about the survey, please don’t hesitate to ask.

Best wishes,

Iain Johnston
_____________
Daily Links:

• Hackers attack Taiwan agencies.
• Party assets committee learns from Germany
• When Taiwanese students go to universities in Norway, it redefines them as Chinese nationals. Or else.
• The Southbound medical plan: Taiwan expands its links
• Former Taipei mayor Hau to lead KMT delegation to China
• Brian H identifies toxic masculinity as a root cause of the constant flow of violence against women in Taiwan. Also check out this journal piece on masculinity in Taiwan
• Brian H again on the singer Amei and feminist politics in Taiwan's aborigines
• Damn. F-16 crashes in N Taiwan mountains.
• Taiwan News: 10% of Taiwan's 1-9th graders are second-generation immigrants. 
• Housing transactions up

_______________________
[Taiwan] Don't miss the comments below! And check out my blog and its sidebars for events, links to previous posts and picture posts, and scores of links to other Taiwan blogs and forums!