Eeeenteresting

[textualdeviance]

So, I'm working on infographics for my religion article, and I'm using data from the 2004 General Social Survey.

There's a very cool bit of analysis of this data here.

A few fascinating things to point out:

-Almost 55 percent of blacks described their religion as fundamentalist, compared to only 25 percent of whites.*
-100 percent of Catholics described their religion as moderate.
-22 percent of people describing themselves as liberal also describe their religion as fundamentalist.
-74 percent of fundies have a high school education or less.
-Only 5 percent of fundies have a graduate degree or higher, and 44 percent of people with a graduate degree describe their religion as liberal.

*This is probably explained in part by population distribution. About 55 percent of American blacks live in the south, where fundamentalist religion is dominant.

Comments

[twangs.livejournal.com]

I hate to put on my statistics hat (particulary when I'm not at work) but...

They are some interesting statistics but its probably better to know where they collected the stats from and how. As yours is a free country most people are paid for the information they give. Plus are (rounding up to a rounder number here) 3000 people really going to be that representative of 301,276,978??

There's your real story - the fact that most government policy is developed and backed up by statistics based on tiny populations such as these.

Makes you wonder really.

I worked on the Australian General Social Survey, and its really quite interesting to me how much emphasis there is on religion in the US version. This year 'ours' focused on volunteering and social capital by age/sex/income/state. The only mention of religion was if someone volunteered for an organisation, one of the types of organisations they could choose was "Religious Organisation".

And the other emphasis was whether the respondent had watched or participated in sport.

It shows where the money comes from for a survey really...

[textualdeviance.livejournal.com]

Religion is a huge component of US cultural life. Not including it in a general survey of such wouldn't make sense.

3,000 is a small sample size, yes, but the data here are similar to other (older; I used this one due to recency) studies on the same subject, particularly the American Religious Identification Survey, which had a sample size of more than 50,000.

People aren't paid to take the GSS, btw. The study itself is funded by a large collective, with varying agendas. (Data collection details are here.)

Incidentally, as a statistician, why would you say that a carefully selected sample isn't necessarily representative? I understand sample bias when the participant pool is in a small geographic area, or only of a certain demographic (age, race, etc.) but if care has been taken to get a cross-section of the entire population, the data should be valid for any sample size over a handful.

The one sample bias I think this study would have is that the interviews were conducted in person. I feel people are less likely to be candid about controversial subjects when face-to-face with an interviewer. More honest answers on those things come from anonymous surveys with no direct contact. Even phone surveys can be problematic. One of the problems with predicting elections is that many polls are conducted only via landlines, and a lot of young people no longer have those.

I'm usually careful about what studies I choose to draw data from. I've seen too many screwy ones (like the sex survey that only sampled people from Minnesota, and asked them questions about their practices while other family members were in the room!) to not give them a skeptical eye. One study that I threw out, for instance, was conducted by Baylor University, which is part of the Baptist church.

[pixxelpuss.livejournal.com]

The polling in person thing is problematic. I have the same problem with the last really major and comprehensive sex survey that was done in this country.

[twangs.livejournal.com]

First:I can't believe that people were asked about their sexual practices with the rest of the family in the room!!! Such tact.

Why I said such a small sample, no matter how carefully selected (even taking into account their nifty "list sample" and weighting) is not going to be truly representative of a population so big.

Because the smaller the sample size in relation to actual population the larger the Relative Standard Error (RSE) is going to be. Meaning that the possibility that the data is inaccurate increases.

Estimates (weighted numbers) derived from very small samples attract such high RSEs that it seriously limits their value for most reasonable uses. Only estimates with an RSE less than 25% are considered (mostly)reliable for informing any sort of decision or theory.

For instance:
In a survey of 9000, estimates between 4,582 and 1,300 will have a RSE between 25% and 50% . Estimates smaller than 1,300 have a RSE greater than 50% and should be read and used with a grain of salt (more tub of salt).

That said, its always usful to know what a small population is doing so that the Census questionnaire design can be modified to capture any identified potential trends during census (if neccessary - and hence my statement about these sort of surveys influencing policy decisions - because Censuses don't happen all that often even in little sub samples).

In my experience as both an interviewer and in survey design/processing people are universally honest about most things (except income where people will refuse to answer) in a survey. No matter the type of interview they will honestly give you want they think you want to hear (whether their motive is to get rid of you by giving the most succinct answer or wanting to give you the "right" answer).

So I'm not saying its all bad, and agreeing with you that it's very interesting for that sample of the population.


P.S. I'm taking the coding instructions for the GSS to work - if only for the one that mentions "The Devil" as a response category.

[textualdeviance.livejournal.com]

Also, US government policy isn't based on surveys like this. It's based on Census data.

Individual politicians may go off of other data, but when it comes to specific agency things like how much aid a given state gets, they go by the hard numbers there.