Using Weights in the Analysis of Survey Data

UsingWeightsintheAnalysisofSurveyDataDavidR.JohnsonDepartmentofSociologyPopulationResearchInstituteThePennsylvaniaStateUniversityNovember2008WhatisaSurveyWeight?•Avalueassignedtoeachcaseinthedatafile.•Normallyusedtomakestatisticscomputedfromthedatamorerepresentativeofthepopulation.•E.g.,thevalueindicateshowmucheachcasewillcountinastatisticalprocedure.•Examples:–Aweightof2meansthatthecasecountsinthedatasetastwoidenticalcases.–Aweightof1meansthatthecaseonlycountsasonecaseinthedataset.–Weightscan(andoftenare)fractions,butarealwayspositiveandnon-zero.•[inStata,thesearethepweights]TypesofSurveyWeights•Twomostcommontypes:–DesignWeights–Post-StratificationorNon-responseweights•DesignWeight:–Normallyusedtocompensateforover-orunder-samplingofspecificcasesorfordisproportionatestratification.–Example:Itisacommonpracticetoover-sampleminoritygroupmembersorpersonslivinginareaswithlargerpercentageminority.Ifwedoubledthesizeofoursamplefromminorityareas,theneachcaseinthatareawouldgetadesignweightof½or.5–Thedesignweightwhenwewantthestatisticstoberepresentativeofthepopulation.Post-StratificationWeights•Post-StratificationorNon-responseWeight.–Thistypeisusedtocompensateforthatfactthatpersonswithcertaincharacteristicsarenotaslikelytorespondtothesurvey.–Example.Mostgeneralpopulationsurveyshavesubstantiallymorefemalethanmalerespondents(often60/40)althoughthereareoftenmoremalesinthepopulation.Becausethesurveyover-representsfemalesandunder-representsmalesinthepopulationaweightisusedtocompensateforthisbias.–Therearemanyrespondentcharacteristicsthatarelikelytoberelatedtothepropensitytorespond.•Age•Education•Race/ethnicity•Gender•PlaceofresidenceHowDoWeCalculateWeights?•Foranalysis,onlyoneweightpercasecanbeused.Ifweweightfordifferentfactors,theseweightsmustbecombinedtogetherintooneweight.•Letssaywehaveadesignweight(Dwate)andapost-stratification(PSwate)weightforeachcase.•Tocalculateatotalweightthesearemultipliedtogether:•TotalWeight=Dwate*Pswate•Note:nevergiveaweightthevalueof0unlessyouwantthecaseexcludedfromtheanalysis.Itshoulddefaultto1.CalculatingDesignWeights•Ifweknowthesamplingfractionforeachcase,theweightistheinverseofthesamplingfraction.•DesignWeight=1/(samplingfraction)•Thesamplingfractioncouldalsobetheover-samplingamountforagivengrouporarea.•Example:IfweoversampledAfricanAmericansatarate4timesgreaterthantherateforWhites,thanthedesignweightforanAfricanAmericanwouldbe¼andforaWhiterespondentwouldbe1.CalculatingPost-StratificationWeightsorNon-responseWeights•Thisisnormallymoredifficultthendesignweights.•Itrequirestheuseofauxiliaryinformationaboutthepopulationandmaytakeanumberofdifferentvariablesintoaccount.•Informationusuallyneeded:–Populationestimatesofthedistributionofasetofdemographiccharacteristicsthathavealsobeenmeasuredinthesample–Forexample,informationfoundintheCensussuchas:•Gender•Age•Educationalattainment•Householdsize•Residence(e.g.,rural,urban,metropolitan)•RegionSourcesforAuxiliaryStatisticsforcalculatingPost-Stratificationweights•Populationdataforcommunity-basedsamples:–U.S.Censustabulations–TheCurrentPopulationSurvey(CPS)–TheAmericanCommunitySurvey(ACS)•Forothertypesofsurveyssourcecanbe:–Reportsorenrollmentdatafromaschooloruniversity.–Organizationalstatisticsdataarefromanorganization.•Findinggoodestimatesforthepopulationcharacteristicsissometimesachallenge.CalculatingPost-StratificationWeightsGenderPopulationProportionSampleProportionPopulation/SampleWeightFemale.5.6.5/.6.8333Male.5.4.5/.41.25Total11Censusreportisusedtofindthegenderdistributioninthepopulation(50%female).Thisiscomparedtothegenderdistributioninthesampleofcompletedinterviews(60%female.Problem:Whatifyouhavemorethanonecharacteristictobalancewiththepopulation?AdjustingforMultiplePopulationCharacteristics•Optionsforcombiningcharacteristics:–Youcancombinecharacteristicsinasingletabletodothecalculation:•Males18-25•Males26-45•Males46+•Females18-25•Females26-45•Females46+•However:–Youneedtohavethesecrosstabtablesavailableforthepopulationsource–Thenumberofcasesineachcellinthesamplecannotbetoosmall.•Therefore:ItmaybebettertouseseveralseparatefrequencytablesratherthanonebigN-waycrosstabtocomputetheweights,especiallywhenseveralcharacteristicsarebeingbalanced.CalculatingPost-StratificationWeightswhenyouuseseparatefrequencytables•Example:Youhaveseparatetablesfortheage,gender,education,race/ethnicity,metropolitanstatusforthepopulation.[thesearenotcrosstabedwitheachother]•Singlevariablefrequencytablesaremorelikelytobeavailableforthepopulation.•UseoffrequencytablesmayreduceunstableweightsduetosmallNsinthesamplethatmayoccurifcomparingN-waycrosstabs.•Thebigproblemishowdoyoucombinetheweightsforeachcharacteristic?CalculatingPost-StratificationWeights•Differentoptionsforcombiningtheweights.–1.Computeaweightforeachcharacteristicindependentlyandthenmultiplyalltheseweightstogether.NOTRECOMMENDED.Willusuallynotyieldgoodweights.–2.Computeweightsseparatelybutsequentially.•Calculateagenderweightcomparingthepopulationandsamplegenderdistributions.•Weightthesampledatabythegenderweight