Topic-3-Validity

Topic3ValidityValidityDefinition:Atestissaidtobevalidifitmeasuresaccuratelywhatitisintendedtomeasure.Typesofvalidity:ContentvalidityCriterion-relatedvalidityConstructvalidityFacevalidity3.1ContentvalidityAtestissaidtohavecontentvalidityifitscontentconstitutesarepresentativesampleofthelanguageskills,structures,etc.withwhichitismeanttobeconcerned.Inordertojudgewhetherornotthetesthasacontentvalidity,weneedaspecificationoftheskillsorstructures,ect.thatitismeanttocover.AcomparisonoftestspecificationandtestcontentisthebasisforjudgmentsastocontentvalidityTheimportanceofcontentvalidityFirstly,thegreateratest’scontentvalidity,themorelikelytobeanaccuratemeasureofwhatissupposedtomeasure.Secondly,suchatest,atest-specificationsnotrepresentedatall,islikelytohaveaharmfulbackwasheffect.Areaswhicharenottestedarelikelytobecomeignoredinteachingandleaning.3.2Criterion—relatedvalidity(效标关联效度)Definition:ClassificationDefinition:Anotherapproachtotestvalidityistoseehowfarresultsonthetestagreewiththoseprovidedbysomeindependentandhighlydependableassessmentofthecandidatesability.(CET摸拟题与CET实考题的各项分数的相关性)效标就是考察检定考试效度的参照标准，效标关联效度也就是与该参照标准有关的效度。效标关联效度就是以某一种测验分数与其效标分数之间的相关来表示的效度。换言之，一组考生参加某种考试的分数与同一组考生参加另一公认、的有较高效度的测验结果的一致性程度就是该考试的效标效度。Therearetwokindsofcriterion—relatedvalidity:ConcurrentvalidityConcurrent(共时性)validityisestablishedwhenthetestandcriterionareadministeredataboutthesametime.(例如:设计一次口语测试，因为没有充足的时间去测试课程标准要求的全部语言功能，所以，对每位测试者进行10钟的口试。但我们不能确定10分钟的口试能否反映被试着的实际口语能力。我们先设计了一次考试，考试时间45分钟，并覆盖较多的语言功能，以此，建立了一个口试标准；再将着10分钟的口语考试结果与这个标准作相关分析，然后确定这十分钟的效度。）AnoralachievementtestCourseobjectivesAlargenumberof‘functions’45minutes10minutes?FulltestscoresshortenedtestscoresEvidenceforconcurrentvalidity:Degreeofagreementcalculationofdegreeofagreement:correlationcoefficient(possiblevalue:-1~1,thehigherthebetter)1:perfect(positive)agreement0:zeroagreement-1:negativeagreementAtestoforalabilityforahighleveldiplomaticpostNotrationaltoruntheriskofappointingsomeonewithinsufficientoralabilityLoweragreementNOTallowedAbriefinterviewformingpartofaplacementtestLoweragreementallowedCalculatingcorrelationcoefficientinSPSS.PredictiveValidityThisconcernsthedegreetowhichatestcanpredictcandidates’futureperformance.e.g.Couldaproficiencytestpredictastudent’sabilitytocopewithagraduatecourseataBritishuniversity.Discussion:Validatingaplacementtest.How?3.3ConstructValidity(结构效度):Atest,partofatest,oratestingtechniqueissaidtohaveconstructvalidityifitcanbedemonstratedthatitmeasuresjusttheabilitywhichitissupposedtomeasure.Theword‘construct’referstoanyunderlyingability(ortrait)whichishypothesizedinatheoryoflanguageability.Forexample,whentestingreadingability,wehypothesizethattheabilitytoreadinvolvesasetofsub-abilities,suchasguessingunknownwordmeaningsfromthecontext.Thenweshouldrefertoresearchliteraturetoestablishwhetherornotsuchanabilityexistedandcouldbemeasuredinapartofthetest.Ifwewereabletodemonstratethatwewereindeedmeasuringthatability,thenwesaythetesthasconstructvalidity.结构效度研究是一个搜集证据证实某一考试的确在测试我们想测试的那种心理属性的过程。一个考试如果被证实是在测量我们想测量的那种能力，那么这个考试就具有较好的结构效度，它是一个编制假设再证实假设的过程。Howdoweknowthatthetestismeasuringwritingability?Hypothesis:Ourtheoryofwritingtellsusthatunderlyingwritingabilityareanumberofsub-abilities:controlofpunctuationsensitivitytostylesstructurescoherenceandcohesion….Inordertomeasurethesesub-abilities,weadministerapilottestwhichinvolvestwosteps:用选择题测试写作能力（间接考试）步骤：1.找一组被试（第一次参加考试），对他们的写作能力全面抽样（让他们做包含标点、语法、体裁等的选择题）；2.把测试结果与范文相比较：如果结果与范文相关系数高，说明可以用选择题测试写作能力1.Weobtainextensivesamplesofthewritingabilityofthegrouptowhomtestisfirstadministered,andhavethesereliablyscored.2.Wethencomparescoresonthepilottestwiththescoresgivenforthesamplesofwriting.Iftheagreementishigh,wearemeasuringwritingabilitywiththetest.按照假设分别测量各项能力：假设：标点语法体裁写作能力↙↓↘／∣﹨／∣﹨／∣﹨单选填空改错单选填空改错单选填空改错7066685560628580823.4FaceValiditye.g.Anoraltestdoesn’trequirecandidatestospeakwouldnothavefacevalidity.Atestissaidtohavefacevalidityifitlooksasifitmeasureswhatitsupposedtomeasure.ReaderActivities:Consideranytestwithwhichyouarefamiliar.Assesseachofthemintermsofthevariouskindsofvaliditythathavepresentedinthistopic.End!