Personality Assessment

B. Latent Variables and Measured Variables

1. Attributes as Latent Variables

• a) physical and psychological constructs are unobservable
• b) inferred from observed effects
• 2. Observed Scales as transformations of Latent Variables
• a) temperature and heat
• b) weight and mass
• c) test score and ability
• 3. Scales and levels of measurement
• a) scales defined in terms of their sensitivity to transformations
• (1) ordinal transformations
• (a) preserve order but not distance: x>y <==> x' > y'
• (b) without zero
• (c) with zero
• (2) partial orders
• (a) some distances are preserved
• (b) some distances are indeterminant
• (3) interval scales preserve distance
• (a) x-y > s-t <==> x'-y' > s' - t'
• (b) Kelvin <-> Celcius <-> Farenheit
• (4) ratio scales preserve distance from 0
• (a) x/y > s/t <==> x'/y'>s'/t'
• (b) interval with zero point
• (c) Meters to Miles
• b) the meaning of transformations
• (1) statistics based upon observed data
• (2) inferences about latent variables
• (a) what kind of inferences can one make about latent variables?
• (b) main effect differences
• (c) interactions
• C. Basic Concepts

1. Measures of Central Tendency

• a) Mode as the most frequent observation
• b) Median as the middle measure
• (1) is not sensitive to distributional shape
• (2) not sensitive to transformations
• c) Mean -- multiple types
• (1) Arithmetic
• (2) Harmonic -- reciprocal of arithmetic mean of reciprocals
• (3) Geometric -- the nth root of n products
• 2. Measures of Dispersion
• a) range
• b) interquartile range
• c) variance
• (1) variance of composites
• 3. Measures of relationship
• a) covariance
• b) regression as the best fitting linear relationship
• (1) expressed in scale units
• c) correlation
• (1) simple correlation
• (a) geometric mean of regression slopes
• (b) scale free
• (2) types of simple correlations -- different forms of the PPMCC
• (a) Pearson product moment correlation
• (b) Spearman rank order
• (c) Point-Biserial
• (d) Phi
• (3) multiple correlation -- n predictors, 1 criterion
• (4) partial correlation -- removing the effect of other predictors
• D. Reliability and Validity

1. Reliability

• a) estimates of rank order
• (1) classic test theory
• (a) parallel tests
• i) alternate form
• ii) stability
• (b) congeneric measurement
• (c) correction for attenuation
• (d) estimation of true score
• (2) consistency
• (a) domain sampling
• i) item-domain correlations
• ii) test - test correlations
• (1) coefficient alpha
• (2) coefficient beta
• (b) alpha as average of all possible split halfs
• (3) generalizability theory
• b) item response theory
• (1) estimates of attribute
• (2) estimates of item difficulty
• (3) 1, 2 and 3 parameter models
• 2. Validity
• a) internal and external sources of validity
• b) face ("faith") validity
• c) concurrent validity
• d) predictive validity
• (1) the use of tests in decision theory
• (a) selection ratio
• (b) success rate
• (2) utilities
• (a) valid positives and valid negatives
• (b) false positives and false negatives
• e) construct validity
• (1) convergent
• (2) discriminant
• (3) incremental
• f) Threats to validity
• (1) Domain specificity versus Generality
• (a) reliability is maximized if items are completely redundant
• (b) predictive validity is maximized if items are completely independent
• (2) Response styles
• (a) "Yea saying"
• (b) Social Desirability
• (c) Extreme response set
• 3. Causal modeling
• a) reliability+validity=causal model
• b) importance of alternative models
• c) goodness of fit
• E. Scale construction

1. Methods of Keying

• a) Rational Keying
• (1) Ask items with direct content relevance
• (2) Example: California Personality Inventory
• (3) Problems:
• (a) Not all items predict in face valid direction
• (b) Need evidence for validity
• b) Theoretical Keying
• (1) Ask items with theoretical relevance
• (2) Example: Jackson Personality Research Form
• (3) Problems:
• (a) Theoretical circularity
• (b) Need evidence for validity
• c) Empirical Keying
• (1) Ask items that discriminate known groups
• (a) Administer wide range of items to People in General and the the criterion group
• (b) Select those items that discriminate criterion group from People in General
• (c) Select items that are most independent of each other
• i) reduces redundancy
• ii) reduces effect of any single domain of items
• (d) Create scale made up of discriminating items
• (e) Validate on different group
• i) Validation on hold out sample
• ii) Cross validation to determine shrinkage
• (2) Example : MMPI, Strong-Campbell
• (3) Problems
• (a) What is meaning of scale?
• (b) Need to continually develop new scales for new groups
• d) Homogeneous Keying
• (1) Select items to represent single domains
• (2) Example: Cattell's 16PF, Eysenck's EPI, EPQ
• (3) Problems:
• (a) Garbage in-Garbage out
• (b) need evidence for validity
• (4) Methods
• (a) Factor Analysis
• i) Factor analysis model: R = FF' + U2
• (1) Explains the inter-test covariances
• (2) explains the reliable (common) part of test variance
• ii) Vocabulary
• (1) Hyperspace and hyperplanes
• (2) Communalities versus uniqueness
• (a) amount of test variance explained by all factors
• (b) "row wise"
• (3) Eigenvalues and eigenvectors
• (a) amount of total variance explained by one factor
• (b) ³Column wise"
• (4) Residual Matrix = R - FF'
• (5) Methods of extraction
• (a) centroid
• (b) principal factors
• (c) minimal residual
• (d) maximum likelihood
• (6) Simple Structure
• (7) Rotations and Transformations
• (a) Orthogonal rotations
• i) VARIMAX
• ii) QUARTIMAX
• iii) BiFactor
• (b) Oblique transformations
• i) OBLIMIN
• ii) BiQuartiMin
• (8) Higher order - 2nd strata factors
• (9) Goodness of fit of model
• (a) size of residual correlations
• (10) Factors versus components
• (a) variables are sums of (hypothetical) factors
• (b) components are sums of (observed) variables
• (11) Indeterminancy of factor scores
• (12) parameters versus observables
• (13) clusters as group factors
• iii) Exploratory
• (1) Number of factors problem
• (a) Scree test
• (b) Chi square
• (c) Very Simple Structure
• (d) Parallel analysis
• (e) Eigen values > 1
• (2) Goodness of fit
• (3) parsimony versus fit -- multidimensional preference function
• iv) Confirmatory
• (1) theory testing
• (2) goodness of fit of one model versus another
• (3) meaning of failure to fit
• (a) non-normal distributions
• (b) large sample sizes ==> non-fit
• v) Problems:
• (1) Sensitive to number of subjects versus number of items
• (2) not encouraged for items (low communalities)
• (b) Principal Components Analysis
• i) model: R = CC'
• ii) Components can be described at data level
• iii) Components are sums of items
• (1) Describe the data as they are
• (2) describe covariances as well as variances
• iv) Items are sums of factors
• (1) factors go beyond the data to estimate common part
• (2) factors are latent
• v) for more than ‰ 20 variables, these distinctions become less important (pragmatically)
• (c) Cluster Analysis "Poor man's factor analysis"
• i) Non-hierarchical
• (1) find most similar pair
• (2) combine them
• (3) add items to this pair until alpha of total fails to increase
• (4) repeat a-c on remaining items
• ii) Hierarchical
• (1) find most similar pair
• (2) combine them
• (3) repeat a-b until some criterion is reached
• (1) Simple to understand
• (2) Aims for direct solution (clusters as group factors)
• (3) Robust to poor correlations
• (1) not well defined as maximizing any particular criterion
• (2) "cut and try"
• (d) General problems -- when not to factor analyze:
• i) too many factors in the data
• ii) too few factors in the data
• iii) unique factor -- defined on only 1 test
• iv) variables are complex
• (1) use Factorially Homogeneous Item Dimensions (FHIDs)
• (2) Homogeneous Item Composites (HICs)
• v) artificially inflated or deflated correlations
• (1) item overlap inflates correlations
• (2) ipsative scoring deflates correlations
• vi) heterogeneity of population
• (1) within group versus between group corrations
• (2) correlations of aggregates ‚ aggregate correlation
• vii) homogeneity of population
• (1) restriction of range will reduce correlations
• ix) different difficulty levels of scales or items
• (1) factorially homogeneous test varying in difficulty will produce multiple factors
• (2) Guttman simplex pattern
• x) Small sample sizes
• (1) error of correlation depends upon sample size
• (2) do not need 10* number of variables
• xi) low correlations and low communalities (e.g. items) make structure harder to identify if using factorial techniques
• e) Does it make a difference?
• (1) For theory construction
• (2) For predicting criteria
• 2. Guidelines for test development
• a) Review theory of attribute to be measured
• (1) Convergent measures
• (2) Discriminant measures
• b) Write items based upon theory
• (1) items drawn from different facets of theory
• (2) items balanced for response styles
• (3) screen items for readability, bias, understandability
• (4) Include "hyperplane stuff"
• (a) possible related constructs
• (b) theoretically important alternatives
• c) Define target population
• (1) Consider issues of homogeneity/heterogeneity
• (2) Consider issues of generalizability
• d) Administer items and record responses
• (1) Monitor for serious, engaged test taking
• (2) Double check for data entry errors
• e) Examine the distribution and search for outliers
• (1) data entry errors
• (2) uncooperative subjects
• f) Form proximity (correlation) matrix
• g) Extract optimal number of factors or clusters
• (1) statistically (chi square and maximum likelihood)
• (2) psychometrically (maximize alpha, beta, VSS)
• (3) for interpretation (to maximize understanding)
• h) Form scales based upon these factors, clusters
• (1) score salient items
• (2) drop non salients
• i) Purify scales -- item analysis
• (1) high correlation with scale
• (2) low correlations with other scales
• (3) low correlations with measures of response styles
• (4) moderate levels of endorsement
• j) Validate against other measures of same and different constructs
• (1) Assess reliabilty
• (a) internal consistency
• (b) stability
• (2) Demonstrate convergent, discriminant and incremental validity
• F. Sources of data

1. Self Report

• a) Direct subjective
• (1) empirical scales: MMPI
• (2) factorial scales: EPI/16PF
• (3) rational scales: PRF
• b) Indirect/projective
• (1) TAT
• (2) Rorschach
• c) Indirect/objective
• (1) Cattell objective test battery
• d) Indirect/other
• (1) Kelly Construct Repetory Grid
• (2) Carroll INDSCAL
• e) Structured interviews
• 2. Other ratings
• a) Peer ratings
• b) supervisory ratings
• c) subordinate ratings
• 3. archival/unobtrusive measures
• a) unobtrusive measures
• b) historical record
• (1) GPA
• (2) Publications
• (3) Citations
• 4. Neuropsychological
• a) neurometrics
• b) "lie detection"
• 5. Performance tests
• a) OSS stress tests
• b) New faculty job talks
• c) Clinical graduate applicant interviews
• G. What is actually measured