Personality Assessment

A. The role of measurement theory in personality research

1. as a tool for prediction and control

2. as an aid to theory building and testing

B. Latent Variables and Measured Variables

1. Attributes as Latent Variables

a) physical and psychological constructs are unobservable
b) inferred from observed effects

2. Observed Scales as transformations of Latent Variables

a) temperature and heat
b) weight and mass
c) test score and ability

3. Scales and levels of measurement

a) scales defined in terms of their sensitivity to transformations
- (1) ordinal transformations
  - (a) preserve order but not distance: x>y <==> x' > y'
  - (b) without zero
  - (c) with zero
- (2) partial orders
  - (a) some distances are preserved
  - (b) some distances are indeterminant
- (3) interval scales preserve distance
  - (a) x-y > s-t <==> x'-y' > s' - t'
  - (b) Kelvin <-> Celcius <-> Farenheit
- (4) ratio scales preserve distance from 0
  - (a) x/y > s/t <==> x'/y'>s'/t'
  - (b) interval with zero point
  - (c) Meters to Miles
b) the meaning of transformations
- (1) statistics based upon observed data
- (2) inferences about latent variables
  - (a) what kind of inferences can one make about latent variables?
  - (b) main effect differences
  - (c) interactions

C. Basic Concepts

1. Measures of Central Tendency

a) Mode as the most frequent observation
b) Median as the middle measure
- (1) is not sensitive to distributional shape
- (2) not sensitive to transformations
c) Mean -- multiple types
- (1) Arithmetic
- (2) Harmonic -- reciprocal of arithmetic mean of reciprocals
- (3) Geometric -- the nth root of n products

2. Measures of Dispersion

a) range
b) interquartile range
c) variance
- (1) variance of composites

3. Measures of relationship

a) covariance
b) regression as the best fitting linear relationship
- (1) expressed in scale units
c) correlation
- (1) simple correlation
  - (a) geometric mean of regression slopes
  - (b) scale free
- (2) types of simple correlations -- different forms of the PPMCC
  - (a) Pearson product moment correlation
  - (b) Spearman rank order
  - (c) Point-Biserial
  - (d) Phi
- (3) multiple correlation -- n predictors, 1 criterion
- (4) partial correlation -- removing the effect of other predictors

D. Reliability and Validity

1. Reliability

a) estimates of rank order
- (1) classic test theory
  - (a) parallel tests
    - i) alternate form
    - ii) stability
  - (b) congeneric measurement
  - (c) correction for attenuation
  - (d) estimation of true score
- (2) consistency
  - (a) domain sampling
    - i) item-domain correlations
    - ii) test - test correlations
      - (1) coefficient alpha
      - (2) coefficient beta
  - (b) alpha as average of all possible split halfs
- (3) generalizability theory
b) item response theory
- (1) estimates of attribute
- (2) estimates of item difficulty
- (3) 1, 2 and 3 parameter models

2. Validity

a) internal and external sources of validity
b) face ("faith") validity
c) concurrent validity
d) predictive validity
- (1) the use of tests in decision theory
  - (a) selection ratio
  - (b) success rate
- (2) utilities
  - (a) valid positives and valid negatives
  - (b) false positives and false negatives
e) construct validity
- (1) convergent
- (2) discriminant
- (3) incremental
f) Threats to validity
- (1) Domain specificity versus Generality
  - (a) reliability is maximized if items are completely redundant
  - (b) predictive validity is maximized if items are completely independent
- (2) Response styles
  - (a) "Yea saying"
  - (b) Social Desirability
  - (c) Extreme response set

3. Causal modeling

a) reliability+validity=causal model
b) importance of alternative models
c) goodness of fit

E. Scale construction

1. Methods of Keying

a) Rational Keying
- (1) Ask items with direct content relevance
- (2) Example: California Personality Inventory
- (3) Problems:
  - (a) Not all items predict in face valid direction
  - (b) Need evidence for validity
b) Theoretical Keying
- (1) Ask items with theoretical relevance
- (2) Example: Jackson Personality Research Form
- (3) Problems:
  - (a) Theoretical circularity
  - (b) Need evidence for validity
c) Empirical Keying
- (1) Ask items that discriminate known groups
  - (a) Administer wide range of items to People in General and the the criterion group
  - (b) Select those items that discriminate criterion group from People in General
  - (c) Select items that are most independent of each other
    - i) reduces redundancy
    - ii) reduces effect of any single domain of items
  - (d) Create scale made up of discriminating items
  - (e) Validate on different group
    - i) Validation on hold out sample
    - ii) Cross validation to determine shrinkage
- (2) Example : MMPI, Strong-Campbell
- (3) Problems
  - (a) What is meaning of scale?
  - (b) Need to continually develop new scales for new groups
d) Homogeneous Keying
- (1) Select items to represent single domains
- (2) Example: Cattell's 16PF, Eysenck's EPI, EPQ
- (3) Problems:
  - (a) Garbage in-Garbage out
  - (b) need evidence for validity
- (4) Methods
  - (a) Factor Analysis
    - i) Factor analysis model: R = FF' + U2
      - (1) Explains the inter-test covariances
      - (2) explains the reliable (common) part of test variance
    - ii) Vocabulary
      - (1) Hyperspace and hyperplanes
      - (2) Communalities versus uniqueness
        (a) amount of test variance explained by all factors
        (b) "row wise"
      - (3) Eigenvalues and eigenvectors
        (a) amount of total variance explained by one factor
        (b) ³Column wise"
      - (4) Residual Matrix = R - FF'
      - (5) Methods of extraction
        (a) centroid
        (b) principal factors
        (c) minimal residual
        (d) maximum likelihood
      - (6) Simple Structure
      - (7) Rotations and Transformations
        (a) Orthogonal rotations
        i) VARIMAX
        ii) QUARTIMAX
        iii) BiFactor
        (b) Oblique transformations
        i) OBLIMIN
        ii) BiQuartiMin
      - (8) Higher order - 2nd strata factors
      - (9) Goodness of fit of model
        (a) size of residual correlations
      - (10) Factors versus components
        (a) variables are sums of (hypothetical) factors
        (b) components are sums of (observed) variables
      - (11) Indeterminancy of factor scores
      - (12) parameters versus observables
      - (13) clusters as group factors
    - iii) Exploratory
      - (1) Number of factors problem
        (a) Scree test
        (b) Chi square
        (c) Very Simple Structure
        (d) Parallel analysis
        (e) Eigen values > 1
      - (2) Goodness of fit
      - (3) parsimony versus fit -- multidimensional preference function
    - iv) Confirmatory
      - (1) theory testing
      - (2) goodness of fit of one model versus another
      - (3) meaning of failure to fit
        (a) non-normal distributions
        (b) large sample sizes ==> non-fit
    - v) Problems:
      - (1) Sensitive to number of subjects versus number of items
      - (2) not encouraged for items (low communalities)
  - (b) Principal Components Analysis
    - i) model: R = CC'
    - ii) Components can be described at data level
    - iii) Components are sums of items
      - (1) Describe the data as they are
      - (2) describe covariances as well as variances
    - iv) Items are sums of factors
      - (1) factors go beyond the data to estimate common part
      - (2) factors are latent
    - v) for more than ‰ 20 variables, these distinctions become less important (pragmatically)
  - (c) Cluster Analysis "Poor man's factor analysis"
    - i) Non-hierarchical
      - (1) find most similar pair
      - (2) combine them
      - (3) add items to this pair until alpha of total fails to increase
      - (4) repeat a-c on remaining items
    - ii) Hierarchical
      - (1) find most similar pair
      - (2) combine them
      - (3) repeat a-b until some criterion is reached
    - iii) Advantages
      - (1) Simple to understand
      - (2) Aims for direct solution (clusters as group factors)
      - (3) Robust to poor correlations
    - iv) Disadvantages
      - (1) not well defined as maximizing any particular criterion
      - (2) "cut and try"
  - (d) General problems -- when not to factor analyze:
    - i) too many factors in the data
    - ii) too few factors in the data
    - iii) unique factor -- defined on only 1 test
    - iv) variables are complex
      - (1) use Factorially Homogeneous Item Dimensions (FHIDs)
      - (2) Homogeneous Item Composites (HICs)
    - v) artificially inflated or deflated correlations
      - (1) item overlap inflates correlations
      - (2) ipsative scoring deflates correlations
    - vi) heterogeneity of population
      - (1) within group versus between group corrations
      - (2) correlations of aggregates ‚ aggregate correlation
    - vii) homogeneity of population
      - (1) restriction of range will reduce correlations
    - viii) bad distributions
    - ix) different difficulty levels of scales or items
      - (1) factorially homogeneous test varying in difficulty will produce multiple factors
      - (2) Guttman simplex pattern
    - x) Small sample sizes
      - (1) error of correlation depends upon sample size
      - (2) do not need 10* number of variables
    - xi) low correlations and low communalities (e.g. items) make structure harder to identify if using factorial techniques
e) Does it make a difference?
- (1) For theory construction
- (2) For predicting criteria

2. Guidelines for test development

a) Review theory of attribute to be measured
- (1) Convergent measures
- (2) Discriminant measures
b) Write items based upon theory
- (1) items drawn from different facets of theory
- (2) items balanced for response styles
- (3) screen items for readability, bias, understandability
- (4) Include "hyperplane stuff"
  - (a) possible related constructs
  - (b) theoretically important alternatives
c) Define target population
- (1) Consider issues of homogeneity/heterogeneity
- (2) Consider issues of generalizability
d) Administer items and record responses
- (1) Monitor for serious, engaged test taking
- (2) Double check for data entry errors
e) Examine the distribution and search for outliers
- (1) data entry errors
- (2) uncooperative subjects
f) Form proximity (correlation) matrix
g) Extract optimal number of factors or clusters
- (1) statistically (chi square and maximum likelihood)
- (2) psychometrically (maximize alpha, beta, VSS)
- (3) for interpretation (to maximize understanding)
h) Form scales based upon these factors, clusters
- (1) score salient items
- (2) drop non salients
i) Purify scales -- item analysis
- (1) high correlation with scale
- (2) low correlations with other scales
- (3) low correlations with measures of response styles
- (4) moderate levels of endorsement
j) Validate against other measures of same and different constructs
- (1) Assess reliabilty
  - (a) internal consistency
  - (b) stability
- (2) Demonstrate convergent, discriminant and incremental validity

F. Sources of data

1. Self Report

a) Direct subjective
- (1) empirical scales: MMPI
- (2) factorial scales: EPI/16PF
- (3) rational scales: PRF
b) Indirect/projective
- (1) TAT
- (2) Rorschach
c) Indirect/objective
- (1) Cattell objective test battery
d) Indirect/other
- (1) Kelly Construct Repetory Grid
- (2) Carroll INDSCAL
e) Structured interviews

2. Other ratings

a) Peer ratings
b) supervisory ratings
c) subordinate ratings

3. archival/unobtrusive measures

a) unobtrusive measures
b) historical record
- (1) GPA
- (2) Publications
- (3) Citations

4. Neuropsychological

a) neurometrics
b) "lie detection"

5. Performance tests

a) OSS stress tests
b) New faculty job talks
c) Clinical graduate applicant interviews

G. What is actually measured