Counseling researchers often utilize behavioral observation as a data collection strategy. Cohen’s Kappa or weighted Kappa is a common coefficient for demonstrating interrater reliability with categorical data between two independent raters/coders. Researchers may encounter an issue with high agreement between two raters but low or even negative Kappa values. This session will discuss the reasons for this issue in Kappa and demonstrate two alternative coefficients, Prevalence-adjusted bias-adjusted Kappa and AC1, as recommended solutions.