6 | go green rating scale
ratings during a second administration of the Go Green
Rating Scale would be the anticipated and intended
outcome. The Go Green Rating Scale Handbook provides
research-based information on how to take proper steps
The area of most interest and concern for the Go Green
Rating Scale is that of inter-rater consistency, that is, how
well the Go Green Rating Scale accurately assesses the
environment of the child care setting regardless of who
is completing it. Inter-rater consistency for the Go Green
Rating Scale was addressed during its development and
then again after the Go Green Rating Scale was finalized.
First, the wording of the Go Green Rating Scale’s sixty-
four guidelines was reviewed to ensure that each guide-
line could be interpreted in a consistent and reliable
manner by child care providers in diverse settings and
regions. An early pilot study with the Go Green Rating
Scale was completed at twenty-eight different sites in
the United States, Canada, and Korea in March 2009.
(A listing of the pilot sites is given on page 107.) Feed-
back from these pilot sites was used to edit and refine
the wording of the guidelines so that the rating of each
guideline could be done consistently.
Next, to assess this consistency of rating, an inter-rater
agreement study was completed using the final edited
Go Green Rating Scale. Four different raters completed
the Go Green Rating Scale for the same child care center.
Results of this study are as follows: The agreement
across the sixty-four guidelines was 71 percent for exact
agreement and 88 percent for adjacent agreement. Scores
from the sixty-four guidelines were combined using the
rules of the Go Green Rating Scale into forty-three rating
scales. The percent of agreement for the forty-three scales
was 66 percent for exact agreement and 87 percent for
adjacent agreement. Scores from the forty-three rating
scales were assigned to nine sections and the scores of each
section were averaged and rounded to two decimal places
by the rules of the Go Green Rating Scale. The percent of
agreement for the nine section scores was 42 percent for
exact agreement and 94 percent for adjacent agreement.
The nine section scores were totaled for a final score.
Based on the final score, an assignment of an overall
rating was made. The scoring by the four different raters
resulted in the same overall rating of “minimal” for this
study’s child care center. The final scores from the four
raters were also very similar in value: 36.39, 33.03, 33.22,
and 33.58, respectively.
As a measure of the final inter-rater reliability, correla-
tions were calculated between two observers for the full
Go Green Rating Scale. The Pearson product moment cor-
relation was .96 and the Spearman rank order correlation
The Go Green Rating Scale can be reported with a high
degree of precision. Users can expect Go Green Rating
Scale scores to be consistent across observers when the
guidelines and rules of the Go Green Rating Scale are ap-
propriately followed as explained in the administration
and scoring section.
reliability and validity |