GreatPhillySchools Ratings Methodology (2018)

  • st malachy 73
(Updated March 2018)
 

GreatPhillySchools (“GPS”) school profiles include an Overall rating, as well as  ratings in Math, Reading, Science, School Incidents, and Attendance. High schools also receive a rating called College Bound. The ratings provide a snapshot of school performance in each category by comparing the school’s outcomes to other schools serving similar grade levels. Student outcomes for each rating category come from various measures. See our Data Sources Article for a description of the specific measures used for each rating.

Overall Ratings Calculation

A school’s overall rating is calculated by weighting each of the other ratings as follows:

Elementary and Middle Schools (Grades K-5 and Grades 6-8):
  • Academics: 70% (35% math and 35% reading)

  • Attendance: 25%

  • School Incidents: 5%

High Schools (Grades 9-12):
  • Academics: 50% (25% math and 25% reading)

  • College Bound: 20%

  • Attendance: 25%

  • School Incidents: 5%

Note: For the purposes of calculating 'Overall' ratings, schools are grouped based on the highest grade level served at the school. For example, a school serving grades 6 through 12 would be characterized as a "High School".  

For more detail on the data sources used to determine school ratings, see GPS Data Sources.

Ratings Scale

The ratings are determined by school performance in each category on a scale from 1 to 10 based on their decile ranking. GPS uses interval-based deciles to produce ratings. Unlike traditional deciles that require forced differentiation between schools with similar performance, interval-based deciles allow for cut points defined by the range of existing school performance and groups schools with similar performance.

GPS uses two variations of interval-based deciles:

  • Equal Interval Deciles: The range between the highest-performing school and the lowest-performing school is divided into 10 equal tiers based on the performance metric. Any two schools with the same relative gap in performance will have a similar relative gap in performance tier. Equal interval deciles are used for the Overall rating and the Academic, Attendance and College Bound ratings.

  • Logarithmic Interval Deciles: The range between the highest-performing school and lowest-performing school is divided into 10 exponentially increasing size tiers based on the performance metric. Any two schools with the same relative exponential difference in performance will have a similar relative gap in performance tier. Logarithmic interval deciles are used for GPS’ School Incident ratings.

Rating Schools Based on Grade Levels

Schools are grouped together in three categories based on highest grade level served (3-5, 6-8, and 9-12). The decile ratings for schools are the averaged together on a weighted basis for elementary/middle schools, in addition to high schools (see above for rating weights). Schools are then ranked according to their weighted overall averages and deciled in equal-size increments from the top end to the bottom end of the range. All non-special admit schools are ranked and sorted independently from District Special Admit Schools.   

Grouping schools based on grade levels ensures that we are comparing performance of students in the same grade levels with one another. This allows us to compare the performance of schools serving only elementary and/or middle school grades (i.e., K-5 or 6-8) to the performance of schools serving a wider grade band (i.e. K-8). 

Academic Sub-Ratings

Math: This rating represents the percentage of students performing in the four performance bands in math, as measured by the most recently available standardized test data. For reference, the performance bands are as follows: (1) Below Basic; (2) Basic; (3) Proficient; and (4) Advanced.

The weighting for each band is listed below and is used to calculate each school’s raw score. The raw score is then compared to other schools in Philadelphia using equal interval deciles. See Data Sources for information about standardized test data sources.

Reading: This rating represents the percentage of students performing in each of the four performance bands in reading, as measured by the most recently available standardized test data. For reference, the performance bands are as follows: (1) Below Basic; (2) Basic; (3) Proficient; and (4) Advanced.

The weighting for each band is listed below and is used to calculate each school’s raw score. The raw score is then compared to other schools in Philadelphia using equal interval deciles. See Data Sources for information about standardized test data sources.

Science: This rating represents the percentage of students performing in each of the four performance bands in science, as measured by the most recently available standardized test data. For reference, the performance bands are as follows: (1) Below Basic; (2) Basic; (3) Proficient; and (4) Advanced. NOTE: The science rating is not factored into a school’s overall rating.

The weighting for each band is listed below and is used to calculateget each school’s raw score. The raw score is then compared to other schools in Philadelphia using equal interval deciles. See Data Sources for information about standardized test data sources.

  • 3 x Percent of Student Scoring Advanced

  • 2 x Percent of Students Scoring Proficient

  • 1 x Percent of Students Scoring Basic

  • No points for Students Scoring at Below Basic

This change in calculating academic Sub-Ratings is intended to better differentiate between schools at both the top and the bottom ranges. While a score of ‘Basic’ is not sufficient, this weighting recognizes schools with fewer students performing in the ‘Below Basic’ tier and rewards schools with students scoring in the ‘Advanced’ tier.

GPS then creates a list of all schools sorted highest to lowest by their raw scores for each subject area. Equal Interval Deciles are used to sort those schools into 10 groups with equal increments between them, known as deciles. The highest-performing group of schools at the top of that list is assigned a “10”, the next group assigned a “9”, and so on. All non-special admit schools are ranked and sorted independently from  “Special Admission Schools”.

GPS ratings are comparative -- meaning a school that is assigned a “10” represents the highest performing amongst all Philadelphia schools for that particular metric; likewise, a school that is assigned a “1” represents the lowest performing amongst all Philadelphia schools for that metric. For example, a school assigned a “10” in reading does not mean that 100% of their students performed at or above grade level in reading in the most recent standardized test. Instead, a “10” rating means the school is ranked in the top group of all Philadelphia schools for their reading outcomes.

Comparing Academic Performance Between Public and Catholic Schools

Grades K-8:
 

All elementary and middle school students (grades K-8) enrolled in public and public charter schools take the Pennsylvania System of School Assessment (PSSA) exam in various subjects. The PSSA is a standardized, criterion-referenced exam primarily designed to assess grade-level proficiency in Math, Reading and Science. Students enrolled in Catholic schools take the TerraNova exam -- a standardized, norm-referenced assessment primarily designed to assess student performance relative to all other TerraNova test-takers in Math and Reading

To establish comparative ratings among all Public and Catholic Schools in Reading and Math, GPS uses Catholic schools student outcomes on the TerraNova as measured by the National Percentile, which is then aligned to PSSA performance bands using national percentile thresholds published by NWEA MAP in their Concordance Report.

Grades 9-12:

All public high schools and public charter high schools administer Keystone exams in Algebra 1, English 1, and Biology. Catholic high schools do not administer Keystone exams; however, they do administer the Standard Aptitude Test (SAT). To establish comparative ratings among all Public and Catholic Schools in Reading and Math,GPS uses the Catholic school students SAT performance against the SAT’s College Ready Benchmarks listed below.

The College Ready Benchmarks have three categories published; however the top category, ‘Green’, is considered to include both proficient and advanced performance. GPS created an additional ‘Blue’ category that separates ‘Proficient’ from ‘Advanced’ based on the same percentile performance that defines the ‘Advanced’ category on the Keystone exams for Algebra 1 (top 22%) and English 1 (top 9%). These thresholds are used to determine student outcomes in each of the Keystone performance bands as shown below.

SAT Benchmark
  Evidence-based
  Reading and Writing
       Math      
   Keystone Tier
Red 200-450 200-500 Below Basic
Yellow 460-470 510-520 Basic
Green 480-660 530-600 Proficient
"Blue" 670-800 610-800 Advanced

 

High Academic Growth (green arrow) indicator (Not Included in Ratings)

Beginning in 2018, GPS incldues an indicator for public district and charter schools that have shown significant student growth on state assessments over the most recent three years. The underlying data for this indicator comes from the Pennsylvania Value Added Assessment System (PVAAS). GPS obtained from PVAAS 3-year Average Growth Index (AGI) data, separately in reading and math, for each school. Schools with a 3-year AGI of 2.0 or higher in both reading and math receive a “High Academic Growth” indicator (green arrow) on their profiles. An AGI of 2 indicates significant evidence that the school exceeded the norm for Pennsylvania Academic Growth.

Note: Catholic and private schools do not receive a PVAAS report and therefore are not eligible for the the “High Academic Growth” (green arrow) indicator. This does not mean that Catholic and private schools are not demonstrating significant student growth. Rather, there is not currently a system to reliably compare private school growth to public district and charter schools.

Non-Academic Sub-Ratings

Attendance: This rating represents the percentage of students who attended 95% or more of enrolled instructional days. The attendance outcomes for all schools are sorted from highest to lowest, and Equal Interval Deciles generate ratings on a scale of 1 to 10 for each school.

School Incidents: This rating represents the reportable incident count per 100 students. We calculate this rating based on the number of reportable incidents, as reported by schools (see data sources for more information).

College Bound: This rating represents the percentage of annual graduates who enroll in any postsecondary institution in the first fall following their graduation date. Schools are then divided into equal-sized groups to create ratings from 1 to 10, with 10 representing the highest college enrollment rates.

Overall Rating: This rating represents a combination of each school’s academic, college bound (for high schools), attendance, and school incident ratings. The data is combined into one master list ranked by weighted formulas (see above). Schools raw scores are then grouped into equal interval deciles from 1 to 10 ito produce an overall rating.

School Incident Calculation

Each school’s total incident count is divided by 100 students in order to calculate an incident rate comparable across schools with various student enrollment. For example, a school with 1,000 students and 20 incidents has an incident rate of 2-per-100 students, while a school of 300 students and 9 incidents has an incident rate of 3-per-100. Incident Count data for all schools is then sorted from highest to lowest, divided into ten different groups, then assigned school incident ratings on a scale of 1 to 10.

Unlike other rating categories, the groups for school incident ratings are established on a logarithmic scale, rather than in equal-sized increments, meaning each successive increment is exponentially larger. This allows for greater articulation of differences at the low end of the scale. On an Equal Interval decile scale, a school with four incidents per 100 students might be rated the same as a school with one incident per 100, even though that represents four times as many incidents. A 10 rating for school incidents means a school falls into the band of schools with the fewest number of incidents per 100 students. 

 

Disclaimer: Data collection and analysis for GPS was managed by the Philadelphia School Partnership in consultation with Data Partners LLC, a data analytics consulting firm with significant experience analyzing and modeling student and school performance data. Guidance and assistance was also provided by the Pennsylvania Department of Education, McGraw-Hill’s CTB division (producer of the TerraNova) and Frontier 21 Solutions, an education research company.