SC Assessment Summit March 2013

NWEA’s
South Carolina
Assessment
Summit
March 25-28, 2013

http://www.slidesh
are.net/NWEA/sc-
assessment-
summit-march-
2013
Slideshare

Your NWEA Team
Laura Riley
Sue Madagan
Alison Levitt
Andy Hegedus

NWEA Update
Updated SC
Linking Study
Common
Core
New Items
Field Testing
Teacher
Evaluation
Virtual
Comparison
Growth
Norms
College
Readiness

MAP Aligned with Common
Core: Ongoing Process
The Bridge Years
(2012) State aligned versions built for
states who adopted CCSS +15%
Technology Enhanced Items
under development
(2013) New test versions being built to
reflect grade level standards
Stand Alone Field Testing
commences
The Early Years
(2010) A Review and Validation of the
Common Core Standards
PRESENTED TO CCSSO, ACHIEVE, AND THE NATIONAL
GOVERNORS’ ASSOCIATION
(2011)NWEA releases first Common
Core version of MAP-
• Hand-aligned- resulting in tight
alignment in each content area

Assessment
Version
Release Timing Alignment & Item Types
MAP Aligned with
CC, Version Three
MAP Aligned with
CC, Version Four
Fall 2012
Fall 2013
•Better content coverage.
•Deeper assessment of a
student’s depth of knowledge
(DOK).
•Increase item/test validity.
•More engaging test experience
for students.
Blended
Assessment
TBD Measures both proficiency and
growth . Release a new MAP
assessment solution based on the
CCSS that will provide kids,
educators and parents with both
on-grade performance and off-
grade performance information
MAP Aligned with Common
Core: Ongoing Process

School Year Implementation Phase
2011-2012 Transition Year
2012-2013 Transition Year
2013-2014 Bridge Year
2014-2015 Full Implementation
South Carolina CCSS
Timeline

Impact on Data and Reports
Functional Area Impact
Growth
Measurement
No impact, comparisons are made at the measurement scale level
and are not affected by changes in state standards
Projected
Proficiency
Updated SC linking study will continue to be used.
New linking study will be conducted once there is a sufficient
number of students who completed the CCSS (+15%) aligned state
Assessments.
Goal Level
Reporting
No impact, automatically aggregate the data by goal structure
Norms
Content-independent (used across multiple state’s standards)
Can use these norms for Common Core Assessment
DesCartes/
PGID
No impact, DesCartes/PGID will be available for the currently
licensed Assessments

Link to
http://www.nwea.org/cc_sample/item2.html
Additional Item Types
• Common Stimulus
• Hot Spot
• Drag and Drop
• Click and Pop
• Graphing Calculator
• Keyboard Entry
• Drop Down Lists
• Turn and Slide
• Multiple Enhancements

Technology-Enhanced Items With Interactive
Elements http://www.nwea.org/common-core-new-item-types-map
11
• Better content coverage
• Deeper assessment of a student’s
depth of knowledge (DOK)
• Increase item/test validity
• More engaging test experience
• Interactive elements supported in
July 2012
– Drag & drop
– Click and pop
– Hot spot

Common Stimulus Items
12
• Share a common item asset with other
items
• Chosen adaptively; if selected, a number of
items associated with the common stimulus
item are presented consecutively
• Common stimulus items available in July
2012
– Use passages as the common asset
– Allow deeper assessment of reading
comprehension

Sample Item - Common
Stimulus Item

Sample Item - Drag &
Drop Item

Sample Item - Click &
Pop Item

Link to Additional Item Types
• Graphing Calculator
• Keyboard Entry
• Drop Down Lists
• Turn and Slide
• Multiple Enhancements

• When should our districts consider switching over to CCSS
MAP tests?
– You may change when it’s the best fit/time for YOUR
DISTRICT.
• Can we give SOME students the CCSS version and other
students the state version that we currently give?
– Yes. You may test any grade level you wish on Common
Core and any other grade levels on your state version.
• When must we inform NWEA of our choice to move to CCSS
or to stay with our state version?
– We would like to know your decision about one-two weeks
prior to your district’s downloading of the next season of
data.
FAQ’s about NWEA’s CCSS

• How will proficiency to CCSS be forecasted?
– Until students are scored under the new CCSS, we will use
the 40/70 cuts.
• How will Reports change if we switch to Common Core?
– Teacher reports and all other reports that list goal strands
will now list the goal strands as defined by the Common
Core instead of the strands aligned to your state goal
structures.

• Do the 2011 Norms apply to the Common Core aligned
tests?
– The 2011 norms are carefully constructed to be
independent of any specific test.
• How can we make the change to Common Core MAP tests?
– The process is simple. Just call/email Laura Riley or Sue
Madagan at NWEA

Recorded Webinar- Expires May 13, 2013
• A Guide to MAP and the Common Core for
Teachers: http://nwea.adobeconnect.com/cc1_resources/
• A Guide to MAP and the Common Core for Leaders:
http://nwea.adobeconnect.com/cc2_resources/
• Handout- Resources- Guide to Common Core and Measures of
Academic Progress (MAP): What Leaders Need to Know
Resources

Bridging the Gaps:
What gaps exist in your
district between what you
need and what’s provided?

• No K-2- New Literacy Movement in state
• Longitudinal Growth Measures- historical data
• Growth Projections
• Predictive Capabilities- Linking Studies (ACT and
future Common Core)
• Blended Assessment- growth and proficiency
• Links to Instructional Providers- Compass, E2020,
Study Island
• Transition Year
NWEA is Distinctive
Things to consider in SC…..

• Interim Assessment- optional
• Adaptability- accurate for high and low performing
students
– . Summative- only grade level results
• Norms- National Norms - 30 million kids (so far)
• Valid and reliable status and growth data necessary for
Teacher Evaluation- supports Teacher Effectiveness
Models
• GRD- Growth Research Data Base-most extensive
collection of student growth data in the country. 4.5
Billion pairs of test items and responses.
Things to consider in SC…..
NWEA is Distinctive

Understanding your
District’s unique needs?

1. Selection of an appropriate test:
• Used for the purpose for which it was designed
(proficiency vs. growth)
• Can accurately measure the test performance of all
students
2. Alignment between the content assessed and
the content to be taught
3. Adjust for context/control for factors outside a
teacher’s direct control (value-added)
Three primary conditions for using tests
for teacher evaluation

-12.00
-11.00
-10.00
-9.00
-8.00
-7.00
-6.00
-5.00
-4.00
-3.00
-2.00
-1.00
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
10.00
11.00
12.00
Average
Growth
Index
Score
and
Range
Mathematics Growth Index Distribution by Teacher - Validity Filtered
Q5
Q4
Q3
Q2
Q1
Each line in this display represents a single teacher. The graphic
shows the average growth index score for each teacher (green
line), plus or minus the standard error of the growth index estimate
(black line). We removed students who had tests of questionable
validity and teachers with fewer than 20 students.
Range of teacher value-added
estimates

Leadership Courage Is A Key
0
1
2
3
4
5
Teacher 1 Teacher 2 Teacher 3
Ratings can be driven by the assessment
Observation Assessment
Real
or
Noise?

• I know my students didn’t make typical
growth . . . my school and my students
aren’t typical
– Need an apples to apples comparison to
demonstrate what is possible – A Proof
Point
Problems we can help you
solve
47

What if we could compare to
similar schools and students?
48
We identify your
schools and students
Identify all matching students from GRD
School Income, Urban vs. Rural Classification
Grade, Subject, Starting Achievement, Assessment
dates
Randomly select comparison group

• Compare your
student’s
growth to
similar students
in similar
schools
• Compare to
State cut scores
• Compare to a
catch-up
Growth Target
49
Virtual Comparison
Groups
(VCGs)

Virtual Comparison
Groups
(VCGs)
Think about
the power in
these
conversations
50

-6.00
-4.00
-2.00
0.00
2.00
4.00
6.00
8.00
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71
Students taking 10+ minutes longer spring than fall All other students
Ensure similar testing conditions since
they matter in the reliability of results
Mean value-added growth by school

• Graphs at classroom, school, district
level
–Two hours of remote support
–Tool with student and testing condition
data
• Optional
–Data Coaching
–Research Consultation
Virtual Comparison Groups
52

Problems we can help you
solve
• I think our students are learning more
now than before, but I don’t have the
time or expertise to prove it
–Need rigorous and defensible analysis of
your longitudinal growth and achievement
data
53

• Supports your work
– District and School Improvement
– Communication with Board, Administrators, Parents
and Community
• We provide you a defensible analysis of your
growth and achievement data
– Grades 2 through 8
– Contiguous grades
– Four or more years
– Fall and Spring
Learning Pattern Reporting

• We use all your Fall and Spring MAP data
– All students are linked and data is cleaned
– Broken into over-lapping four year blocks of data,
each being lagged by one year
What do we do?
55
2
3
4
5
Grade
F S F S F S F S
2009 2010 2011 2012
School Year
F S F S
2007 2008
Data-block 1
Data-block 2
Data-block 3

• A statistical growth model is then fitted
including both achievement and growth
• Represents the “real” school
What do we do?
(cont.)
56
2
3
4
5
Grade
RIT
F S F S F S F S
2009 2010 2011 2012
School Year
Age-cohort
Grade-level

Analyze your data and find patterns
in growth and achievement
57

Findings reported using defensible
indicators about your patterns
2010
2011 2012
2010
2011
2012

• Report on patterns
found in your data
– Four hours of
remote support
– Tool you can use
• Optional
– Data Coaching
– Research
Consultation
Learning Pattern Reporting
59

South Carolina ESEA Waiver
Linking Study
NWEA
Updated January 2013

• Demonstrate CCR expectations
for all students
• Develop high-quality plans to
implement a system of
differentiated recognition and
accountability and support for all
Title 1 districts and schools
4 Principles for ESEA
Waiver approval

• Commit to developing, adopting,
piloting and implementing teacher
and principal evaluation systems
that support student achievement.
• Provide assurance that you will
evaluate and, if needed, revise
administrative requirements to
avoid duplication and unnecessary
burden to the district.
4 Principles for ESEA
Waiver approval

• Arne Duncan currently has
approved the waivers through the
2013-14 school year. At that
time, districts may request
extensions.
ESEA Waiver Timeframe

• Annual Measurable Objectives
(AMO) are now based on
school level mean scale scores
rather than the percent of
students meeting their
proficiency targets.
• (see graphic on next screen)
SC ESEA Waiver Study

• As the AMO increases over the
next several years, so will the
mean RIT scores increase to meet
the escalating demand.
• NWEA correlation tables include
data through the 2017-2018
school year.
AMO increases

Example
Table 2 (below) contains the 2012-
2013 elementary-level AMO targets
and the associated NWEA RIT
targets by grade for a school.
While this study is not able to directly estimate the probability of an
elementary school meeting its AMO target, the school could be considered on
track for success if all students in grades three, four and five have NWEA RIT
scores greater than 206, 212 and 224 respectively.

Performance
Level
Projections
2011 - 2014

Performance
Level
Projections
2014 - 2018

AMO vs. PASS Proficiency
Pass Linking Study
ESEA Study

Correlation and Accuracy by year

Thank you!
Questions?
Laura Riley
Account Manager, NWEA
Laura.Riley@nwea.org
317-893-2413

Partnering to help all kids
learn
Paths to
Postsecondary

NWEA Research
The NWEA Research Team created an alignment
study between students who have valid MAP scores
and also have valid Explore, Plan and ACT scores.
The results showed a correlation between ACT
Entrance scores, MAP RIT scores and the year-to-
year growth path to achieve the desired Entrance
score.

Where did the numbers
come from?
• Active NWEA districts that use EXPLORE,
PLAN, and ACT
• ACT data was matched to corresponding MAP
data at the individual level
• No formal sampling strategies employed other than
to cut extreme residuals

ACT Says:
• The ACT composite “entrance” scores used are
scores of students who the ACT data indicates
have a 50% likelihood of achieving a “B”
average in a freshman-level course.
• The demands of the courses are different in
various post-secondary institutions.

“Types” of Post-Secondary
Institutions in these Examples:
A “State”
College
A “Top-Tier”
Public University
An Elite, “Ivy League”
Institution

ACT Entrance Score for:
A “State” College
Entrance ACT Reading/Math
Composite =
24

A “Top-Tier”
Public University
Composite = 29

Composite = 32
An Elite,
“Ivy League”
Institution

Student “Paths”
Introducing Three Normal Students,
and their (potential)
Postsecondary Paths
Theodore Thirdgrader
Sandra Seventhgrader
Nate Ninethgrader

APPROPRIATE
Postsecondary Paths
Each child is
unique.
Each Postsecondary Path
will be different and MUST
be appropriate for each
individual student.

Theodore Thirdgrader’s
Path
NWEA data indicates that for Theodore to achieve the
entrance Composite ACT score for these institutions, his
spring RIT score should approach:
Entrance ACT
24
Spring RIT
213 (78th %ile)
Entrance ACT
29
Spring RIT
224 (99th %ile)
Entrance ACT
32
Spring RIT
229 (99th %ile)

Another Way to Look at Third
Grade Spring RIT Scores
Average ACT Composite
entrance score for an
Education Major is 20.8
Third grade spring RIT for
a student on a 20.8
trajectory is 209
Average ACT Composite
Engineering Major is 23.7
Third grade spring RIT for
a student on a 23.7
trajectory is 219

Nate Ninthgrader’s Path
NWEA data indicates that for Nate to achieve the entrance
Composite ACT score for these institutions, his spring RIT
score should approach:
Entrance ACT
24
Spring RIT
237 (53rd %ile)
Entrance ACT
29
Spring RIT
246 (71st %ile)
Entrance ACT
32
Spring RIT
251 (79th %ile)

Another Way to Look at Ninth
Grade Spring RIT Scores
Average Composite
Engineering Major is 23.7
Ninth grade Spring RIT
for a student on a 23.7
trajectory is 254

Standalone Field Testing
• Improve ability to assess current and new standards
• Introduce your students to innovative new
technology-enhanced assessment items
• Increase the breadth and depth of MAP® item pool
• Partner with NWEA to maintain the integrity of the
RIT scale
EARN CREDITS TOWARD PRODUCT AND SERVICES FOR
PARTICIPATION!
© 2013 by Northwest Evaluation Association

Current Needs
We are currently seeking students to participate in field
tests and research studies in:
– Spanish Math – grades 2-5 (must be able to read Spanish)
– Math and Reading constructed response – grades 4-10
– Writing Prompt – grades 4-10
– MAP Reading and Math on iPads – grades 4, 6, 8
– Technology–enhanced items - Math – grades 2-10
• Coming back in Fall 2013: Reading, Language Usage, and General
Science
SIGN UP NOW – LIMITED AVAILABILITY!

Participation
Requirements
• Do not have to be a current MAP Partner
• No limit to the number of students who can participate
• Can take multiple tests in different subject areas
• Each test will take ~ 1 hour
• PC or Macs must meet minimum technical requirements (same
as MAP)
• Constructed Response testing - short pre-testing webinar for
proctors required
• Mobile assessment participants must provide their own iPads
MORE INFORMATION AT WWW.NWEA.ORG/SFT

For More Information
• Website: www.nwea.org/sft
• Contact your Account Executive or Account
Manager
• Email questions/requests to the Field Test
Recruiting Team at fieldtest@nwea.org
WE APPRECIATE YOUR PARTNERSHIP!

Thank you for
your continued
partnership with
NWEA.
Sincerely,
Your NWEA Team
Laura Riley
Sue Madagan
Alison Levitt
Andy Hegedus

SC Assessment Summit March 2013

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (13)

Similar to SC Assessment Summit March 2013

Similar to SC Assessment Summit March 2013 (20)

More from NWEA

More from NWEA (20)

Recently uploaded

Recently uploaded (20)

SC Assessment Summit March 2013

Editor's Notes