Lessons from the Frontier of Courseware Development
•Download as PPTX, PDF•
1 like•297 views
This is a presentation I did last week (6/16/16) at the New Media Consortium (NMC) summer conference: The number of ed tech startups and the significant investment in adaptive courseware solutions can remind people of the 1860s gold rush: some prospered, but many failed. In the past two years of developing OpenStax Tutor, the OpenStax team has learned several valuable lessons that can benefit educators, digital courseware providers, and students from making costly miscalculations. Lessons learned and presented are 1) technology with high-quality content and assessment is most valuable; 2) instructor workflow is a highway, not a fork in the road; 3) emergence of data privacy standards can accelerate acceptance of digital courseware; and, 4) it's not about price, it's about value.
Recommended
Recommended
More Related Content
What's hot
What's hot (17)
Viewers also liked
Viewers also liked (20)
Similar to Lessons from the Frontier of Courseware Development
Similar to Lessons from the Frontier of Courseware Development (20)
Recently uploaded
Recently uploaded (20)
Lessons from the Frontier of Courseware Development
- 1. Associate Director, Marketing and Communication Dani Nicholson Lessons from the frontier of courseware development
- 2. full digital courseware that uses machine learning algorithms and principles from cognitive science to improve learning
- 6. 1. Technology is dependent on high- quality content and assessment
- 7. Quality content drives usage • Instructors evaluate content before technology • Instructors are more accepting of technology deficiencies than content deficiencies 7
- 8. Decoupling technology has risks • Instructors are not generally interested in adding content to a tool • A platform needs to have high-quality content out of the box 8
- 9. Student needs should drive technology development • Technology is now expected to adapt to user preferences • Key questions for OpenStax have been - How do we introduce spaced practice without frustrating students? - What’s the right level of interactivity? - How do reading passages flow with the courseware? 9
- 10. 2. Instructor workflow is a lane on a highway, not a fork in a road
- 11. Tech designed within the workflow can flourish • Students and instructors have limited time • There is no incentive for them to change their workflow 11
- 12. Technology must improve efficiency through workflow • Efficiency doesn’t mean students learn faster, it means greater return on effort. • Allow instructors to focus on more meaningful engagement with students 12
- 13. Technology must incorporate reflection and reaction • For instructors, where do they need to focus their teaching efforts? • For students, where do they need to focus their studying efforts? 13
- 15. 3. Emergence of data privacy standards can accelerate acceptance of digital courseware
- 16. Preserve anonymity and academic freedom • For ed tech providers - Student data must be anonymous - Not used for commercial purposes • For campus administrators - Use data to help students - Use data to enhance on academic freedom 16
- 17. Improve the science of learning • With data privacy standards, we can share research more easily and make industry-wide improvements • Without sharing data, it’s more difficult to see positive and null effects of technology 17
- 18. knowledge in the right hands • Institutions: do you want ed tech providers to know more about your students than you do? • We need to share data with institutions so that they can help students improve and grow. 18
- 19. It's not about price, it's about educational value • Today: there isn’t enough research on efficacy, so pricing is generally based on input and marketing costs • The future: rigorous research and iterative improvement based on that research can determine value and hopefully price to students 19
- 20. Questions?
Editor's Notes
- Introduction I’m Dani Nicholson and I handle marketing and communications at OpenStax. OpenStax is Rice University’s non-profit education initiative and is probably best known for its free, high-quality textbooks for college courses. Although we’re part of Rice, we receive grant funding to create high-quality textbooks that have the look and feel of traditional textbooks, but are available for free online and in PDF, and for a very low cost in print. We also have special partnerships with for-profit companies who sell products and services around our free textbooks and make them available for very low cost to students.
- For the past 24 months, we’ve been developing OpenStax Tutor. And OpenStax Tutor is a full digital courseware solution that includes reading passages, homework assignments, and interactivity. It uses machine learning algorithms based on principles of cognitive science to help students and teachers improve their return on effort. This tool has been tested in several schools in a few different ways and throughout each stage of the process we’ve conducted research. We’ve hosted teacher advisory boards, we’ve had subject matter experts review the tool, we’ve conducted focus groups, and we have a research team at OpenStax that is studying Tutor, including the effects on learning outcomes.
- To give a little more background on Tutor… OpenStax Tutor has several components, one of them is the performance forecast that can show instructors how their students are performing at the individual level, the group level, and see where they really need to focus their teaching efforts. Students also have the ability to see where they need to improve.
- OpenStax Tutor is powered by our standard high quality OER content.
- And, OpenStax Tutor has the ability build assignments and allow instructors to customize. OpenStax Tutor has several features that we’ve been working hard to create and to measure their effectiveness and usability, however today I want to share with you four key lessons we’ve learned over the past 2 years as we’ve been working on OpenStax Tutor.
- The first lesson learned: Technology is dependent on high-quality content and assessment.
- Quality content drives usage. We’ve learned that instructors will generally evaluate the content first. Does it meet scope and sequence requirements? Are key concepts explained in a way they think their students will understand? Are the assessment items appropriate for their course? These types of things are evaluated before technology. And, our research has shown that instructors are more accepting of technology deficiencies than content deficiencies. So without content, there isn’t much value. For technology, content is the foot in the door.
- De-coupling technology has risks In surveying instructors, we’ve learned that platforms — especially courseware — without content, won’t be adopted. Instructors are looking for platforms that have high quality content already. This doesn’t mean that instructors don’t want some level of customization, but it does mean that the tool must have high-quality content out of the box to be considered. A good example of this lesson learned is through our college ecosystem partners. OpenStax now has almost 50 partners that use our content to power their tech products and services. We call them ecosystem partners. They’ve recognized that technology without content is hard to drive forward in the market, which is why the number of our ecosystem partners has increased dramatically in just a few years. We started with 2-3, and now we’re near 50 partners. ----- Meeting Notes (6/16/16 11:07) ----- high quality content --
- Student needs should drive technology development. Instructors acknowledge that students have learning preferences and therefore one size fits all content delivery models are now less valuable. Instructors and students now expect their unique learning preferences be accommodated in an ed tech platform. The challenge then becomes how do we bring in technology that accommodates learning preferences? How do we introduce review content like spaced practice in a way that challenges students without frustrating them? What’s the right level of interactivity? How do we work in reading passages to flow within the courseware? Technology works only when it’s driven by content and student needs. We don’t have all the answers to these things yet, but it’s something that OpenStax thinks the edtech community needs to investigate.
- If when developing courseware you see the instructor (and student, for that matter) workflow is being optional, you will likely have a hard time getting widespread usage of the courseware.
- Tech designed within workflow can flourish Instructors and students have limited time. They have lives! If you asked an educator why they choose this career, they won’t tell you that they choose it so that some ed tech company could show them the best way to teach, or so each year they could ride the wave of the latest ed tech. Similarly with college students, for example, if you ask them why they’re in school it’s because they have goals and dreams that don’t include becoming a master of each new piece of technology thrown at them by their professors. When introducing technology, you have to meet instructors and students on their terms. There is no incentive for them to change their workflow just because we said so. While this is a simple concept, we’ve found it’s crucial in getting instructor buy-in and widespread usage. Some good examples of technology that has been designed within the workflow and has had success are WebAssign, OWL, APLEA, ALEKS. They all live within the workflow differently, but they drive efficiency.
- Technology must improve efficiency through workflow Remember, efficiency doesn’t mean students learning faster. It means they have a greater return on effort. If they put in an hour on your technology, that hour should be more productive than how they would have done that without technology. For example, if a students spends an hour studying, they should be focusing that hour on the areas where their performance is weak as opposed to spending equal amounts of time on every section. Also, technology should decrease mundane tasks for instructors. More time to do meaningful work, like working with students, is the goal. Tasks like grading, analyzing large quantities of data, sourcing content, etc. are all things that can be either accomplished or made significantly easier and more efficient with technology. As organizations developing technology, we need to ask ourselves — how are we freeing time for instructors and how are we improving the quality of time spent on course work for students?
- Technology must incorporate reflection and reaction. Technology must create a path forward for the instructor and student. Instructors need to see how their individual students and their class as a whole is performing so they can focus their teaching efforts. Students need this path as well to focus their studying efforts. Within the existing instructor and student workflow, we need to ask ‘how does our technology allow both instructors and students to examine their progress and to move forward?
- Here is the OpenStax Tutor screen shot again. This is one way that our tool has addressed this. This shows how students performed and given them areas on where they need to work
- Preserve anonymity and academic freedom For ed tech providers, student data must be anonymous and that data shouldn’t be used for commercial purposes. For example, if a student is performing poorly in a certain area, that data shouldn’t be used to market add-on paid services like tutoring. On campus, administrators need to establish policies so student data isn’t used punitively against faculty or to encroach on academic freedom. However, administration should use data to help students. For example, if you examined week three of a developmental math course, and saw that the majority of students who got a particular question or set of questions wrong all failed the course, you could use that data to do targeted intervention and help those students through that problem. This type of data could also be used to empower instructors to find solutions to student performance problems.
- Improve the science of learning …and accept that no one organization will have the answers to everything. OpenStax advocates open data policies. If everyone hoarded data, it will be more difficult to see positive and null effects of technology. To improve the science of learning and make real advances in technology, we must accept that not one organization will have all the answers and all the good ideas — it’s impossible. But if we have industry-wide standards that allow the safe sharing of data, it will be much easier to learn from each other and improve the effectiveness of technology across the industry.
- Knowledge in the right hands It is not standard practice (yet) for ed tech providers to share their findings with institutions. So institutions need to answer this question: do you want ed tech companies to know more about your students than you do? OpenStax advocates that this data be shared with institutions so that institutions can help students improve and grow.
- It's not about price, it's about value. The last point of this lesson is that it’s not about price, it’s about educational value. Right now, there is inadequate efficacy research on the impact of ed tech platforms designed to improve learning outcomes. When there isn’t adequate efficacy research, price for a product is de-coupled from value. If you don’t really know how effective something will be, how can you really set the price? OpenStax strongly recommends rigorous research trial periods that can help determine effectiveness. When you have rigorous research and make iterative improvements to the tool based on that research, you can price the product based on its value to the end user. Pricing based on operational costs is not fair, however pricing based on value is fair to the user, and you’re paving the way for a long-term relationship with them and long-term success of the technology.