One key challenge facing the Exams Institute these days is the changing demands that instructors face in terms of assessment. In some sense, as chemistry teachers increasingly find themselves expected to participate in departmental level assessment efforts, the way they view the testing of student learning starts to change. Within such a dynamic educational environment, it’s not surprising that we hear from more people about what ACS Exams can help them do. It’s also not surprising that these newer interests in assessment point towards needs that are not always met with the traditional materials that our test development committees produce. So are there ways in which the Institute can remain true to its 80 year tradition and at the same time adjust to help provide materials that meet the new assessment demands?
We certainly hope our responses are heading in that direction. One important activity that comes to mind is that we have become more intentional in the way that we design our traditional products so that they yield more helpful results for users. A particularly important aspect of this is to empower the exam development committees to ask more questions about user needs and assessment theory in order to craft better exams. Three examples of how this works are operative in the current development portfolio of ACS Exams.
First, we are initiating the development of a new General Chemistry – Conceptual exam. While each exam committee routinely spends time at the outset of development to identify content coverage parameters, this new committee is going to take this process a step further. There have been three previous General Chemistry exams that were designed to explicitly test conceptual knowledge, so we’ve learned some things about how items accomplish their intended task. Therefore, in addition to looking at content knowledge, the new committee is going to carry out research on how to best measure conceptual understanding in light of the needs of the community. Part of this process is likely to be a survey of general chemistry instructors, so we encourage you to participate if you receive an email from us about your needs in assessing student conceptual understanding later this spring or early this summer!
Second, for the ACS High School Chemistry exam, we expect to carry out a similar but broader research effort to understand the needs of high school chemistry teachers for assessment materials. At the high school level, the complication of state demands for testing also play a role in what teachers need from an ACS Exam. Moreover, the content coverage is also being looked at closely as the Next Generation Science Standards (http://www.nextgenscience.org/) begin to influence curricular change in the US. Once again, part of the response to these changes will be to undertake survey research, so we hope if you are a high school teacher reading this newsletter that you’ll be able to help us with this survey if (or when) you receive an email inviting you. We are going to take an extra year in development for the next high school exam to make sure we are building assessment materials that teachers need.
Third, we have a new exam development committee starting to create the next exam for Inorganic Chemistry. In addition to preparing the traditional Inorganic exam, which is directed at the senior level course, this committee is also going to explore whether or not a sophomore, or introductory, Inorganic exam can be devised. Anecdotal evidence suggests more departments are teaching such a course (and an introductory course is an expected component of the curriculum for chemistry departments that meet the guidelines of the Committee of Professional Training.) Nonetheless, it’s not certain that the content coverage for this course has reached a level of consensus that will allow for the development of a nationally normed exam. Once again, we anticipate working with the committee to develop surveys to identify the nature of coverage in this course nationally. And, once again, we hope that members of DivCHED will be willing participants in this needs assessment work!
I realize that each of these activities ultimately involve participation of chemistry instructors in surveys. Rest assured that our survey development methods are designed to make sure we use the time participants give us wisely. We’ve conducted two needs assessment surveys in the past two years and both cases we tried to make sure that they would (a) take roughly 20-30 minutes to complete and (b) the results would inform both the Institute and the greater chemistry education community. We have published 3 papers from these surveys thus far, with three more in various states of readiness (from “accepted” to “in review” or ready to submit in the near future.)
In conclusion, we try to respond to the needs of the chemistry education community whenever possible. If you have ideas for new tests or products that you think the Institute should consider working on, drop me a note. My email is firstname.lastname@example.org and I would love to hear from you. We aren’t always able to develop new assessment tools, but we can always look into the possibility.