Scientific Reasoning Rubric

Overview

The American Association for Higher Education and Accreditation (AAHEA) describes assessment as an ongoing process aimed at understanding and improving student learning. The principal goal of this assessment process is to provide faculty with data that can be used to guide planning and implementation of appropriate curricular and instructional changes to support and improve student learning (AAHEA, 2013).

Scientific Reasoning may be divided into reductionist science approaches and integrative science approaches or systems thinking. Reductionist science aims to create study and control groups that are as similar as possible except for the factor under investigation. Reductionist sciences begin with hypothesis generation which may result from inductive or deductive logic. Reductionist sciences aim at explanation or establishing the existence of cause and effect relationships including the efficacy of an intervention. Integrative sciences often build upon reductionist sciences. They draw from multiple disciplines incorporating multiple influences or determinants of outcomes; look for interactions between factors; and use evidence-based approaches to understand and propose strategies for addressing complex problems. (A A C & U STIRS project aacu.org/stirs/framework).

SFC GELO Scientific Reasoning: Understand scientific 1) concepts and 2) reasoning and 3) analyze and 4) interpret various types of data.

The extent to which students engage in Scientific Reasoning can be assessed using the criteria in the following rubric modeled after that reported by Puncochar and Klett (2013). Construction of their rubric followed assessment methods and practices recommended by Sundre et al. (2009) in a NSF funded project (DUE 0618599) to further the development of collegiate scientific and quantitative reasoning assessment tools and procedures. The rubric, devised by a team of professors, incorporates competency guidelines from the American Association for the Advancement of Science (AAAS), the National Research Council (NRC), and the National Science Teachers Association (NSTA). The four learning outcome criteria in the paragraphs below mirror the college’s bulletin description of the core competencies expected of students associated with the SR GELO.

The following table lists the three learning outcomes that students should demonstrate proficiency in, with the level of proficiency commensurate with the level of the course. Also included are suggestions that instructors may wish to consider using to evaluate student proficiency in those areas. It is important to note that not all of the suggestions need be implemented for a successful evaluation, and this list of suggestions is not exhaustive; the instructor may find other ways to complete the evaluation.

Student learning outcome Assessment suggestions
Demonstrate an understanding of key science concepts associated with the course
  1. Vocabulary definitions
  2. Identification and Classification (use of a key or scheme of analysis, etc.)
  3. Understanding of discipline related fundamental theories
  4. Application of a model to new situations
  5. Interpretations based on fundamental theories
The ability to engage in a scientific process with the proper use of the scientific method (steps of the scientific method, induction, deduction, hypothesis testing, data analysis, mathematic tools, etc.)
  1. Distinguish between inductive and deductive inferences
  2. Recognize and develop testable (falsifiable) hypotheses
  3. Use of a hypothesis to develop testable predictions
  4. Understanding of qualitative vs. quantitative data
  5. Able to distinguish dependent, independent, and controlled variables
  6. Appropriately records data
  7. Able to recognize relationships between variables
  8. Interpretation of data in both tabular and graphical forms
  9. Manipulation of relationships to solve problems (calculus, algebra, statistical or proportional arguments)
  10. Able to perform error analysis and order-of-magnitude estimates
  11. Demonstrate an understanding of fact vs. theory
  12. Demonstrate an understanding cause/effect vs. correlation
  13. Understanding of randomization and study vs. controlled groups
  14. Distinguish between experimental vs. observation study designs
The ability to interpret results and communicate their knowledge in both oral and written form appropriate to the discipline
  1. Presentation of data in both tabular and graphical form
  2. Proper use of scientific terminology and language
  3. Proper technical writing format
  4. Clear presentation of evidence to support conclusions
  5. Class discussion assignments
  6. Projects (papers and/or presentations)
  7. Scientific posters
  8. Lab reports (individual reports, lab notebooks)

Puncochar and Klett (2013) report that laboratory reports, term papers, essays, and short answer problem-based items proved to be excellent articles for assessing science understanding and the use of scientific concepts, recognition and use of scientific reasoning methods, understanding general scientific principles, and the use of mathematics in scientific reasoning and / or problem solving.

The following rubric lists how to gauge students’ mastery of the three student learning outcomes:

Assessment Rubric for Scientific Reasoning

Definition: Understand scientific concepts and reasoning and analyze and interpret various types of data.

Learning Outcome Exceeds Expectations Meets Expectations Needs Improvement
Understanding of Key Science Concepts Correctly uses some science concepts to explain phenomena Provides proper descriptions or definitions of the concepts Recognizes the concepts in descriptions or definitions
Engages in Scientific Process Correctly develops (of their own design) a project using all the steps in the process. Collects, graphs, and/or interprets data and/or performs appropriate calculations Recognizes and/or correctly applies all the steps in the process. Collects, graphs, and / or interprets data and/or perform calculations. Recognizes most of the steps in the process. However, struggles with collecting, graphing and/or interpreting data and/or performing calculations
Communicate Knowledge Communicates a general understanding of concepts and writes papers, essays, and/or lab reports with minimal errors or maintains a scientific document in proper format with minimal errors Communicates verbally, graphically, or in written form a general understanding of concepts. Uses some key words or phrases but does not communicate a general understanding of concepts. Writes papers, essays, lab reports with many errors.

Notes for the GELO committee that evaluates the artifacts:

The committee members who evaluate the artifacts should check whether the artifact is capable of assessing each of the three student learning outcomes. This does not mean that the artifact must implement all of the assessment suggestions.

Secondly, the committee members may assess how finely the artifact can distinguish between different student mastery levels within each learning outcome, as per the mastery definitions within the Assessment Rubric.

References Cited
  • American Association for Higher Education and Accreditation. (2013). AAHE/AAHEA.
    Retrieved from aahea.org/aahea/

  • Puncochar and M. Klett. 2013. A Model for Outcomes Assessment of Undergraduate Science Knowledge and Inquiry Process.
    Research and Practice in Assessment. Vol.8:42-54.

  • National Research Council. (2002). Scientific Research in Education. R. I. Shavelson & L, Towne (Eds.).
    Washington, DC: National Academies. Retrieved from nap.nationalacademies.org

  • National Science Teachers Association. (2011). Positions: Official NSTA positions on a range of issues.
    Retrieved from nsta.org

  • Sundre, D. L. , Murphy, C. and Handley, M. (2009) Advancing Assessment of Scientific and Quantitative Reasoning.
    The National Numeracy Network. Retrieved from nnn-us.org