Topics and Contributions of ICER 2017 Research
Session 1: Novice Programmers
- Comprehension First: Evaluating a Novel Pedagogy and Tutoring System for Program Tracing in CS1
- Problem: How to teach program comprehension skills? Write first or understand semantics first?
- Contribution: Pedagogy for teaching program semantics before writing code, tutor system tool, and experimental comparison that showed 60% higher learning gains.
- Sometimes, Rainfall Accumulates: Talk-Alouds with Novice Functional Programmers
- Problem: How do novice functional programmers solve the Rainfall problem?
- Contribution: Narratives of four students’ attempts at solving the Rainfall problem in a functional language (spoiler: they didn’t do well).
- Using Learners’ Self-Explanations of Subgoals to Guide Initial Problem Solving in App Inventor
- Problem: Subgoal labels require a lot of time to come up with.
- Contribution: A study to examine whether student’s self-explanation of subgoals can improve student performance when provided with their own explanations for practice problems. The results show that scaffolding initial problem solving with learners’ explanations of the problem solving process can lead to better problem solving performance than scaffolding from experts if the learners construct explanations with adequate support.
Session 2: Student Perceptions, Conceptions, Reactions
- Students’ Emotional Reactions to Programming Projects in Introduction to Programming: Measurement Approach and Influence on Learning Outcomes
- Problem: Emotional responses to assignments can lead to self-efficacy issues and adaptive/maladaptive behaviors.
- Contribution: First study of emotional reactions and student learning outcomes. Reports the results of a pilot study on a emotion survey with a large undergraduate intro course.
- The ‘Art’ of Programming: Exploring Student Conceptions of Programming through the Use of Drawing Methodology
- Problem: Little is known about students emotional reactions to their learning experiences.
- Contribution: Asked 396 students to draw a picture of “what does programming mean to you?” and analyzed them.
- Social Perceptions in Computer Science and Implications for Diverse Students
- Problem: Certain demographics are underrepresented in CS regardless of initiatives to change this.
- Contribution: Analyzed surveys from thousands of students, parents, principals, and superintendents to understand the social beliefs regarding students’ fit and ability.
Session 3: When Things Go Wrong
- Taking Advantage of Scale by Analyzing Frequent Constructed-Response, Code Tracing Wrong Answers
- Problem: How many constructed-response question answers does it take cover some space of incorrect responses by students? (The incorrect responses can be used to reveal student misconceptions.) How to choose answers to inspect to get the most information?
- Contribution: Findings of a qualitative study of test responses by ~4k students over 3 large courses. In particular, a large proportion of the students make the same mistakes, and looking at the most frequently occurring mistakes (top 5%) covers a large proportion (~60%) of all the mistakes students made. Additionally, some types of misconceptions observed were consistent with prior studies, whereas others were new (e.g., syntax errors, sloppy reading/writing).
- Investigating Static Analysis Errors in Student Java Programs
- Problem: What errors do student programmers make that can be detected by off-the-shelf static analysis tools? How do such errors vary across students with different levels of experience.
- Contribution: Formatting and Javadoc issues are the most common. Students who make lots of static analysis detectable errors are more likely to produce incorrect programs (even if they later fix the errors). The types of errors that students make most frequently are very common across both computer science majors and non-majors, and across experience levels.
- On Novices’ Interaction with Compiler Error Messages: A Human Factors Approach
- Problem: Why have past experiments on enhanced compiler error messages in automated assessment tools for programming assignments produced inconsistent results (sometimes they help and sometimes not)?
- Contribution: Findings of a mixed methods study comparing Athene messages to normal ones. Although students were qualitatively observed reading and understanding Athene messages, the messages had not measurable effect on student learning.
- Theorem Provers as a Learning Tool in Theory of Computation
- Problem: investigating whether an interactive theorem prover like Coq can be used to help undergraduate computer science (CS) students learn mathematical proving within the field of theory of computation
- Contribution: Set within an educational design research approach and building on cognitive apprenticeship and socio cultures cognition theories, we have collected empirical, mainly qualitative observational data focusing on students’ activities with Coq in an introductory course specifically created for that matter. Our results strengthen the assumption that a theorem prover like Coq, indeed, can be beneficial in mediating undergraduate students’ activities in learning formal proofing.
- RoboBUG: A Serious Game for Learning Debugging Techniques
- Problem: can game-based learning be applied to debugging education; build and evaluate RoboBUG software
- Contribution: evaluation of RoboBUG showed that the game helps students to achieve learning outcomes, but has a non-statistically significant impact on enjoyment (positive and negative affect). In addition, the game seemed to be most effective at aiding students who were not initially skilled at debugging
- Students and Teachers Use An Online AP CS Principles EBook Differently: Teacher Behavior Consistent with Expert Learners
- Problem: How does teacher use of the eBooks differ from student use? In what ways do they learn from the eBook differently?
- Contribution: quantitative log file data from use of the student and teacher eBooks showed students interacted more with the eBook than teachers, on average; they theorize that students did more activities in ways that did not apply appropriate learning strategies
Session 5: Social Interaction and Support
- Describing Elementary Students’ Interactions in K-5 Puzzle-based Computer Science Environments using the Collaborative Computing Observation Instrument (C-COI)
- Problem: How do students behave during CS instruction (K-12 education)? How do students learn CS? How to provide positive computing experiences? How do students interact with each other?
- Contribution: Findings of an observational study of nine students recorded using the Collaborative Computing Observation Instrument (C-COI) as they engaged in CS activities within Code.org’s Code Studio. The study confirmed three predominant types of collaborative interactions: collaborative problem solving, excitement and accomplishment related to CS activities, and general socialization.
- Understanding Student Collaboration in Interdisciplinary Computing Activities
- Problem: When working in groups, how do students cope with challenges of skill balance and how do they negotiate decision making.
- Contribution: Findings of an observational study of two pairs of students (one pair successful and one not) working on statistics-coding problems in R. The study characterized several patterns of behavior (dynamics) between the successful and unsuccessful pairs.
- Factors Influencing Students’ Help-Seeking Behavior while Programming with Human and Computer Tutors
- Problem: How do human teachers and smart tutors interactions with students currently differ (with an eye on improving smart tutors)?
- Contribution: Findings of a qualitative analysis of 15 students interviews where they reflect on solving two programming problems with human and computer help, respectively. The study identified 3 categories of factors that directly and indirectly impact students’ help-seeking behavior: Inputs, Student Mindset, and Attributes of Help. Specifically, a human tutor seemed more trustworthy, perceptive and interpretable, while a computer tutor seemed more accessible and less threatening to a students sense of independence.
Session 6: Teacher Conceptions and Experiences
- Conceptions and Misconceptions about Computational Thinking among Italian Primary School Teachers
- Problem: “Computational thinking” is the term often used to denote the conceptual core of computer science but also the lack of a widely accepted definition.
- Contribution: They investigated the Italian primary school teachers’ conceptions about computational thinking by analyzing the results of a survey (N=972) conducted in the context of “Programma il Futuro” project. Teachers have been asked to provide a definition of computational thinking and to answer three additional related closed-ended questions.
- Folk Pedagogy: Nobody Doesn’t Like Active Learning
- Problem: How active learning is described in the computing-education literature?
- Contribution: They report on an investigation into the folk pedagogy of computing education, specifically those beliefs surrounding active learning. The data include the results of a faculty survey and a sample of computing education papers about active learning. Finally, propose some dimensions along which distinctions among techniques could usefully be made.
- Understanding the “Teacher Experience” in Primary and Secondary CS Professional Development
- Problem: What are the affective patterns and cognitive themes that arise throughout an effective Computer Science Education Professional Development(K-12 grade level) program?
- Contribution: They investigates the affective experiences of a cohort of ten in-service teachers (nine middle school and one high school) as they participate in an intensive, multifaceted summer CSE PD program at a Midwestern metropolitan university in North America. Teachers’ experiences were documented in their written daily journals, which were analyzed qualitatively using thematic and sentiment analysis techniques.
Session 7: External representations for understanding & Learning Trajectories
- Using Tracing and Sketching to Solve Programming Problems: Replicating and Extending an Analysis of What Students Draw
- Problem: Does sketching code traces decrease program-understanding problems involving loops, arrays, and conditionals?
- Contribution: Findings of a study that validated previous study findings (sketching is successful technique to distribute cognition and manage cognitive load; no sketching had lower success). Also found that completeness of sketches was more predictive of correctness than strategy used to create sketches.
- The Affordances and Constraints of Diagrams on Students’ Reasoning about State Machines
- Problem: What barriers do students face in translating finite state machine diagrams into synchronous, sequential logic circuits?
- Contribution: Findings of a qualitative research study interviewed 24 students as they transformed FSMs into logic circuits. The study found that that students’ ability to use conceptually appropriate information varies based on the task they are performing and the representational tools they are provided, and that a primary challenge may be that students lack the conceptual distinctions necessary to navigate each new representation of a state machine.
- K-8 Learning Trajectories Derived from Research Literature: Sequence, Repetition, Conditionals
- Problem: How to create evidence-based K-8 computing curricula—what topics should be addressed at each grade level, at what depth, and in what order?
- Contribution: Findings of an in-depth review of over 100 scholarly articles in CSEd research. The review identified two study attributes common in current research literature that limit the studies’ usefulness in creating full, empirically-supported learning trajectories. The paper also contributes a decision-making process, grounded in education theory, to connect related learning goals.
Session 8: Students’ Use of Time in Programming
- Quantifying Incremental Development Practices and Their Relationship to Procrastination
- Problem: To determine whether proposed measures of student behaviors such as incremental development and procrastination during program development process are significantly related to the correctness of final solutions, the time when work is completed, or the total time spent working on a solution
- Contribution: Most significant effect of incremental development was effective time management practices— working on a project early and often and avoiding the pitfalls of procrastination. Incremental test writing, or incremental self-checking of work using interactive program launches or execution of software tests were not significant;
- Comparison of Time Metrics in Programming
- Problem: Explored task indicators through the analysis of a multi-source dataset that contained information about use of a programming environment, use of the learning material; self-reported data on the amount of time invested in the course, per-assignment, perceptions on workload, educational value, and difficulty
- Contribution: Found traditionally used metrics from the same data source tend to form clusters that are highly correlated with each other but correlate poorly with metrics from other data sources. Suggests researchers should utilize multiple data sources to gain a more accurate picture of students’ learning.
Session 9: Validating Assessments & Dual Modality Teaching
- Principled Assessment of Student Learning in High School Computer Science
- Problem: Measuring student learning on hard-to-measure CS outcomes (K-12 focus).
- Contribution: A two-year validation study on end-of-unit and cumulative assessments designed by the authors using an Evidence-Centered Design process. The study found that reliability was moderate to high for each of the unit assessments; the assessment tasks within each assessment are well aligned with each other and with the targeted learning goals; and average scores were in the 60 to 70 percent range.
- An Instrument to Assess Self-Efficacy in Introductory Algorithms Courses
- Problem: Assessing self-efficacy (introductory algorithms course focus).
- Contribution: An instrument (based on Ramalingam and Wiedenbeck) and a multi-institutional evaluation. The evaluation found the instrument measures to be consistent with self-efficacy theory (and thus, suggest construct validity).
- Dual Modality Code Explanations for Novices: Unexpected Results
- Problem: Do auditory explanations of code result in improved learning performance over written explanations?
- Contribution: Experiment comparing text-only explanations, auditory-only explanations, and both text and auditory explanations. The study found no clear differences between these treatments.
Session 10: Outside the Conventional Classroom
- Computing Mentorship in a Software Boomtown: Relationships to Adolescent Interest and Beliefs
- Problem: Test the effects of mentorship on learning computing.
- Contribution: Two studies on mentorship. The first study surveyed 57 students (14-18 years old) in a coding course (week long?), and found that mentoring was the most significant factor in students’ interest in computing. In the second study, the lead author provided semi-structured mentorship to 11 students in the context of a course, and the students interest in computing generally increased following the course.
- Barriers Faced by Coding Bootcamp Students
- Problem: Learn what barriers coding bootcamp students face.
- Contribution: Interview study of 26 coding bootcamp students. It found that bootcamps can be part of an alternate path into the software industry and they provided a second chance for those who missed computing education opportunities earlier, particularly for women. However, students entering the industry through bootcamps often spent significant time, money and effort before, during, and after bootcamps. Career change could take a year or more (for 3-6 month boot camp).
- Hack.edu: Examining How College Hackathons Are Perceived By Student Attendees and Non-Attendees
- Problem: Understand student perceptions of college hackathons, specifically: (1) Why are students motivated to attend hackathons? (2) What kind of learning environment do these events provide? (3) What factors discourage students from attending?
- Contribution: Mixed methods study of college hackathon event. Findings included that students were motivated to attend for both social and technical reasons, that the format generated excitement and focus, and that learning occurred incidentally, opportunistically, and from peers. Those who chose not to attend or had negative experiences cited factors such as physical discomfort, lack of substance, an overly competitive climate, an unwelcoming culture, and fears of not having enough prior experience.