PhD Defense: A Framework for Testing Concurrent Programs

Rice University
Department of Computer Science

Presents

Mathias Ricken

Ph.D. Thesis Defense

Title: A Framework for Testing Concurrent Programs

Date: January 10, 2011
Time: 9 AM
Location: Duncan Hall 3076

This study proposes a new framework that can effectively apply unit testing to concurrent programs, which are difficult to develop and debug. Test-driven development, a practice enabling developers to detect bugs early by incorporating unit testing into the development process, has become wide-spread, but it has only been effective for programs with a single thread of control. The order of operations in different threads is essentially non-deterministic, making it more complicated to reason about program properties in concurrent programs than in single-threaded programs. Because hardware and operating systems influence the order in which operations in different threads are executed, debugging is problematic since a problem often cannot be reproduced on other machines. Multi-core processors, which have replaced older single-core designs, have exacerbated these problems because they demand the use of concurrency if programs are to benefit from new processors.

The existing tools for unit testing programs are either flawed or too costly. JUnit, for instance, assumes that programs are single-threaded and therefore does not work for concurrent programs; ConTest and rstest predate the revised Java memory model and make incorrect assumptions about the operations that affect synchronization. Approaches such as model checking or comprehensive schedule-based execution are too costly to be used frequently. All of these problems prevent software developers from adopting the current tools on a large scale.

The proposed framework (i) improves JUnit to recognize errors in all threads, a necessary development without which all other improvements are futile, (ii) places some restrictions on the programs to facilitate automatic testing, (iii) provides tools that reduce programmer mistakes, and (iv) re-runs the unit tests with randomized schedules to simulate the execution under different conditions and on different machines, increasing the probability that errors are detected.

The improvements and restrictions, shown not to seriously impede programmers, reliably detect problems that the original JUnit missed. The execution with randomized schedules reveals problems that rarely occur under normal conditions.

With an effective testing tool for concurrent programs, developers can test programs more reliably and decrease the number of errors in spite of the proliferation of concurrency demanded by modern processors.

PhD Proposal: A Framework for Testing Concurrent Programs

Rice University
Department of Computer Science

Presents

Mathias Ricken

Ph.D. Thesis Proposal

Title: A Framework for Testing Concurrent Programs

Date: December 2, 2010
Time: 9 AM
Location: Duncan Hall 3076

This study proposes a new framework that can effectively apply unit testing to concurrent programs, which are difficult to develop and debug. Test-driven development, a practice enabling developers to detect bugs early by incorporating unit testing into the development process, has become wide-spread, but it has only been effective for programs with a single thread of control. The order of operations in different threads is essentially non-deterministic, making it more complicated to reason about program properties in concurrent programs than in single-threaded programs. Because hardware and operating systems influence the order in which operations in different threads are executed, debugging is problematic since a problem often cannot be reproduced on other machines. Multi-core processors, which have replaced older single-core designs, have exacerbated these problems because they demand the use of concurrency if programs are to benefit from new processors.

The existing tools for unit testing programs are either flawed or too costly. JUnit, for instance, assumes that programs are single-threaded and therefore does not work for concurrent programs; ConTest and rstest predate the revised Java memory model and make incorrect assumptions about the operations that affect synchronization. Approaches such as model checking or comprehensive schedule-based execution are too costly to be used frequently. All of these problems prevent software developers from adopting the current tools on a large scale.

The proposed framework (i) improves JUnit to recognize errors in all threads, a necessary development without which all other improvements are futile, (ii) places some restrictions on the programs to facilitate automatic testing, (iii) provides tools that reduce programmer mistakes, and (iv) re-runs the unit tests with randomized schedules to simulate the execution under different conditions and on different machines, increasing the probability that errors are detected.

The improvements and restrictions, shown not to seriously impede programmers, reliably detect problems that the original JUnit missed. The execution with randomized schedules reveals problems that rarely occur under normal conditions.

With an effective testing tool for concurrent programs, developers can test programs more reliably and decrease the number of errors in spite of the proliferation of concurrency demanded by modern processors.

ConcJUnit: Unit Testing for Concurrent Programs

In test-driven development, tests are written for each program unit before the code is written, ensuring that the code has a comprehensive unit testing harness. Unfortunately, unit testing is much less effective for concurrent programs than for conventional sequential programs, partly because extant unit testing frameworks provide little help in addressing the challenges of testing concurrent code. In this paper, we present ConcJUnit, an extension of the popular unit testing framework JUnit that simplifies the task of writing tests for concurrent programs by handling uncaught exceptions and failed assertions in all threads, and by detecting child threads that were not forced to terminate before the main thread ends.

ConcJUnit: Unit Testing for Concurrent Programs Practice Talk

Practice Talk for PPPJ 2009 Presentation

In test-driven development, tests are written for each program unit before the code is written, ensuring that the code has a comprehensive unit testing harness. Unfortunately, unit testing is much less effective for concurrent programs than for conventional sequential programs, partly because extant unit testing frameworks provide little help in addressing the challenges of testing concurrent code. In this paper, we present ConcJUnit, an extension of the popular unit testing framework JUnit that simplifies the task of writing tests for concurrent programs by handling uncaught exceptions and failed assertions in all threads, and by detecting child threads that were not forced to terminate before the main thread ends.

Java Annotations on Types and Expressions

As defined in Java 1.5, Java annotations can only be attached to very few targets: Classes, methods, method parameters, fields, and other annotations. The Java community intends to extend the list of targets and has issued Java Specification Request (JSR) 308, "Annotations on Types". The current proposal for JSR 308 is considering allowing annotations on statements, but not on expressions.

In this talk, I discuss how the changes proposed by JSR 308 and its Checker Framework allow programmers to enhance Java's type system and provide additional static guarantees. Additionally, I demonstrate that the same mechanism proposed for annotations on statements can be used to describe multi-stage programs in Java.

Java Annotations for Invariant Specification

Java annotations allow programmers to attach metadata to programs. During normal execution, the annotations are ignored, but in contrast to comments in the source code, annotations can be manipulated programmatically at compile- or run-time.

In this talk, I will provide a brief introduction to Java annotations and show how limited their use currently is. I will then describe a framework for specifying program invariants using Java annotations, and how extending Java to support subtyping for annotations allows programmers to write the invariants in a succinct manner.

Testing Concurrent Programs

A talk directed mostly at undergraduate students.

Programming examples from the presentation:
http://www.concutest.org/download/RCSC-20071004/listings

A Framework for Testing Concurrent Programs

Incremental, test-driven development is sweeping the software industry, elevating testing from an ancillary activity to an integral part of the programming process. Unfortunately, in our recent experience developing production programs in Java, unit testing has only proven effective in assuring the reliability of code with a single thread of control; it is much less effective in concurrent programs. To facilitate the development of concurrent programs, we are developing:

      1. An extension of the JUnit framework that actively supports the developer by treating tests that could silently ignore failures in auxiliary threads as test errors;
      2. A lightweight Java annotation language that can be used to specify and check the threading invariants of both existing and new code;
      3. A testing framework that can record and analyze the schedules of unit tests, detect deadlocks, and run the tests using modified schedules, increasing the likelihood that concurrency problems are discovered.

www.concutest.org

Design Patterns for Parsing

We provide a systematic transformation of an LL(1) grammar to an object model that consists of
• an object structure representing the non-terminal symbols and their corresponding grammar production rules,
• a union of classes representing the terminal symbols (tokens).

We present a variant form of the visitor pattern and apply it to the above union of token classes to model a predictive recursive descent parser on the given grammar. Parsing a non-terminal is represented by a visitor to the tokens. For non-terminals that have more than one production rule, the corresponding visitors are chained together according to the chain of responsibility pattern in order to be processed correctly by a valid token. The abstract factory pattern, where each concrete factory corresponds to a non-terminal symbol, is used to manufacture appropriate parsing visitors.
Our object-oriented formulation for predictive recursive descent parsing eliminates the traditional construction of the predictive parsing table and yields a parser that is declarative and has minimal conditionals. It not only serves to teach standard techniques in parsing but also as a non-trivial exercise of object modeling for objects-first introductory courses.

Nifty Assignment: Marine Biology Simulation

The Marine Biology Simulation is designed as a final project in an objects-first CS2 course.  It provides an entertaining setting that serves as compelling example of the powers of object-oriented design and programming.

TeachJava: Rice Marine Biology Simulation - Milestone 2

TeachJava: Rice Marine Biology Simulation - Milestone 1

TeachJava: Rice Marine Biology Simulation - Introduction

Design Patterns for Marine Biology Simulation

We specify and implement a GUI application that simulates marine biological systems by making extensive use of object-oriented design patterns.

The key design patterns are model-view-control, observer/observable, visitor, command, factory method and decorator.  These design patterns help delineate the roles and responsibilities of the objects in the system, establish loose coupling between objects and arrange for the objects to communicate and cooperate with one another at the highest level of abstraction.  The result is an application that exhibits minimal control flow, yet is powerful, robust, flexible and easy to maintain.

Our work entails a non-trivial redesign of the current AP Computer Science Marine Biology Simulation case study and may serve as a case study for an introductory “object-first” curriculum.