This website is about the 2010 edition of the school. Visit this year's edition of LASER.

LASER Summer School on Software Engineering

 

Empirical Software Engineering

September 5-11, 2010 - Elba Island, Italy

Victor Basili (University of Maryland)
Joshua Bloch (Google)
Barry Boehm (University of Southern California)
Natalia Juristo (Universidad Politécnica de Madrid)
Tim Menzies (West Virginia University)
Bertrand Meyer (ETH Zurich, director)
Walter F. Tichy (University Karlsruhe)

 
   
 

Lectures

Information will be filled in as soon as it is available.

Evolving software engineering knowledge from application, observation, and learning

Speaker: Victor Basili, University of Maryland

Description:

Empirical studies, using formal quantitative and qualitative analysis for supporting evidence of an assertion, has finally become recognized by the community as an important component of the software engineering discipline. One sees more and more empirical studies and experiments in the literature to confirm or reject the effectiveness of some method, technique, or tool. However, these studies are too often used as confirmatory, tried after the concept is considered complete, rather than part of the discovery process. The basis of the scientific method is to apply, observe learn, and evolve our knowledge.

I believe we need to focus more attention on informal exploratory studies that provide insights into directions software development can take, coupled when appropriate with more formal empirical studies to test out pieces of the whole that can be added to the tapestry that helps make the discipline clear. I believe that the study of the software engineering discipline is exploratory and evolutionary. It follows the scientific method, but because of its nature, real experiments are not always possible or useful.

To support this argument for informal learning I will demonstrate our experiences in the NASA Software Engineering Laboratory (SEL) and other areas of learning where we learned a great deal not just by experiments, but by trying to understand the problems, applying potential solutions, and learning where they fell short.

Short biography:
see Speakers page



Developing and Calibrating Parametric Software Engineering Estimation Models

Speaker: Barry Boehm, University of Southern California

Description:

Parametric estimation models enable project planners to estimate the resources needed (effort, cost, calendar time) to perform an upcoming project as a function of various parameters (size, personnel, platform, process, project characteristics) that have been determined to affect the level of resources needed. Similar parametric models can enable projects to estimate the results achieved by performing the projects (delivered defect density, mean time between failure, reuse savings, business value). The models can also be used for additional useful functions such as option analysis, tradeoff analysis, and return on investment analysis.

In two 2-hour sessions, the tutorial will review various parametric estimation model forms (analogy-based, unit-cost-based, activity-based, relationship-based) and criteria for evaluating candidate estimation models. It will present an 8-step process successfully used for developing and calibrating several parametric software and systems engineering estimation models in the Constructive Cost Model (COCOMO) family. It will show how these steps were applied to steer different estimation models toward more accurate capabilities (e.g., by developing hybrid models and determining Bayesian combinations of imperfect expert-judgment and data analysis results). It will also show how combinations of the models were applied to various decision situations, and interactively demonstrate one or more models.

Short biography:
see Speakers page



Integrating Empirical Software Engineering Research and Education

Speaker: Barry Boehm, University of Southern California

Description:

Most software projects are not well-instrumented and/or take several years to complete. This makes it particularly difficult to perform empirical software engineering research across several projects in a small number of years. Over the past 14 years, we have evolved an MS-level, real-client software engineering project course in which 15-20 teams per year of 6-8 people develop and independently verify and validate fully-negotiated, architected, developed, tested, and transitioned e-service software applications in 24 work weeks across two semesters. The activities and artifacts; associated effort, defects, and risks; team grading results; and client and student critiques; are fully captured in archives that can be used both for identifying course improvements and for PhD students and researchers to perform empirical research on the improvements and their effects on project results.

Over the 14 years, the empirical data and analysis has formed the partial basis for 12 PhD dissertations. These include 3 in requirements engineering, 2 each in value-based methods and COTS integration; and 1 each in agile methods, quality attribute tradeoffs, risk analysis, cost estimation, and software sizing. This session will summarize the nature of the project course, the results of several of the resulting empirical software engineering research activities, and the critical success factors for integrating the research and educational objectives.

Short biography:
see Speakers page



Replication of SE Experiments

Speaker: Natalia Juristo, Universidad Politécnica de Madrid

Description:

Replication is one of the key mechanisms of the experimental paradigm. Through replication it is possible to verify the findings observed in earlier experiments. Campbell and Stanley explain the role of replication as "The experiments we do today, if successful, will need replication and cross-validation at other times under other conditions before they can become an established part of science, before they can be theoretically interpreted with confidence".

The immaturity still plaguing the experimental paradigm applied to SE (most likely a consequence of its short lifetime) has been an obstacle to replications. At present, any change in a replication, no matter how small, can lead to alterations in the experiment outcomes that are hard (if not impossible) to interpret. Non-systematic changes to the experiment components cause the replication results to be highly variable ruling out the combination of results. Corroboration of the previous experiment findings was impossible.

This state of affairs regarding SE experiment replications has caused some disagreement with respect to how previously observed findings should be verified. Some recommend reusing the baseline experiment materials to run replications and thus assure that the replications are similar enough for their results to be comparable. Others recommend using methods and materials not used in the baseline experiment to run replications and thus confirm that the results of the replications are not dependent on local experimental conditions.

This tutorial discusses in depth the concept of replication. We will learn there are different types of replications, all of which are necessary to verify different aspects of an experiment. We will also see which are all those experiment aspects that need to be double checked. Some questions all SE experimenters performing replications wonder (How close a result need to be to consider it similar, Which replication is priority after this one, How many more replications I need to do and which ones, etc.) will be discussed.

Short biography:
see Speakers page



Empirical Software Engineering (Version 2.0) and Data Mining

Speaker: Tim Menzies, West Virginia University

Description:

As the pace of software engineering innovation increase, we turn to data mining methods to handle the pressing problem of how to reason faster about more data coming from a more diverse set of software projects. Participants will learn general principles of data mining such as general methods of evaluation, issues with bias in the learner, and how the learners explore (or "carve") out a model from software project data.

Short biography:
see Speakers page



Credible objective answers to fundamental software engineering questions

Speaker: Bertrand Meyer, ETH Zurich

Description:

The aim of empirical software engineering research is to provide an objective basis for evaluating proposed answers to software engineering questions. Too many ideas indeed -- process models, methodological approaches, quality assurance techniques, teaching methods -- are advocated through purely conceptual arguments, without good evidence that they actually improve on the state of the art.

More precisely, there is now a large body of empirical research, but much of it is disappointing to the manager or software engineer in search of clear answers: many studies wrap their results in so many caveats, qualifications and excuses -- euphemistically called "threats to validity" -- as to frustrate any attempt to draw concrete, usable conclusions. This situation is not acceptable: analyzing software products and process through the methods of natural sciences (the mission of empirical software research) is a pointless endeavor if it does not yield general lessons.

The six lectures in this LASER course examine a handful or two of open software engineering questions, drawn from various areas such as agile methods, design methodology and project management. They analyze the current empirical software engineering literature for useful answers, partial or complete, and discuss the credibility and limits of these answers. They also highlight the missing elements in today's empirical answers and define areas of further research.

Short biography:
see Speakers page



Empirical Methods in Software Engineering

Speaker: Walter F. Tichy, University Karlsruhe

Description:

What is the role of the scientific method in software engineering? What can we expect from applying it? These and related questions will be clarified, followed by an introduction to qualitative and quantitative empirical methods. A number of examples form the software engineering literature will illustrate the various methods and experimental designs. The basic statistical concepts such as correlation, significance, power, and statistical tests will be introduced. Participants will gain an understanding of what empirical methods can do and be able to evaluate the trustworthiness of empirical studies.

Short biography:
see Speakers page



How To Design a Good API and Why it Matters

Speaker: Joshua Bloch, Google

Description:

Software developers spend much of their time working with a variety of Application Programming Interfaces (APIs). Some are integral to the core platform, some provide access to widely distributed frameworks, and some are written in-house for use by a few developers. Nearly all programmers occasionally function as API designers, whether they know it or not. A well-designed API can be a great asset to the organization that wrote it and to all who use it. Good APIs increase the pleasure and productivity of the developers who use them and the quality of the software they produce. Conversely, poorly written APIs are a constant thorn in the developer's side, and can cause severe harm. Given the importance of good API design, surprisingly little has been written on the subject. In this talk, I'll attempt to help you recognize good and bad APIs and I'll offer specific suggestions for writing good APIs and coping with bad ones.

Short biography:
see Speakers page



Performance Anxiety

Speaker: Joshua Bloch, Google

Description:

Not too long ago, it was reasonably straightforward to predict the performance of computer programs. But as the performance of the underlying systems improved, their performance models got worse; in other words, our ability to predict, or even measure, the performance of a program eroded. In this talk, we explore the roots of the problem, demonstrate it in action, and discuss what we can do to cope with it.

Short biography:
see Speakers page



 

Chair of Software Engineering - ETH Zürich Last update: 02.09.2010