Below are the details of the keynote speakers confirmed for EASE2007:
- Khaled El-Emam
- Barbara Kitchenham
- Software Engineering or Computer Science articles from January 2004 to February 2007
- Were either systematic literature reviews (SLR) with reported procedures for inclusion/exclusion, searching,
data extraction and data aggregation; or Evidence-based guidelines (EBG); or Meta-analyses (MA).
- Was the search likely to find all relevant articles?
- Are the reviewers’ inclusion and exclusion criteria defined and appropriate?
- Did the reviewers assess the quality/validity of the included studies?
- Were the basic data/studies adequately described?
"The Development, Reporting, and Use of Evidence: Lessons from Medicine"
Dr. El Emam is a Senior Scientist at the Children's Hospital of Eastern Ontario Research Institute, where he is leading the eHealth research program. He is also an Associate Professor at the University of Ottawa, Faculty of Medicine, and a Canada Research Chair in Electronic Health Information. In addition, Khaled is the Chief Scientist at TrialStat Corporation and a Senior Consultant with Cutter Consortium's Agile Software Development & Project Management Practice. Previously Khaled was a senior research officer at the National Research Council of Canada, where he was the technical lead of the Software Quality Laboratory, and prior to that he was head of the Quantitative Methods Group at the Fraunhofer Institute for Experimental Software Engineering in Kaiserslautern, Germany.
In 2003 and 2004, Khaled was ranked as the top systems and software engineering scholar worldwide by the Journal of Systems and Software based on his research on measurement and quality evaluation and improvement, and ranked second in 2002 and 2005. Currently, he is a visiting professor at the Center for Global eHealth Innovation at the University of Toronto (University Health Network) and at the School of Business at Korea University in Seoul. He holds a Ph.D. from the Department of Electrical and Electronics Engineering, King's College, at the University of London (UK).
"The Current State of Evidence-Based Software Engineering"
Recently, Dybå, Kitchenham, and Jørgensen wrote a series of articles proposing the adoption of the
evidence-based paradigm in Software Engineering research and practice.
The evidence-based paradigm relies on the use of systematic literature reviews to
provide a rigorous method for identifying, analyzing, and synthesizing research on a
specific topic or research question.
It is also concerned with providing best practice guidelines for practitioners.
Aims: The goal of this presentation is to describe the extent to which the evidence-based paradigm has been adopted in Software Engineering since 2004.
Method: We performed a tertiary review to determine the current extent of adoption of the evidence-based paradigm. A tertiary review is a review of systematic literature reviews (which are themselves secondary reviews). We manually searched articles in IEEE Trans on SE, IEEE Software, JSS, IST, CACM, ACM Surveys, ISESE05, ICSE05 and ICSE06 and identified all articles that conformed to the following inclusion criteria:
We excluded papers that were subjective reviews (with no defined question, minimal search criteria, no data extraction process, and no formal aggregation process).
Papers were classified by type (SLR, EBG, MA), by subject area, scope (research trends or specific research question). SLR papers were also evaluated for quality against four criteria:
Results: We found 23 papers. There was one meta-analysis, 20 systematic literature reviews (SLRs), 4 evidence-based guidelines (of which two included SLRs), and two papers that positioned themselves as Evidence-based software engineering papers (of which both included SLRs). It is interesting to note that we found no SE or CS SLRs in ACM Surveys.
Nine of the 20 SLRs addressed research trends rather than specific research questions. In terms of topic area, nine articles related to cost estimation, four related to research trends in SE experiments and three related to test methods. In the area of cost estimation, researchers are addressing well-defined research questions and obtaining useful results. This is also an area where researchers have proposed evidence-based guidelines for practitioners.
EBSE is strongly supported by the Simula Research Laboratory (11 out of 23 papers) and European researchers (17 papers). There are relatively few North American researchers (4 papers).
In terms of quality, very few SLRs provide a quality assessment of their primary studies (only 3 fully and 4 partly).
Conclusions: Currently, researchers in Europe (particularly those interested in cost estimation) have been in the forefront of Evidence-based Software Engineering but it seems to have limited take-up in North America.
The large number of papers addressing research trends is rather disappointing since research trends are primarily of interest to academics rather than practitioners. However, five of these papers suggest ways in which software engineering experiments can be improved. In the longer term this should have a benefit for practitioners by improving the quality of software engineering research and hence increasing the reliability of software engineering evidence.
Another particular concern is the lack of quality assessment of primary studies, since this is a critical factor in SLR methodology. However, this is partly due to the relatively large number of SLRs that consider research trends rather than specific research questions.