The Second IEEE International Contest on Software Testing

Over the past decades, we have built software to improve our efficiency, reliability, and safety in production, business, daily life, etc. These goals, however, cannot be accomplished if the software is not properly tested. Some universities provide classes to teach students the fundamental knowledge and techniques of software testing. These classes often ignore industry practices and can hardly offer real-world testing experiences to students. To address this, we partner with industry sponsors and host a software testing contest along with an appropriate tutorial. Through the contest and tutorial, we bring real-world testing and tool experience to the participants and provide excellent opportunities for them to practice their learned testing techniques to overcome industry testing challenges.

The Second IEEE International Contest on Software Testing (ISTC) will be held in conjunction with QRS 2018 (the 18th International Conference on Software Quality, Reliability and Security) on July 17 (Tuesday) at Instituto Superior Técnico, University of Lisbon, Portugal.

We cordially invite you to participate in the contest and win a CASH prize!

Please click here to download the pdf version of the contest flyer.


All interested contestants must send an email to Mr. Linghuan Hu before July 12. The subject of the email should be “IEEE 2018 ISTC -- Contestant”.

The content should include the first name, last name, and affiliation. There is no registration fee.

Format & Evaluation

All applications have to be submitted before the end of July 12 (CDT). All contestants must bring their own laptops. The contest has two parts:

  • The first part is a 2-hour tutorial (10:00 - 12:00). During this tutorial, we will help contestants properly configure their laptops, and explain how to use platform. Each contestant is required to have basic knowledge about Java programming and JUnit.

  • The second part is the contest (14:00 - 17:00) where contestants are required to design test cases based on the given specifications to test some Java programs.

The ranking of all the contestants is based on the quality of test cases they generate, which is measured in terms of a combination of the statement and branch coverage, and the mutation score achieved. If test cases from multiple contestants achieve the same coverage, contestants with fewer test cases have a higher ranking than those with more test cases.

Platform & Configuration

Every contestant must bring his/her own laptop and have the following software installed:

  • Windows 7 or later versions
  • Java JDK 1.8 (cannot be other JDK versions)
  • Eclipse Luna
  • Maven
  • [more...]

If you have trouble installing the above software, we will help you set up the environment during the tutorial session before the official contest.


The first place receives a cash prize of $400 USD, followed by the 2nd and the 3rd places for $300 USD and $200 USD, respectively. The 4th to the 10th place each receives $100 USD.

Contest Venue

The Second International Contest on Software Testing will be held at the Instituto Superior Técnico, University of Lisbon, Portugal.

Important Dates

    July 12, 2018
  Application deadline
    July 17, 2018
  Tutorial & Contest
    July 18, 2018
  Award Ceremony

General Inquiry

For any inquiry related to this contest, please contact Professor W. Eric Wong or Mr. Linghuan Hu.