Assignment 1 - Testing (resit)

This assignment is individual.

To resit, you must fill out a form so that we can arrange the presentation day. Deadline for registering: 07/07/2024 (Sunday).

1. Goal

The goal of this assignment is to experiment with coverage measurement and improvement. You will select an open-source GitHub project, measure its test coverage, and enhance it by creating new test cases or improving existing ones.

2. Deliverables and Presentation

Each student will fork the source code repository of the GitHub project of their choice and perform modifications there. The URL of the forked repository must be submitted on Canvas. The README.md file of the forked project should follow the report template for this assignment, available here.

Deadline (Canvas): 11/07/2024.

Each student must present their work with slides through screen sharing and answer questions. Each student must sign in for a presentation slot. We will share a spreadsheet for that as soon as the deadline for registering for the resit passes.

Presentation day (Zoom): 12/07/2024.

In summary:

1. Forked Repository URL: Submit on Canvas.

2. Final Report (README.md): Follow the provided template.

3. Presentation: Present slides and answer questions.

3. Tasks

3.1 Project Choice

Each student can choose a project written in any programming language. The project must:

• Be hosted on GitHub;

• Have an open-source initiative-approved license (see Open Source Licenses);

• Use automated unit testing on the part written in the programming language chosen by the group;

• Have existing branch coverage (by the existing tests) less than 100%;

• Have at least two contributors outside the course;

• Have at least 1,000 lines of code (1 KLOC) in the programming language the group is using;

• Not be listed here.

After you find a project, register it here. Each student should choose a unique project, i.e., a project that is not registered in the list.

3.2 Coverage Measurement with Existing Tool

Measure the coverage of the tests with an existing tool (see TOOLS for suggestions). These will give you an output that you can use to then analysis your selected project's code base.

3.3 Coverage Improvement

Improve branch coverage by creating new test cases or enhancing existing ones. For this, identify regions of code that are in need of testing by analyzing the branch coverage with the currently existing tests from the previous task.

You must create/enhance test cases (the number of actual test cases does not matter) to cover two functions of the application code. The suggested steps are:

• Consider the existing tests and check how they cover the conditional branches;

• Identify the requirements that are tested or untested by the existing tests;

• Use the requirements for creating inputs and assertions;

• Follow naming conventions for unit test names based on the repository or create one as a group (for example {Function Name}_{Condition or Parameters}_{Expected Result}).

You should execute the existing coverage tool in such a process so as to verify if you are actually improving coverage. Look at getting to at least 80% branch coverage of each function you have selected.

3.4 Report (README.md)

Your work should be reported as the README.md file in the root of the forked repository (the original README.md file should be renamed to README_original.md). The README.md file should follow the report template for this assignment (see Section 2).

4. Evaluation and Grading

4.1 Grading Rubrics

Criteria	Ratings	Points
1) Execution of existing coverage measurement tool.	2 Pts: There is evidence (screenshot of the results) that a tool was executed on the project. 0 Pts: Otherwise.	2 pts
2) Creation/enhancement of new tests.	2 Pts: There is evidence (code) that each student created/enhanced tests to reach 80% conditional branch coverage. 1 Pt: There is evidence (code) that the group created/enhanced tests 2 tests per member. 0 Pts: Otherwise.	2 pts
3) Effectiveness of each new/enhanced test in improving coverage.	2 Pts: There is evidence (screenshot of the old and new results) that the tests created/enhanced by each member improve coverage. 1 Pt: There is evidence (screenshot of the old and new results) that some tests improve coverage, but not all. 0 Pts: Otherwise.	2 pts
4) Overall coverage improvement.	2 Pts: There is evidence (screenshot of the results) of the overall coverage improvement as measured with an existing tool. 0 Pts: Otherwise.	2 pts
5) Organization and clarity of the report (README.md).	2 Pts: Good. 1 Pt: Sufficient. 0 Pts: Insufficient.	2 pts

Total: 10 pts

4.2 Eligibility

To qualify for a grade, the following criteria must be met:

• The chosen project meets the requirements (Section 3.1) and is not used by another student;

• The forked repository URL is submitted on Canvas by the deadline;

• There are no modifications in the forked repository after the deadline;

• The report (README.md) follows the template;

• The student presents the work with slides on the presentation day.

5. Ethics

Cases of plagiarism and assignment contents exchange will be reported to and managed by the official fraud committee. In case of fraud, the consequences of those acts may potentially lead to formal warning, inclusion of the formal warning in your VU student file, suspension from taking exams for a given period, expulsion from the VU.