COMSATS University Islamabad

Department of Computer Science

CSE455 – Software Testing

BS(SE) – Section 7A

Assignment 3

Submitted by:

Ibrahim Abdullah
SP20-BSE-038

Ashar Irfan
SP20-BSE-017

Submission to:
Mam Najmun Nisa
 Test Plan Document for Urdu-fi

1. Test Plan Identifier:

Identifier:
MP-FYP-Urdufi-SP20

Date:
23-05-2023

Version:
1.1

Author:
 Ibrahim Abdullah
 Ashar Irfan

Contact Information:
 Ibrahim Abdullah: [email protected]
 Ashar Irfan: [email protected]

Revision History:
 Version 1.1: Addition of module 8 (Dubbing)
 2. Introduction:

Level of Test Plan:

Master Test Plan

Purpose of the Test Plan:

The purpose of the proposed plan is to develop an Artificial Intelligence and web-based
application that addresses the language barrier between English and Urdu by providing high-
quality Urdu subtitles for English movies. The plan aims to use Natural Language Processing
(NLP) techniques to transcribe English audio into Urdu subtitles using context and filter
offensive language. The system's objective is to enhance the viewing experience for Urdu-
speaking audiences and facilitate future research in Urdu language NLP.

Scope of the Test Plan:

The scope of the plan includes the development of various modules such as Translation,
Expression Detection, Offensive Language Detection, Subtitle Generation, Analytics Generation,
and Profiling. The Translation module involves extracting English audio from videos,
transcribing it into Urdu text using NLP techniques, and detecting background audio cues. The
Expression Detection module uses text and image datasets to identify characters' expressions in
the video. The Offensive Language Detection module filters out offensive language from the
transcript. The Subtitle Generation module formats the transcript into subtitle files and provides
them to the users. The Analytics Generation module generates usage statistics and reports. The
Profiling module allows users to sign up, sign in, and manage their subscription plans.

The plan references related systems such as Happy Scribe, Simon Says AI, Check Sub, Veed.io,
and academic research on Word-Order Issues in English-to-Urdu Statistical Machine Translation
and Urdu Language Translator using Deep Neural Network. It highlights the weaknesses of these
systems and how Urdu-fi aims to overcome those limitations.
Resource Constraints:

3 developers and these 3 developers will test the system.

Cost Constraints:

Budget: $700K

Process of Change Control:

Identification:
Any proposed change or modification to the system will be identified by the stakeholders, users,
or project team members. This can include enhancements, bug fixes, or any other changes
deemed necessary.

Evaluation:
The proposed change will be evaluated by the project team to assess its feasibility, impact on the
system, resources required, and alignment with project objectives. This evaluation will determine
whether the change should be accepted or rejected.
Impact Analysis:
If the change is accepted, an impact analysis will be conducted to determine the potential effects
of implementing the change. This analysis will consider factors such as cost, schedule, risks, and
other dependencies.

Approval:
The change request, along with the impact analysis, will be presented to the relevant stakeholders
or change control board for approval. The approval decision will be based on the analysis and the
overall project priorities.

Implementation:
Once the change is approved, it will be implemented according to the project plan. This may
involve updating project documentation, modifying code, or making other necessary adjustments
to the system.

Testing and Validation:

After implementing the change, thorough testing and validation will be performed to ensure that
the system functions as expected and that the change has not introduced any unintended
consequences or errors.

Documentation:
Documentation related to the change, such as updated system specifications, user manuals, and
other relevant documents, will be updated to reflect the implemented changes.

Communication and Coordination Process:

Stakeholder Communication:
Regular communication channels will be established to keep stakeholders informed about project
progress, milestones, and any significant updates or changes. This can include meetings, emails,
project status reports, and other communication tools.

Project Team Coordination:

The project team members will have regular meetings to discuss tasks, progress, challenges, and
dependencies. Collaboration tools and project management software can be used to facilitate
communication and coordination among team members.
Issue Tracking and Resolution:
Any issues or conflicts that arise during the project will be tracked and resolved through
effective communication and coordination. This can involve identifying the issue, discussing it
with relevant stakeholders, and finding solutions or compromises to resolve the issue.

Change Management Communication:

The communication plan will include specific channels and processes to communicate changes
to the system, including change requests, approvals, and implementation details. This will ensure
that all stakeholders are informed about the changes and their impact.

Documentation Sharing:
Relevant project documentation, such as project plans, requirements, designs, and test cases, will
be shared among team members to ensure everyone has access to the necessary information. This
will facilitate coordination and collaboration among team members.

Project Meetings:
Regular project meetings will be conducted to discuss project progress, address any issues or
concerns, and coordinate upcoming activities. These meetings can be held in person or virtually,
depending on the project team's location.

Feedback and Review:

Feedback and review sessions will be conducted to gather input from stakeholders and team
members on the project progress, deliverables, and any improvements or adjustments that may
be needed. This feedback will help in continuous improvement and effective coordination.

3. Test Items:

The following are the test items that will be tested for Urdu-fi system:

1. Web-based Urdu-fi Application.

2. Models used for translation and expression detection.
3. Features and functionalities of Urdu-fi system
Features to be Tested:

 Audio Extraction and Conversion:

Test the functionality of extracting English audio from the provided video file and converting
it to Urdu text using natural language processing techniques.
- Level of risk: High

 Background Cue Detection:

Test the detection and identification of audio cues in the background of the video and their
addition to the Urdu transcript.

- Level of risk: Medium

 Expression Detection:
Test the detection of characters' expressions in the video using machine learning tech-
niques and their addition to the Urdu transcript.

- Level of risk: Medium

 Offensive Language Detection: Test the system's ability to detect offensive language in
the Urdu transcript and apply censorship and filtering.

- Level of risk: High

 Subtitle Generation: Test the overall functionality of generating Urdu subtitles for the
provided video, including the integration of all the above modules.

- Level of risk: High

 Subtitle Format Conversion: Test the ability of the system to provide subtitle files in
various formats such as TXT, SRT, etc.

- Level of risk: Low

 Dubbing Functionality: Test the functionality of dubbing English audio to Urdu and in-
tegrating it with the video.

- Level of risk: Medium

 User Input: Test the acceptance and handling of user input in the form of a video file for
transcription.

- Level of risk: High

 Profile Management: Test the functionality of user profile creation and management, in-
cluding payment and subscription plans.

- Level of risk: Medium

 Performance: Test the performance of the system in terms of speed and accuracy of
transcription, subtitle generation, offensive language detection, and dubbing.

- Level of risk: High

 Compatibility: Test the compatibility of the system with different video formats and
configurations.

- Level of risk: Medium

 Security: Test the security measures implemented in the system to protect user data and
prevent unauthorized access.
- Level of risk: High

 Integration: Test the integration of the different modules of the system and ensure
smooth communication between them.

- Level of risk: High

 Error Handling: Test the system's ability to handle and display appropriate error mes-
sages for different scenarios, such as invalid input, network errors, etc.

- Level of risk: Medium

Features NOT to be Tested:

 Video Compression:
The testing scope does not include the evaluation of video compression algorithms or
techniques.
 Video Playback:
The testing scope does not cover the assessment of video playback functionality, such as
play, pause, rewind, or fast forward.
 Audio Enhancement:
The testing will not focus on improving or enhancing the quality of the audio in the video.
 Video Editing:
The testing scope excludes the evaluation of video editing capabilities, such as trimming,
merging, or applying visual effects.
 Speech Recognition:
The testing will not involve the assessment of speech recognition accuracy, or the underlying
algorithms used for speech-to-text conversion.
 Translation Accuracy:
The testing will not specifically target the accuracy of the translated text from English to
Urdu.
 Language Localization:
The testing will not include the assessment of the system's ability to support multiple
languages other than English and Urdu.
 Voice-over Narration:
The testing scope does not involve evaluating the functionality of adding voice-over
narration to the video.
 Caption Styling:
The testing will not focus on customizing or styling the appearance of the generated subtitles.
 Cloud Storage Integration:
The testing will not cover the integration with cloud storage platforms for storing or
retrieving video files.
 4. Test Strategy/Approach:

The overall test strategy for the Urdu-fi system encompasses the testing approach and rules to
ensure the quality and functionality of the system. It covers the use of special tools, metrics
collection, configuration management, regression testing, handling untestable elements, project
testing approach, and any special requirements. The strategy is designed to validate the system's
performance, accuracy, and reliability.

 Special Tools:
The testing process for the Urdu-fi system may require the use of special tools such as testing
frameworks, automation tools, bug tracking systems, and performance monitoring tools.
These tools aid in efficient testing, bug identification, and tracking throughout the
development lifecycle.

 Tool Training:
If any special tools are used for testing, the testing team may require specific training to
effectively utilize those tools. Training sessions should be conducted to ensure the testers are
proficient in using the tools and can maximize their potential for accurate and efficient
testing.

 Metrics Collection:
Metrics play a vital role in measuring the system's performance and identifying areas for
improvement. The following metrics can be collected during testing:

 Accuracy of subtitle translation

 Efficiency of offensive language detection
 Time taken for subtitle generation.
 Number of false positives or negatives in expression and cue detection

The metrics mentioned above should be collected at the component level, where each module
is individually tested for its functionality and performance. Additionally, metrics can be
collected at the system level to assess the overall performance and integration of all modules.

 Configuration Management:
Configuration management ensures that the system is tested across different hardware and
software configurations to identify any compatibility issues. It involves testing the system on
various hardware setups, software versions, and combinations of vendor packages to ensure
broad compatibility and stability.
 Number of Configurations:
The Urdu-fi system should be tested on multiple hardware configurations, including different
processors, memory sizes, and operating systems. Similarly, it should be tested on various
software configurations, including different browsers, media players, and relevant software
versions. Testing combinations of hardware, software, and vendor packages will help identify
potential issues and ensure system compatibility.

 Regression Test Rules:

Regression testing aims to validate that the system's existing functionality remains intact after
introducing new changes or fixes. The following rules can be established for regression
testing:

 Full regression testing should be performed for critical modules or components.

 Partial regression testing can be performed for less critical modules or components.
 The selection of regression test cases should be based on the impact analysis of the
changes made.

 Severity-based Regression Testing:

Regression testing can prioritize test cases based on the severity of defects detected. Critical
or high-severity defects should undergo rigorous regression testing, ensuring they are fixed
and do not reoccur. Lower-severity defects may undergo partial or selective regression testing
based on their impact on the system's functionality.

 Handling Untestable Elements:

In cases where requirements or design elements are deemed untestable or do not make sense,
they should be escalated to the appropriate stakeholders, such as the project manager or
system architects. Discussions should be initiated to clarify and address these elements,
ensuring the testable aspects of the system are adequately covered.

 Project Testing Approach and Coverage:

As a master test plan, the overall testing approach and coverage requirements should be
identified. This includes identifying the testing levels (unit testing, integration testing, system
testing, etc.), test types (functional testing, performance testing, security testing, etc.), and the
coverage criteria for each level. It ensures that the testing process covers all necessary
aspects of the system.

 Special Requirements:
Any special requirements related to the testing process should be identified and addressed.
For example, if the Urdu-fi system requires specific testing techniques or methodologies,
such as natural language processing or machine learning-based testing, they should be
incorporated into the test strategy.

 Test Environment:
A dedicated test environment should be set up to simulate real-world scenarios and
configurations. It should replicate the production environment as closely as possible to
ensure accurate testing and minimize any discrepancies between the test and live
environments.

 Test Data Management:

Test data plays a crucial role in validating the system's functionality and performance. A
comprehensive test data management strategy should be established to ensure the availability
of relevant and representative test data. This includes creating test data sets, managing data
privacy and security, and refreshing or updating test data as needed.

 Test Execution and Reporting:

The test strategy should outline the test execution process, including test case execution,
defect tracking, and reporting. It should specify the frequency of test execution cycles and the
channels for reporting and communicating test results to stakeholders. Clear and concise test
reports should be generated, highlighting the test coverage, defects found, and overall system
performance.

 Continuous Improvement:
The test strategy should emphasize continuous improvement throughout the testing process.
Feedback from test execution, defect analysis, and stakeholder inputs should be used to
refine and enhance the testing approach. Lessons learned and best practices should be
documented to guide future testing efforts and ensure ongoing quality improvement.

5. Item Pass/Fail Criteria

The completion criteria for the Urdu-fi system test plan are essential for determining the
success of the test process and identifying whether a test item has passed. These criteria
should be tailored to the specific level of the plan and align with the objectives of each
testing phase. Here are the completion criteria for the Urdu-fi system test plan:

At the Unit Test Level:

 All unit test cases have been executed and completed, ensuring that each unit of the Urdu-fi
system functions as intended.
 A specified percentage of test cases have been completed with a defined percentage contain-
 ing an acceptable number of minor defects. This ensures that the majority of test cases have
been executed successfully, even if some minor issues are present.
 Code coverage analysis indicates that all relevant code components of the Urdu-fi system
have been thoroughly exercised and covered by the unit tests. This ensures that the imple-
mented code has been adequately tested.

At the Master Test Plan Level:

 All lower-level test plans, including unit, integration, and system test plans, have been com-
pleted. This ensures that all necessary testing activities have been executed as per the overall
test plan.
 A specified number of plans have been completed without any critical errors, and a percent-
age of plans may contain minor defects that are within acceptable limits. This ensures that the
individual test plans contribute to the overall success of the Urdu-fi system.
 The master test plan demonstrates comprehensive coverage of all required testing activities,
ensuring that all critical aspects of the Urdu-fi system have been thoroughly tested and vali-
dated.

Defect Criteria:
 The number and severity of defects located during testing are recorded and analysed. This
helps assess the quality and reliability of the Urdu-fi system.
 Comparing the number of detected defects to the total number of defects may be challenging
as some defects may go undetected. However, the goal is to identify and address as many de-
fects as possible during testing.
 A defect refers to any flaw or issue in the system that may cause a failure or impact its func-
tionality. Defects can vary in severity, and their acceptability depends on their impact on the
overall system performance.
 A failure is the result of a defect that can be observed by the user, such as system crashes, in-
correct output, or unexpected behaviour. Failures indicate critical issues that need to be ad-
dressed before the Urdu-fi system can be considered acceptable.

Functional Requirements:
 The test plan ensures that all functional requirements specified for the Urdu-fi system have
been thoroughly tested and validated. This includes verifying that the system functions cor-
rectly and meets the desired functional specifications.
 All functional features and modules of the Urdu-fi system are tested to verify their proper
functioning and adherence to user expectations. This includes validating input and output be-
haviour, data processing, and user interactions.

Overall System Quality:

 The Urdu-fi system's overall quality is assessed based on the number and severity of defects
discovered during testing. A lower number of critical defects indicates higher system quality.
 The system demonstrates stability, reliability, and satisfactory performance, meeting the ex-
pected standards for the Urdu-fi application.
  The system meets the predefined functional and non-functional requirements, such as speed,
accuracy, and usability.
 The Urdu-fi system exhibits robustness and resilience, ensuring it can handle expected user
loads and stress conditions without significant failures.

6. Suspension Criteria and Resumption Requirements

Testing suspended if:

 A critical defect is identified that renders the application unusable or poses a high risk.
 More than 50 defects of severity higher than 6 have been identified in a single phase.

Testing can be resumed once the critical defect is fixed, or the acceptable level of defects is
reached. Regression testing should be performed after resumption to ensure the stability of the
application.

7. Test Deliverables for Urdu-fi System:

As part of the test plan for the Urdu-fi system, the following deliverables are to be provided:

 Test Plan: A comprehensive document outlining the objectives, scope, approach, and
schedule of the testing activities for the Urdu-fi system. It serves as a guide for the testing
team throughout the testing process.

 Test Design Specifications: Detailed specifications describing the test design techniques,
strategies, and methodologies to be employed during the testing phase. This document
outlines the overall testing architecture and approach.

 Test Case Specifications: Detailed documentation specifying the individual test cases to
be executed during the testing phase. It includes inputs, expected outputs, preconditions,
and postconditions for each test case.

 Test Procedure Specifications: A document outlining step-by-step instructions for exe-

cuting the test cases. It provides guidance to the testing team on how to perform the tests
accurately and effectively.

 Test Item Transmittal Reports: Reports documenting the transfer of test items, such as
software builds or components, from the development team to the testing team. These re-
ports ensure proper communication and tracking of test items.
 Test Logs: Records capturing the execution details of each test case, including the actual
inputs, outputs, and any observations or issues encountered during testing. Test logs serve
as a valuable reference for troubleshooting and analysis.

 Test Incident Reports: Reports documenting any anomalies, defects, or issues identified
during the testing process. These reports provide a detailed account of the problem, its
impact, steps to reproduce, and any supporting evidence.

 Test Summary Reports: Comprehensive reports summarizing the overall testing activi-
ties, including test coverage, test results, identified defects, and any other relevant met-
rics. These reports provide stakeholders with a holistic view of the testing progress and
outcomes.

 Test Data: Data sets and scenarios specifically prepared for testing purposes. These in-
clude both valid and invalid inputs to evaluate the functionality and performance of the
Urdu-fi system.

 Test Tools: Any specialized tools or software utilized to facilitate the testing process,
such as test management tools, automated testing frameworks, or performance testing
tools.

8. Test Tasks for the Urdu-fi System:

 Define test requirements and objectives:

 Analyze the project documentation, requirements, and specifications specific to the Urdu-
fi system.
 Collaborate with stakeholders to clarify any language-specific requirements or nuances.
 Establish clear objectives and goals for testing the Urdu language processing capabilities.

 Design test cases and procedures:

 Develop comprehensive test cases that focus on validating the Urdu text processing func-
tionalities.
 Design test scenarios that cover various aspects of Urdu text input, output, and language-
specific features.
 Create detailed test procedures outlining step-by-step instructions for executing the test
cases.
  Execute test cases and record results:
 Execute the designed test cases to assess the Urdu-fi system's performance in handling
Urdu text.
 Record and document the actual results, noting any deviations or issues encountered dur-
ing testing.

 Report and track defects:

 Report identified defects or issues using a standardized format, providing clear and con-
cise information.
 Include relevant details such as steps to reproduce, expected and actual behavior, and
severity or priority levels.
 Track the progress of defect resolution, retesting, and closure to ensure timely resolution.

 Perform regression testing after software updates or modifications:

 Conduct regression testing to verify that modifications or updates do not introduce new
defects or impact existing Urdu text processing capabilities.
 Re-execute selected test cases from the test suite to ensure the stability and compatibility
of the Urdu-fi system.

 Generate test summary reports:

 Summarize the testing activities, including the number of test cases executed, passed, and
failed specifically for the Urdu language processing aspect.
 Highlight any significant observations, issues, or risks identified during testing related to
Urdu text handling.
 Provide recommendations for further improvements or actions based on the test results
specific to Urdu-fi system testing.

Repeat these tasks for each subsequent sprint or iteration to ensure continuous testing and
improvement of the Urdu-fi system.
 9. Environmental Needs

The following environmental needs should be considered for Urdu-fi system:

Technologies and Tools:

 Make sure that the testers have proficiency and access to the technologies and tools used
by the system like react app, ffmpeg, Mongo db., Node JS, Mosses Toolkit etc.
Hardware and software configurations for testing:
 Ensure that the testing environment includes a PC with CPU of intel i7 7th generation at
minimum or AMD Ryzen 2600.
 Windows 7, 8, 10 or 11 installed.
 Chrome or any web browser installed.
Access to the web-based Urdu-fi application:
 Testers should have access to the web-based Urdu-fi application for executing test cases
and validating the functionality.
 Provide the necessary credentials or accounts to log in and access the application.
Test data, including models for testing:
 Testers must be provided with samples videos for uploading.
Web and network communication capabilities:
 Ensure that the testing environment has web and network communication capabilities to
validate the web-based functionality of the Urdu-fi application.
 This includes stable internet connectivity, proper network configurations, and access to
any external services or APIs (stripe, Whisper) used by the application.

10. Responsibilities for the Urdu-fi System Testing:

Project Manager:
 Holds overall accountability for the Urdu-fi project, including coordination and decision-making.
 Ensures that the test plan for Urdu-fi is aligned with the project objectives and timelines.
 Allocates necessary resources and provides support for testing activities specific to Urdu-fi.
 Resolve any scheduling conflicts and addresses project-related issues effectively.

Test Manager:
 Assumes responsibility for test planning, execution, and reporting specifically for the Urdu-fi sys-
tem.
 Formulates a tailored test strategy and plan to meet the unique requirements of Urdu-fi.
 Assigns tasks to testers dedicated to Urdu-fi testing and monitors their progress.
 Collaborates closely with stakeholders to ensure the successful testing of the Urdu-fi system.
Testers:
 Conduct test case execution, defect reporting, and provide feedback exclusively for the Urdu-fi
system.
 Collaborate with the development team to address and resolve defects specific to Urdu-fi.
 Actively participate in test case reviews and engage in discussions related to Urdu-fi testing.
 Contribute to the enhancement of the Urdu-fi testing strategy and overall test coverage.

Development Team:
 Collaborates closely with testers dedicated to the Urdu-fi system to address and resolve defects.
 Offers necessary support to ensure the stability and functionality of the Urdu-fi application.
 Work in tandem with testers to understand and tackle any challenges or issues encountered during
Urdu-fi testing.
 Ensures timely delivery of software components pertaining to the Urdu-fi system for testing pur-
poses.

11. Staffing and Training Needs for the Urdu-fi System:

Product Training:
 Testers involved in the Urdu-fi system testing will require comprehensive training on the
product itself.
 This training should cover the functionality, features, and intended usage of the Urdu-fi
system.
 Testers should gain a deep understanding of the system's capabilities, limitations, and ex-
pected behavior to effectively design and execute test cases.

Test Tool Training:

 If any specific test tools are employed for testing the Urdu-fi system, relevant training
should be provided to the testers.
 This training will focus on familiarizing testers with the test tools, their functionalities,
and how to effectively utilize them for test case management, execution, and defect track-
ing.
 Testers should gain proficiency in using the test tools to maximize productivity and effi-
ciency during the testing process.

.
12. Schedule:

13. Risks and Contegencies:

 14. Approval:

The test plan and the test reports are to be approved by the Project Supervisors and the
evaluation committee.