Table of Contents
What is Test Automation Framework?
Need of Test Automation Framework
Types of Test Automation Framework
Supported browsers and Platforms
Base Elements (Primitives)
Verification and Logging
Steps for Test Creation
REFERENCES AND CREDITS
In software industry
the products are releasing more frequently. Whenever we have a new product
release it is required to perform smoke test or regression test to ensure the
stability of the build.
Due to the increase
in the frequency of the build release It becomes very difficult for the manual
testers to execute every test case due to time constraints and urgency of
delivery. To overcome this problem test automation is used as a method to
perform more repetitive task. Test automation means using a software tool to
repeatable tests automatically and within a less time an application can be
tested. Selenium is one of the most popular and open source tool for test
Framework which is created using Selenium RC or web driver
and Java and can be used across projects for testing web based applications,
independent of programming language used during development of the application.
What is Test
A set of guidelines like coding standards, test-data
handling, object repository treatment, folder structure,
reporting formats etc., which when followed during
automation scripting produces beneficial outcomes
like increase code re-usage, higher portability, reduced
script maintenance cost etc.
Need of Test
An automation test suite can be built by simply recording
various test cases using IDE. However, making changes to the recorded scripts
after application changes with new commits and release is very difficult.
However, testing framework that is application independent
and has the capability to expand and incorporate changes is better solution.
Also, organized test framework helps avoiding duplication of tests. In short,
Framework increases efficiency of testing.
Types of Test
Automation frameworks can broadly be classified into three
Data Driven -In
this approach, Test data are stored in external files like (Excel files, text
files, CSV files, ODBC Sources, DAO Objects, ADO Objects) and are loaded into
variables inside the test script during runtime. This approach reduces the
problem of hard coding for input data.
-In this approach, you would first identify a set of Keywords and then
associate an action (or
function) which each of these keywords. So basically a
Keyword Driven Framework is nothing
but a collection of keywords & functions and the
association between these two entities.
approach combines the two approaches outlined above, and brings in benefits
both. Over a period of time, hybrid frameworks have emerged
as the de facto standard for
1. A well-structured Framework should be in place
2. Identify all the Business flow that should be automated
3. Necessary input data has to be created
4. Expected results have to be defined in order one can
judge that an application works
5. Execute a Test
6. Compares expected results with actual results, and
decides whether the test has been passed or
bug is to be raised
After covering basics of Test Automation and details about
Automation Framework, we will now cover
details of a Generic Automation Framework which is created
using Selenium RC, Java and some other
supporting tools like TestNG (execution engine), Apache POI,
Log4J Logging etc. This framework can be
implemented for any web
application, independent of its domain and programming
This framework is designed using Core library. This core can
used with any web application and supports the following
• Selenium Controller and UI primitives
• AssetList functionality.
• DataProvider for XLS input data processing.
• Custom Asserts
• DB connectors
This Framework has the following tools:
1. Selenium RCSelenium
is a test tool that allows you to write automated web
application UI tests in any programming
language against any HTTP website using any mainstream
Control is great for testing complex AJAX-based web user
Selenium RC comes in two parts.
1. A server which automatically launches and kills browsers,
and acts as a HTTP proxy for web
requests from them
2. Client libraries for computer language that is used for
In this framework, Selenium RC provides various APIs that
enables automation for web application.
2. Eclipse IDEEclipse
is an integrated development environment (IDE) for Java.
Eclipse is written mostly in Java and its
primary use is for developing Java applications, but it may
also be used to develop applications in other
programming languages through the use of plugins.
In this framework, Eclipse IDE is used by Automation
developer for accessing /modifying/ managing java
is a testing framework inspired from JUnit and NUnit. It has
extended new functionalities which
made it more powerful and easier than the other testing
It supports ReportNG, a simple HTML reporting plug-in. It is
intended as a replacement for the default
TestNG HTML report. The default report is comprehensive but
is not so easy to understand at-a-glance.
ReportNG provides a simple, color-coded view of the test
In this framework, TestNG is used for defining end-to-end
Log4j is a Java library that specializes in logging.
Typically logs are output to console window, using Log4J
output can also go to an email server, a database table, a
log file, or various other destinations.
In this framework, Log4J is used for logging various information
when test is run.
5. Apache POIApache
POI is a popular API that allows programmers to create,
modify, and display MS Office files using
Java programs. It is an open source library developed and
distributed by Apache Software Foundation.
In this framework, Apache POI is used for dealing with MS
Table below lists different Operating Systems and browser
that are supported by Selenium RC
Browser Selenium (RC) Operating Systems
Firefox 2,3,3.x Start browser, run tests Windows, Linux,
IE 8,7,6 Start browser, run tests
Safari 4,3,2 Start browser, run tests
Opera 10,9,8 Start browser, run tests Windows,
Google Chrome Start browser, run tests Windows, Linux, Mac
The framework that is depicted as part of this document has
specific structure and framework
components. They can be used as-is or can be modified as per
project requirements and specification of
AUT. Below are high level framework components and their
description, detailed explanation will be
covered under Project Structure.
1. Configuration File – Config file is used for storing
parameters such as log file path, database
location, host details, results path, selenium execution
speed, wait period, etc. This is the place
where all global configuration settings will be stored.
2. UI Maps – These are also java classes, however these are
used for storing xpath/IDs of application
elements such as Text Box, Drop downs, Radio buttons, Tabs,
Links etc. To help manage UI Maps,
different classes are created for different panels.
Note: Panel refers to any tabs / web page commonly seen in
any web application and is used in
this framework for grouping each component logically.
3. Meta information – As the name suggests, these are java
classes that provide information. In
context of this framework, meta classes are storage for
constants, variables for e.g. ui label,
dropdown values that will be frequently referred in test
4. Panels –These java classes are used for panel management.
These defines all elements of the
panels stored in UI maps and also contains reusable
application functions pertaining to panels.
For e.g. AddDriver(), CalculatePremium() etc.
1. Test Scripts – These are Java classes used for coding
test case flow. Core Java programming is used
for building test logic in these files.
2. XML Suite – XML suite is required for running tests, they
make call to individual java classes. XML
suite also defines parameter that are required for executing
a java file, for e.g. Data file name
3. Data Files – This framework uses MS Excel for providing
input data to tests. Excel Input data is
structured using Namespace feature of MS Excel.
This section provides details about folder structure and
different files that are part of framework.
As described earlier this framework is packaged into two
layers – 1. Business Layer 2. Test Layer
contains the framework components that are written during framework development
requires changes only when AUT changes. Components from this
layers are used during test case
1. ui: storage for ui related entities. This is the place to
store locators (xpath/IDs) for ui objects,
constants related to ui labels, panel management etc.
2. utilities: storage for different utilities/functions that
aids test case development
3. config: storage for all configuration related entities
that are required during test run
Suggested Folder Location: src/main
contains components that form test logic of the framework. A new file is added
to test layr
every time a new test case is automated. Folder structure is
created based on test case grouping of your
project. For E.g. Functional tests, timepoint tests etc.
Helper files are also part of this layer, and contains
different useful features and functions.
Suggested Folder Location: src/test
UI maps are the files that stores locators (xpath/ID)
details of an UI element, these are used to perform
application screen like create account page
operation on that element via test.
Screenshot : Application screen showing ‘Policy Information’
Usually in any selenium framework, UI maps are stored for
each and every UI element. This results into
creating huge number of locators. In this type of
frameworks, 9 UI map will be created for each of the
fields in Screenshot 1
This framework has very effective way of storing locators
for UI. Only one UI Map is added for UI elements
having same parent. So in this framework, only 1 UI map will
be created as all the 9 fields has same parent
element ‘Policy Information’ as seen below.
Attached open google
& span id field
UI map is organized as a customized enumeration to achieve
– Code unification and segregation
– Ability of processing tricky cases with dynamically
Important: UI Map for Table and AssetList should be always
represented as xpath. AssetList is specific
type of element that is used in this framework for grouping
different UI fields having same parent
Suggested Folder Location: src/main/java/ui/map
Screenshot: Enumeration of UI maps. Two enums Policy and
Renewal Preview is shown.
Under the Metadata term we mean all the constants for e.g.
tab names, dropdown values and variables
such as phone#, email id etc. Metadata is also organized as
customized enumeration for creating
This helps us in controlling hard coding directly into tests
cases. This in turn reduces maintenance effort,
as only changes are required into metadata files and all
tests are updated automatically.
Suggested Folder Structure: src/main/ui/app/metadata
Screenshot: Metadata showing different payment plans for
insurance policy creation.
Panel is referred for Application panels (tabs/web pages).
In general, every application has edit panel
where data is entered by user and summary panels which
presents consolidated data that user has
entered. Panel is used for declaring elements and also
contains reusable functions pertaining to that
Element declaration means creating an element type variable
that refers to its declaration in UI map files.
Element declaration also mentions Ajax associated with that
Edit panels contains elements declaration for simple
elements and AssetList elements. Also has functions
for data filling.
Suggested Folder Structure:
Screenshot: Edit panel screen showing various components
Summary panels contains element declaration and other
operations that can be performed on summary
Folder Structure: src/main/java/ui/model/panel/summary
Screenshot: Summary panel showing element declaration and
In above sections we saw different components that are part
of this framework and all of them revolve
around UI elements. UI elements are base of any automation
project, because if you can’t define and
access UI elements through your automation framework, you
cannot perform operation on those
elements. And test case flow is simply combination of
operation performed on UI elements.
In this section we will focus on types UI elements that are
seen commonly in web applications and how
to declare them so that they can be used in test cases.
This framework deal with standard UI elements such as Text
boxes, radio buttons and some new elements
are created to suit testing requirements such as AssetList,
e-folder, calendar etc. All UI elements have
their own type and these types are implemented in ISTF core.
So in business layer we are just instantiating
new UI elements based on AUT, however their implementation
is part of ISTF Core. Almost all elements
and methods are static. Static declaration permits to get
access to the element without creating class
The main element types:
? Base element (Primitives)
? Customized composite elements
Base Elements (Primitives)
These are typical elements that are available on most web
• TextBox … etc.
Base Element Declaration
All primitive elements can be declared and almost all of
them have 3 constructors:
? Button (locator, waitMode , timeout)
? Button (locator, waitMode)
? Button (locator)
locator – locator (xpath/ID) of the element on the page
defined in UI Maps
waitMode – wait mode that should be called after click/edit
on the element. Wait mode values: PAGE,
AJAX, SLEEP, NONE. If not defined wait mode is defaulted to
timeout – timeout of the wait mode type. If not defined
default one is used
Each element type have its own set of operations that can be
performed with it. These operations are
defined as part of ISTF Core library.
AssetList is unique and outstanding feature of this
framework. Elements on the page that can be grouped
and have the same block parent are stored as AssetList. This
reduces the number of elements declared in
UI Maps and corresponding element declaration in Panels.
Important: One AssetList cannot include 2 elements with the
same name (UI label). Because each of the
field within AssetList is accessed via UI Label.
AssetList is declared using below constructor –
AssetList (String ParentFormLocator, Class
ParentFormLocator – xpath of the block of elements defined
in UI Maps
NameOfAssetTypeRange – AssetList Attribute descriptor name
AssetList attribute descriptor class stores details about UI
elements that are part of an AssetList. Multiple
classes can be created each for different panels. Customized
enumeration is used for grouping these
elements. Elements within this group are represented with
unique name within class and has 3 attributes:
? Label name (name of the element on UI)
? Type of the element (textbox, radiobutton,
? WaitMode (if has).
Screenshot: AssetList Attribute Descriptor class
Each control has 3 main access methods, which return element
attributes accordingly. These can be used
for performing validations in tests.
? SearchAssetType.SEARCH_FOR.getLabel() ; –
returns element Label name
– returns element Control class
returns wait mode of element.
AssetList has its own set of operations defined in Core
Customized composite elements – complex elements, usually
are composed from primitives (common
panels/pop-up’s/etc.) Example: Yes/No pop-up, Calendar,
Document Folder etc. Based on project
requirements, new composite elements can be created.
Composite Element Declaration Individual java classes are
created for defining composite elements.
Suggested Folder Location: ui/common/model/composite
Below screenshot show java class created for YesNoDialog
Table element provides set of methods to manipulate with the
table structures: Cell, Row, Column etc.
Below constructor is used for declaring table element types
Table (String tableLocator).
tableLocator – xpath of table container.
In this case xpath will be: //[email protected]=’ billingDetailedForm:billing_transactions_active’
This framework uses Microsoft Excel for storing input data.
Apache POI is used as middleware API that
allows java classes to interact with MS Excel.
Apache POI (Poor Obfuscation Implementation) is an open
source library or API developed and
distributed by Apache Software Foundation. It provides a
JAVA based library of classes containing a set of
methods to create, modify and display Microsoft Office files
using JAVA. POI has the capabilities to read
and write data to and from the Excel files. Apache POI has
support for both files with “.xls” and “.xlsx”
extension. We have used example file with .xls extension in
This framework uses concept of Excel Namespaces for creating
Key-Value pairing for UI Field labels and
Input Data Values. An Excel Namespace is simply a name
attached to specific cell or group of cell in
worksheet. Creating namespaces helps in referring to group
of cells just by using the name given to that
This framework follows specific structure for managing data
within namespaces. Reading data from
namespaces is part of ISTF Core library that used POI API.
Same or different structure can be used in any
1. Usually web applications have different tabs for
capturing various data
2. Create data for each tab in one worksheet and give sheet
same name as that of tab
3. Tabs also have various sections, covering various data
4. This framework has two level of namespaces, 1- for
grouping different sections on a tab 2- for
grouping different UI labels within a section
5. Example – AAA Insurance application
? New sheet will be added in excel datasheet
for each of the tabs, ‘General’, ‘Applicant’,
? Level 1 namespaces will be created for
grouping ‘List of names insureds’, ‘AAA Membership’
and ‘Dwelling address’ under tab ‘Applicant’.
? Level 2 namespace will be created for each
of the individual sections covering different data
Level 1 – Key for level 1 namespaces should be same as
Strings defined in panels and Value should be
same as namespace for level 2
Level 2 – Key should be same as attributes defined in meta
Below is the details of each functions, implementation of
these functions are part of ITSF Core.
? BaseTest.getDataSuite() – returns excel file
name for the current test, defined in suite parameter.
? getDataGroup(“QuoteApplicant”) –
returns excel sheet. Almost each ui tab has its own sheet.
? getDataTable(dataSet) – returns table with
dataset for the current test and current sheet.
? dataset – namespace of the dataset table for
the current sheet. E.g. “HO3_Dataset1”
getLinkedData(“AccountGeneralInformation”) – returns test data table
for the selected asset list
for the current dataset.
Verification and Logging
ISTF core has functions created for validating test logic,
comparing actual and expected behavior of AUT.
This class is part of this framework that allows you
validate AUT behavior with expected output. Here is
the list of available functions,
Here is the example of usage,
If you need to verify some values, but test must go on even
if verification failed, you should use
CustomAssert.enableSoftMode(). This function allows collect
verification errors without stopping test run.
To stop enableSoftMode() use CustomAssert.disableSoftMode().
And all the collected information will be
displayed in console after execution is complete.
For checking that condition is true use
CustomAssert.assertTrue(). Similarly use use
CustomAssert.assertFalse() for checking a value is false.
Every control has special function verify that can check if
control is present, enabled, its values etc. Here
is a list of available features.
AssetLists verification – It is possible to verify values of
the entire AssetList:
Special functions – Some Framework Panels contain its own
verification functions for different controls.
It is done for easier usage and intuitive tests writing.
Logging is used to provide additional information for the
tester. Logging is displayed on console output
and helps in better understanding of test flow.
Steps for Test Creation
Step 1 – Identify test package where you want to add your
Step 2 – Create xml suite that will be used for calling java
class and pass parameters required for run
Step 3 – Create datasheet as per your test need and pass it
as parameter in xml suite
Step 4 – Import necessary classes from framework in your
java class and begin writing step by step flow
Step 5 – Put necessary validations using CustomAssert and
Step 6 – Run xml suite for executing your test individually.
Step 7 – You can create master suite xml and call multiple
java classes. All tests will run sequentially this
References and Credits