I just setup browser testing framework utilizing Selenium from TeamCity project. The usual suspects are involved, Gallio, MBUnit, Nant and even C# Unittests.
The usual scenario when it comes to automating browser testing is that the QA / testers will create some scripts to run a browser against your website. Somehow those tests are usually not maintained very well and often are run by hand. There is not much value in browser testing scripts if you have to run them by hand.
As I needed browser testing on one of the projects I’m on I decided to look into using more of a automated setup to run the browser tests. There seem to be two big players Waitn and Selenium, Selenium lends itself to broader range of testing, naturally we will go with Selenium.
Here is the scenario I want, the tester installs a recorder on his computer, in this case a FireFox plugin. The tester records the tests and runs them in the browser using the plugin tool. Once the tester is happy with the tests the tester checks them into the repository. After check-in tester lets a developer know that there are new or changed tests. The developer takes the script and turns it into C# UnitTest, simply has Selenium convert it to UnitTest code. Then the developer takes and updates or adds the tests that resulted from the scripts and checks it into the repository. The conversion step could be automated in the future once Selenium supports that. The next step is to run it from TeamCity and after it runs you get email with the results.
So let’s take a closer look at what is needed. We need the UnitTest to be able to run against different servers using different browsers. We will pass values from TeamCity to the Nant script that is responsible for compiling and running the tests. This is how your test C# configuration file might look like.
<add key=“SeleniumAddress” value=“localhost” />
<add key=“SeleniumPort” value=“4444″ />
<add key=“SeleniumSpeed” value=“0″ />
<!– Browser targets –>
<add key=“BrowserType” value=“*firefox” />
<add key=“BrowserUrl” value=“http://10.9.169.198/” />
<add key=“BaseUrlPath” value=“IPCA.Dev/” />
Then the base test class will look something like this.
public virtual void TestFixtureSetup()
{
// Read from config
msBrowserType = getConfigSetting(“BrowserType”, msBrowserType );
msBrowserUrl = getConfigSetting(“BrowserUrl”, msBrowserUrl);
msBasePath = getConfigSetting(“BaseUrlPath”, msBasePath);
//
msSeleniumAddress = getConfigSetting( “SeleniumAddress”, msSeleniumAddress );
miSeleniumPort = int.Parse(getConfigSetting(“SeleniumPort”, miSeleniumPort.ToString()) );
msSeleniumSpeed = getConfigSetting( “SeleniumSpeed”, msSeleniumSpeed );
// Start up the selenium session, using config values
selenium = new DefaultSelenium(msSeleniumAddress, miSeleniumPort, msBrowserType, msBrowserUrl);
selenium.Start();
// Clean errors
verificationErrors = new StringBuilder();
// sets the speed of execution of GUI commands
selenium.SetSpeed(msSeleniumSpeed);
}
[TearDown]
public void TeardownTest()
{
try
{
selenium.Stop();
}
catch (Exception)
{
// Ignore errors if unable to close the browser
}
Assert.AreEqual(“”, verificationErrors.ToString());
}
And a sample Selenium C# Unittest
using System;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
//
using Gallio.Framework.Assertions;
using MbUnit.Framework;
//
using Selenium;
namespace SeleniumTests
{
[TestFixture]
public class LoginPage : WebTestBase
{
[Test]
public void TheLoginPageTest()
{
selenium.Open( this.msBasePath + “TestLogin.aspx”);
selenium.Click(“lbAdmin”);
selenium.WaitForPageToLoad(“50000″);
selenium.Click(“loginLink”);
selenium.WaitForPageToLoad(“50000″);
try
{
Assert.IsTrue(selenium.IsTextPresent(“my responsibilities regarding permissible access”));
}
catch (AssertionException e)
{
verificationErrors.Append(e.Message);
}
selenium.Click(“ctl00_pageContent_btnSubmit”);
selenium.WaitForPageToLoad(“50000″);
try
{
Assert.IsTrue(selenium.IsTextPresent(“Total Unassigned Web”));
}
catch (AssertionException e)
{
verificationErrors.Append(e.Message);
}
}
}
}
After the Nant script compiles the tests and is getting ready to run the UnitTests it needs to startup the Selenium engine. Make sure to spawn in order for the Selenium engine to exist on another thread than your tests.
<property name=“SeleniumPath” value=“C:\apps\selenium\selenium-server-1.0.3\” />
<property name=”SeleniumParams” value=”-jar ${SeleniumPath}selenium-server.jar” />
<!– Start selenium –>
<exec program=”${SeleniumExec}“
commandline=”${SeleniumParams}” workingdir=”${path.base}${WebTest}“
spawn=”true” failonerror=”true” verbose=”true“
/>
<!– Give it a sec to load –>
<sleep milliseconds=”3000” />
In order to run the tests using different browsers, change the configuration file of the tests before run.
<xmlpoke
file=“${path.base.test}${assembly.test.config}”
xpath=“/configuration/appSettings/add[@key='BrowserType']/@value”
value=“*firefox”
verbose=“true”/>
<call target=“runTests” />
<target name=“runTests”
description=“runs tests using Gallio.” >
<echo message=“*** Start runTests: “/>
<gallio
result-property=“exitCode”
failonerror=“false”
report-types=“Html;Xml”
report-directory=“${artifacts}”
report-name-format=“gallioresults”
show-reports=“false”
application-base-directory=“${path.base.test}”
>
<!– Specify the tests assemblies –>
<files>
<include name=“${path.base.test}${assembly.test}”/>
</files>
</gallio>
<!–
Set error for email injector to pick it up and GlobalFailBuildMessage for
the end target to fail the build after cleanup
–>
<if test=“${int::parse(exitCode)!=0}”>
<property name=“GlobalFailBuildMessage” value=“*** One or more tests failed. Please check the log for more details” dynamic=“true” />
<echo message=“EmailInjectMsg=${GlobalFailBuildMessage}” />
</if>
<echo message=“*** End runTests: “/>
</target>
And after the run of the Unittests Selenium needs to be shut down
<get src=“http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer”
dest=“shutdown.txt” failonerror=“false”
/>
As I had a need to setup different configurations in TeamCity to run against different locations on the webserver I used a couple of Nant variables that are passed on the command line from TeamCity, like you normally would do when running Nant script -D:BaseUrlPath=/Test/ etc.
<echo message=“*** BrowserUrl=${BrowserUrl} “/>
<echo message=“*** BaseUrlPath=${BaseUrlPath} “/>
Of course you get the Gallio UnitTest report as well
With this setup once we deploy to a server we can run all the browser tests on it using different browsers with one click of a button from TeamCity.