You are correct that Bamboo must be run in Console mode. If you have not discovered, your UI test machine has to have an active desktop for the UI tests to work. Our wallboard machine sits in a closet far from its big wall monitor and just happens to have a panel that was not used for anything, it now, via remote desktop, always has the UI Test machine active on it.
I have implemented what you are trying a little different though. I am a software developer who also manages our build servers. Our test team uses Telerik Test Studio and just wanted the simplest way to get a bunch of UI tests to run every few hours. One of our build machines, mentioned above, is now tasked with this and has had the ArtOfTest.Runner part of Test Studio installed on it.
Our testers use the Test List feature (mytests.aiilist), but I know this will work with single tests as well (mytest.tstest). Only once was the project exported to visual studio. After that they just build tests and add them to the Test List. Their whole solution is in source control. I did modify the vs project for bin deploy of all the components to make sure all the dlls get to the build machine.
After following the ArtOfTest.Runner installation instructions you should be able to run it from any path. So try this on the build machine: [AnyPath]\ArtOfTest.Runner C:[full path to test or list file]
If you can get that to work you are 1/2 way there.
To kick this off with Bamboo we do this:
- Checkout the whole project from source control
- Build it (make sure dlls get into bin)
- Use a NAnt script to launch tests
The reason a NAnt script was originally used was becuase we did not know if our path would always be the same and needed it to be calculated. We always know the path now though, but it still makes things simpler.
Bamboo Task Configuration:
Executable: NAnt
Build File: [Relative path from root of project]\RunMyTestList.build
Targets: all
Environment Variable: LIST="OneSetOfTests.aiilist"
And the RunMyTestList.build NAnt script looks like this:
<?xml version="1.0"?>
<project default="all">
<property name="ArtOfTest.Executable" value="ArtOfTest.Runner.exe" />
<property name="ArtOfTest.ListFile" value="${path::get-full-path(environment::get-variable('LIST'))}" />
<property name="returncode" value="" />
<target name="all">
<echo message="Start ArtOfTest.Runner: ${ArtOfTest.ListFile}" />
<exec program="${ArtOfTest.Executable}"
resultproperty="returncode"
commandline="list="${ArtOfTest.ListFile}""
failonerror="true"/>
<echo message="Finish ArtOfTest.Runner: ${returncode}"/>
</target>
</project>
So now the tests will run. But we only get a success or fail on the build, not a detailed list from Bamboo as you (and I) might want. This is because it is actually a NAnt script not a test. But this ended up being a blessing. ArtOfTest.Runner writes VERY GOOD results files, way better than any unit test log. We had to share the results folder on the build machine with the testers, who would not normally have access, but now they can map that share and open the results files in Test Studio and even see the step-by-step screen captures.
I don't think this configuration is perfect and I will probably adjust it more. Our critical item was that we use Bamboo for all our builds, testing, and deploys so we had no interest in using anything else to kick off the UI Tests. So far everyone is very happy. Hope I could help.
