Getting Chrome working under ScalaTest

[Very Typesafe-stack-specific, but having spent the time figuring it out...]

Nothing to brighten my day than managing to solve what briefly looked like an intractable problem.

I'm trying to get full-stack functional tests running for Querki. (Not before time, I know.) To do this, I'm using ScalaTest + Play, which seems to be the approved solution. This is built on top of Selenium, the long-standing standard toolset for driving a browser for testing.

Problem was, when I wrote my first trivial test, ScalaTest refused to run it, claiming "Was unable to create a Selenium ChromeDriver on this platform". This was mysterious -- I have Chrome installed on this (Ubuntu) machine, and it works fine. A brief round with Typesafe suggested that the problem might be my X windows setup, which was a tad depressing, but I decided to get stubborn, rip the lid off the code and see what was actually going on.

It turns out, if you look at the relevant code in scalatestplus-play, that ScalaTest is intentionally flattening all exceptions from Selenium's ChromeDriver into a simple "that doesn't work here" cancellation. This is useful -- it's how you can write an all-browsers test suite in ScalaTest, and have it just quietly suppress the ones that don't make sense on this platform (eg, trying to run an Internet Explorer test on Ubuntu, or a Safari test on Windows). But it also loses what turn out to be some extremely helpful and informative exceptions that are raised by ChromeDriver.

So I added and ran this pseudo-test:

class DriverTests
  extends WordSpec
  with Matchers
  with ChromeFactory
{
  "I should be able to get a decent error" in {
    import org.openqa.selenium.chrome.ChromeDriver
    new ChromeDriver()
  }
}

That gave me what I was looking for: the actual exception was

[info]   java.lang.IllegalStateException: The path to the driver executable must be set by the webdriver.chrome.driver system property; for more information, see http://code.google.com/p/selenium/wiki/ChromeDriver. The latest version can be downloaded from http://chromedriver.storage.googleapis.com/index.html
[info]   at com.google.common.base.Preconditions.checkState(Preconditions.java:200)
[info]   at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:105)
[info]   at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:75)

It turns out that, as it says, you need to manually download the relevant native drivers, install them, and point to them. Oddly, I haven't found this documented anywhere.

Anyway, fixing that produced *another* helpful exception, telling me that ChromeDriver required Chrome >= version 46, and mine was too old. I updated that, and *poof* -- it all works.

So: if you are having trouble getting ScalaTest's browser drivers working, I recommend the above little pseudo-test, so you actually get the underlying exceptions. And note that you need those drivers...

Running headless browser tests using ScalaTest

[Continuing the previous story, basically. Again, only interesting to the Scala-using crowd. None of this is rocket science, but since I had to piece it all together from scattered info I might as well post it. Note that this is stream-of-consciousness notes, as I develop this test harness.]

Next step: can we run our browser-based functional tests "headless"? This is awfully useful: Selenium is great, but it is *very* distracting to have a browser window suddenly pop up in front of you and start doing things by itself. (And that simply won't work in a truly headless environment, such as an automated test server.)

Fortunately, this also proves to be easy, if poorly-documented. This answer on SO works perfectly fine for ScalaTest + Play. The key is the useful little program xvfb -- the X Virtual Frame Buffer. Basically, it's a program that lets you set up a "virtual" window that doesn't actually exist, which you can then point your tests at.

So first, you need to install xvfb, which on Ubuntu is:

sudo apt-get install xvfb

Then you run it, like this:

Xvfb :1 -screen 5 1280x1024x8 &

(Note the capital "X" in Xvfb.) That is, run a virtual screen in the background (that "&" at the end), creating display number 1 and screen number 5, with the specified virtual resolution. That kicks the screen off as a background process (you can find it with "ps").

Finally, say this:

export DISPLAY=:1.5

This tells the processes in this shell that, if they're trying to show something in X, they should use display 1, screen 5 -- your new virtual screen.

Now you can run sbt and your ScalaTest + Play functional tests as documented. The browser will start and run as directed, but you won't see anything, since the "display" is all going to xvfb's bit-bucket.

A nice side-effect of this approach is that the xvfb stuff is completely external to your test. You can write this into your standard shell environment for normal use, but you can also turn it off when things are going wrong, or if you simply want to be able to observe the proceedings while you are coding up your tests.