Selenium

[jducoeur]

It occurs to me that, having come across another Massively Wonderful Open Source Tool, I ought to talk it up a little.

My current project at work is implementing an automated GUI test harness using Selenium. If you're doing any sort of serious Web development, you owe it to yourself to check this thing out.

Selenium is a fairly complex toolset, but it's all aimed at browser-level testing. There's Selenium Core, a complex Javascript library that lets you manipulate the browser to a fine level of detail; Selenium IDE, which lets you do macro-style testing; and Selenium Remote Control (which is what we're actually using), which lets you control the browser remotely from a separate harness. The end result is that I'm able to write a detailed test suite in NUnit (the .NET equivalent of JUnit), full of fine-grained browser testing: telling the browser to open windows, check what's on the page, click the buttons, and so on just as I want.

It has all the usual virtues and vices of open source. The documentation is a tad crappy, and there are some rough edges. OTOH, it's free, not terribly hard to integrate and use, and impressively powerful. It runs with both Firefox and IE, and the Remote Control system has integration modules for many major development platforms (Java, .NET, Perl, PHP, Python, Ruby, etc).

In a bit over a week of work, I should be able to set up enough harness and examples to hand the thing off to QE and have them continue to automate the GUI-level testing from there. Combine that with the API and Unit test suites we already have, and I'm starting to think we may actually be able to pull off a reasonably agile development style. This tool looks to be exactly what we needed at this point; take a look at it...

Comments

[siderea]

Excellent! (Re-)Checking out Selenium was, in fact, on this month's to-do list. I just spent some time poking around and it looks fabulous.

So I have a question for you. I've lived in the testing-automation-free world of web development for the better part of a decade. Looking over Selenium, it's very clear that automated QA has developed it's own jargon, concepts, etc. of which I am wholly ignorant. For instance, in looking at the Selenium docs, I've derived "assert" from context (it means "test criterion" more or less, yes?).

Is there some book/site/documentation one can read to get up to speed on the general concepts and terminology of state-of-the-art QAing, that you recommend? So that I could make the most of Selenium?

[jducoeur]

Yeah, there are a fair number of assumptions that underlie the automated-testing field. I have to admit, though, that I'm probably not the best person to ask, for the usual reason: as with all technology, I just sort of picked it up by osmosis. (It's fairly rare for me to crack a book on any aspect of programming -- I mainly skim online reference manuals.) I'd recommend asking goldsquare, who does high-end QA for a living, and is undoubtedly better-read on the subject than I am. I would also suspect that poking around the OpenQA site might be useful, but that's just a guess.

That said, I'm always happy to answer questions. "Assert" is the most primal term in automated testing, and you more or less have the meaning right. It's short for "I assert the following to be true", and indicates that the expression should be tested now: if the test is false, throw an exception, report the error and (usually) cease testing. There are a bunch of common variants of this: "Validate" tends to mean "test, and report if it fails, but keep going", and "Fail" is an imperitive "if we've gotten to this point, something is horribly wrong" (basically an automatic Assert(false)).

[siderea]

... Or it could finally dawn on me to put "QA Terminology" into google...

[asim.livejournal.com]

I looked at Selenium for awhile; really liked the package, esp. as I was building an AJAXy site. The downside was that it couldn't, at the time, do BASIC Auth over HTTP, which I needed rather badly. Ended up back with WWW::Mechanize, a Perl module for doing similar work.
Maybe it's time to take another look.

[crschmidt.livejournal.com]

I should have listened to this a lot longer ago: I just got to using Selenium today, and I'm kind of in love.

I'm sort of assuming it's gotten better in the past year and a half, but the ease I was able to get started with this is pretty incredible.

This (http://crschmidt.net/tmp/Selenium.html) is a mini-tutorial on using Selenium IDE to create tests; not 'published' yet, but something I'm putting together for our internal people and will publish once I'm done.

[jducoeur]

Oh, yeah -- there have been definite improvements over time. I'm still using it on CommYou, albeit with the Java version of Selenium-RC now instead of the .NET version.

Mostly, they've worked on those rough edges, so it tends to work a bit more smoothly, with fewer horrible bugs than we hit in the first version. We actually originally had to take a point build, because the "stable" version had a critical bug we couldn't work around. But the current releases are pretty solid...

[crschmidt.livejournal.com]

Well, I have to work off a snapshot of RC because the current release doesn't support FF3, but that wasn't too horrible to figure out, thankfully. And now I've written it down s othat othere people within MetaCarta don't have to find out the way I did :)