PreviousNext…

Seven testing tips

Background

I can hear the groans already: testing. It’s up there with documentation as one of the banes of a developer’s life. Most of us like to solve problems, tinker with code. Some of us like to play with the user interface, and conjure up dreamy-looking dialog boxes. But I imagine that testing is something very few of us care for.

Why? It takes effort, and it can be pretty tedious, but in this day and age, it really has to be done. Allow me to outline a few reasons for thorough testing:

  • Happy users: the Holy Grail!
  • Happy support staff: indeed, this could be you. Do you want to deal with App A’s bugs all day long whilst trying to code the latest and greatest App B? What about having to deal with bugs from other developers? Annoying isn’t it?
  • Personal pride. Do you want to be known as a good developer? Shipping iffy applications is not the way to do it. Bugs are remembered for longer than nifty features.
  • A little early effort saves head-aches down the line
  • Producing good quality stuff: you’d be the first to complain if your coffee maker insisted on going wrong every morning — why should software be any different?

So, on to the practicalities: how can one get to testing without too much tedium, and more importantly, how can one ensure that this testing will be effective?

1. Write comments

In my last article, 7 Easy Tips for Creative Domino Developers I discussed keeping code “focused and simple”. When faced with something reasonably complex (and therefore prone to bugs and errors), the easiest way to approach the problem is to sit down, really think about what it is you’re trying to achieve, and then write the comments. Keep them short, clear and above all, accurate. When you come around to actually writing code, trying things out, and debugging, keep referring to those original comments. This way you keep to the requirement. I’ve lost count of the number of times I’ve written fiddly stuff over the course of a couple of days, with my code gradually wandering more and more from the point. It’s such an easy trap to fall into, yet equally easy to avoid.

There’s another reason for thinking about your code and writing the comments up front of course. Have you ever hit an impasse with some convoluted agent or similar, and in explaining the problem to a colleague, inadvertently hit upon your solution? I know I have: it’s a known phenomenon we comment upon at work. Well, it turns out that there’s a reason for these frequent bouts of sudden clarity. “Doing” and “expressing” use different parts of one’s brain. So, writing down a “header” for your code before the lines of Java or script come flowing can actually be really useful in the development stage.

So, make sure you keep the comments succinct, accurate, and up-to-date.

2. Test-driven development

Ah yes, a phrase you’ve no doubt heard bandied around. It’s not an especially new idea, but has gained currency through the promotion of so-called “extreme programming” (XP), a process which encourages concepts like pair-programming, iterative software releases, user stories and, above all, an approach based on “test first, code second”. This dove-tails neatly with my tip about using comments, and entails writing the test for your code before you’ve even started on the code itself, if that seems an appropriate way to go.

It sounds odd I know, but let’s think about it: if you can’t visualise how you’re going to test something, then you probably need to re-assess what it is you’re trying to achieve, and set out your code accordingly — chances are you’re not in a position to write anything remotely complex that will function properly unless you can test it.

A second benefit to test-based coding is that your code will be tested at the “micro” level: each component has undergone some testing before it becomes part of the larger picture, which can save a whole lot of time and heart-ache. The more dependencies you introduce, the harder it is to reproduce and pin-point strange little bugs that creep in.

By way of example, recently I wrote a LotusScript routine for document validation. The basic code checked whether specified fields had been populated or not, and collated all omissions into one prompt box for the user at the end of the validation cycle.

Now, what about other forms of validation? How would you work those in to a custom routine? You might want to check that a field contains only alpha-numeric characters, or perhaps you need to validate an email address is correctly formed. Maybe you need to assess typed-in URLs, or ensure that a field is of a certain length.

Pre-tested modular code makes adding this kind of stuff a snap. For each scenario, if you write a test routine, code a routine or function accordingly, and then test it, you can be reasonably comfortable that subsequent additions don’t break anything already there. This is especially do-able with custom classes in LotusScript and Java — more on this below.

You can read more about extreme programming with regards Domino in Thomas Duff’s article, Domino and XP (No, Not That XP)

3. Time machines

Source / version control is your friend: please use it! It doesn’t have to be sophisticated, just something that works for you and enables you to “roll back” your designs at any time with the minimum amount of effort.

It’s all very well to say that you should just fix one thing at a time when you’re in the coding / testing / debugging phase, but who does that? It’s all too easy to tweak a variable here, a function there, and — blam! — you’ve broken something. Documenting the big changes, and taking frequent back-ups of your code will help, even if that just means exporting LotusScript / Java source files from your databases every now and again. My previous article mentioned maintaining a code library, and this is useful here too: keep snippets of code you’ve created and discarded in a library rather than clogging up production designs with commented-out code and experiments.

Versioning and code management is an accepted practice in other programming disciplines, so why should Notes and Domino be any different? Perhaps the most well-known versioning system is Concurrent Versions System (CVS). I’d love to see a Notes / Domino solution that leverages the power of CVS or Subversion, but suspect anyone would have their work cut out writing such a beast. In the meantime, a number of third party products exist to help with Domino source control, so you may wish to investigate some of these. They range from simple code roll-back to fully-blown server / client version control with design locking and so forth. I recommend you look at these products before the über-geek in you decides to code your own CVS / DXL interface-based source control system (or is that just me? Ahem).

4. Code defensively

Think it will never happen? Think again. An oft-quoted rule of thumb in matters XML borrows from the late Jon Postel: “Be liberal in what you accept, and conservative in what you send.” (Postel’s Law).

I extend this to programming in that you should assume nothing. Say for example you have some LotusScript to access a specified NotesDocument object. You might do this:

	Dim session As New NotesSession
	Dim db As NotesDatabase
	Dim vwLU As NotesView
	Dim doc As NotesDocument
	Dim strKey As String

	Set db = session.CurrentDatabase
	Set vwLU = db.GetView("MyLookup")
	Set doc = vwLU.GetFirstDocument
	strKey = doc.GetItemValue("foo")(0)

	getAnotherDoc strKey

Looks OK doesn’t it? But there are way too many assumptions going on in here. First of all, the code assumes that the view exists in the database, and we can get a handle on it OK (i.e. it’s not been inadvertently flagged as private, or had ACL security applied to it).

Secondly, the script assumes that the view is populated, so that we can grab the first document in it OK. Again, the view may be empty. It may be an unpopulated folder. There may be data in there, but we don’t have access to it.

Finally, even if we get the document, there’s no guarantee that strKey will get populated, and then all hell will break loose in the getAnotherDoc() sub-routine.

All of these conditions will result in myriad “Object Variable Not Set” errors. And we don’t like them sir, no we don’t.

So, as Pragmatic Programmers say: code defensively: check that variables and object variables have been instantiated, especially if you depend on them further down the line. If something goes wrong, ensure that it fails gracefully. This kind of work is tedious, and means extra typing, but your code is worth that surely?

5. Error handling

All too often decent error trapping and handling constitutes the most over-looked aspect in Notes and Domino projects. Repeat after me: On Error Resume Next is rarely the answer!

Coding proper error handling into one’s application could quite easily form the basis of a whole series of articles. Suffice it to say, you should be looking at your code with a critical eye, and looking out for common “gotchas” as described in tip number four above — these will save a lot of time come testing and debugging:

If you’re not routinely checking for Object Variable Not Set and the like, you should be. One way you could do this is as follows:

	On Error Goto lblErrs

	Dim session As New NotesSession
	Dim db As NotesDatabase
	Dim vwLU As NotesView
	Dim doc As NotesDocument

	Const ERR_NO_VIEW = "A required view could not be found."

	Set db = session.CurrentDatabase
	Set vwLU = db.GetView("MyLookup")

	If vwLU Is Nothing Then
		Error 1000, ERR_NO_VIEW
		Exit Sub
	End If

	[…]

	lblErrs:
	' // Custom error handler
	errHandler Err, Error$, GetThreadInfo(XXX)
	Exit Sub

Now, what are we doing here? First of all, the whole If… Then Error… bit is about “throwing” your own error. I prefer this approach to others in most scenarios because it’s a little “cleaner” than handling the error (null object or whatever) within the code itself: I like to parcel everything off to the appropriate place, in this case a custom error handler.

In the code example above, we’re sending three pieces of information to this error handler. Err is the error code thrown by Notes: either our custom one above (“1000”) or a familiar internal one (e.g. “13” for “Type Mismatch”). Error$ is the error message generated — again, either our one (via the ERR_NO_VIEW constant) or Notes’ own. Thirdly, I’m using something that doesn’t seem to get used much: GetThreadInfo, which can provide some handy detail for tracing errors. The XXX element is nonsense in the example above, you should replace this with the appropriate constant to get the piece(s) of information you require:

Integer constant Meaning
LSI_THREAD_LINE Current line number
LSI_THREAD_PROC Name of current procedure
LSI_THREAD_MODULE Name of current module
LSI_THREAD_VERSION LotusScript version

(Other constants are available: these are the main ones I use. Note also that to use these constants you need to include lsconst.lss in your script).

Like I say, error handling is a whole topic in itself. There are some excellent articles out there that you may wish to delve into. I particularly recommend the two-part series, Debugging LotusScript by IBM’s own Andre Guirard.

6. Object oriented code

Clearly this only applies to any Java and LotusScript you may have in your application, but it bears detailing, especially when considered with my earlier tip regarding test-driven development.

Nowadays I often find myself placing script in custom classes rather than having lots of subs and functions everywhere. This isn’t suited to everything of course, but by way of example, the custom validation code touched-upon above was implemented as a class in LotusScript. The class, with its “constructor” (Sub New in LotusScript) performed the core validation, whilst additional public sub-routines and functions allowed the checking of specific fields with regards length, invalid characters, and so on. Private routines and functions are also used within the class to perform “housekeeping” — tracking the number of errors hit, collating error messages, and so forth.

This approach has a couple of benefits for the developer:

  1. The main body of the code is all “tucked away” in the class, so agents and things like Querysave can be kept clutter-free.
  2. Once the basic class has been tested, the developer can add new sub-routines without worrying too much about breaking anything. For example, my validation class tracked the errors arising and which field in the user interface to take the user to once validation had failed. So, when I added a new method to check for invalid characters, the basic error-handling and reporting to the user was already in place and tested. This principle extends to sub-classing rather than simply adding code to the base class — even better.

Smashing! Now, when this approach should be used is up to you: it’s not always appropriate and may well be over-kill. In fact, I ought to touch on the down-side of custom classes in Notes and Domino:

  1. Notes has a 64 Kb limit on code in a single script library event: bear this in mind when designing elaborate object models
  2. omino Designer is not the greatest integrated design environment (IDE) for object oriented programming.

To summarise therefore, consider their use carefully, but classes can lead to some pretty quick code / test cycles, and that can’t be bad.

7. Consider testing within your actual application

What do I mean by this? Well, some people out there know I’m a Mac-head (I know, and a Notes developer. What can I say? I’m odd), and as a result I use a web browser called Safari. With a simple tweak, Safari is able to display a “Debug” menu thus:

Apple's Safari web client showing its 'Debug' menu As you can see, this menu lets the developer or tester do all kinds of things to the application, testing and tracking a variety of scenarios.

This led me to thinking — and the idea isn’t at all original, but there you go — what about implementing a similar thing in a Notes application? Here are some suggestions for this kind of thing:

Use a “Developer” access control list (ACL) role

This could be used to permit access to otherwise “hidden” features and options in outlines, form design, action menus, and so forth. In a complex workflow application you might want to reveal all the hidden fields in a form to developers for trouble-shooting.

Testing aids

Consider adding some agents, or similar code tweaks, to your databases which help you out. Typical applications include changing fields on the fly for trouble-shooting and re-setting other data for test purposes.

At the simplest level, changing fields comprises some @formula in an agent or toolbar icon which prompts the developer / tester with a list of fields in the currently-selected document, allowing them to then change any field, in terms of both data type and underlying value(s). You can find some example code by Chad Schelfhout in the LDD Sandbox. My suggestion is that you add this code to a smarticon, and use it forever more!

Other test aids could include scripts which generate large numbers of documents or fully-fledged code to benchmark more complex processes using things like the NotesTimer class.

Indeed, it’s possible to develop a pretty sophisticated suite of test tools, as Nik Shenoy can attest. Nik has developed a beta “LSUnit” script library, taking the ideas behind tools such as the open source Java-based JUnit library and applying them to LotusScript. You can find more on LSUnit at Nik’s site.

The news isn’t all good though. Whilst Notes 6.x makes testing like this fairly painless — because you can compile all LotusScript from one menu command –releases prior to 6 don’t allow this. This is a pain when you make a change to a class tested in an agent, especially if that class is nested in a hierarchy of script libraries.

Logging

Logging is great. You may only wish to log complex scheduled agents in your final production system, but logging just about everything in your other agents can really help during the development phase. There are a few solutions out there, including LotusScript’s very own NotesLog class, and all are up to the job.

The best solution is one that allows error and event logging, with a choice of destination for that log: email, another database, text file, etc. Most solutions offer this, so it comes down to personal preference — and whether you require logging for both LotusScript and Java. A couple of systems you might want to consider are:

Combining your error logging with the aforementioned GetThreadInfo function results in a powerful debugging solution: you’re effectively getting a “stack trace” which helps you pinpoint where the error’s occurring. In anything remotely involved, this is a God-send.

Conclusion

I hope this has provided some food for thought. Testing is tedious, so anything that can alleviate the pain has to be worth-while — especially if it means writing a wee bit more code! I’ve lifted some ideas from the wider world of programming and tried to apply them in a sensible way to Notes and Domino development.

A few of the ideas may be impractical for you, others simply too much work for some applications, but all bear thinking about — the time has long gone when Notes was a really basic rapid application development (RAD) platform: robust business solutions are the order of the day!

Further reading

Comments

  1. Ben -
    this is a great write-up! As an admin that develops many his own tools (not a programmer), you have given me a couple great ways to hopefully correct some current headaches.
    Thanks a million!!! =)

    ChrisChris LeRoy#
  2. Ben

    What about Asserts? I have my own lotusscript assert routine that I pepper the code with …very simple.

    spuggy

    assert(spuggy)#
  3. Oooh yes, very useful.

    Check out LSUnit, it has some assertion testing in there, amongst other things. As you say, nice and straightforward to implement, and makes you look triffic to boot ;-) Ben Poole#
  4. Ben, This is a wonderful article! Lots of good tips, and links to more info and freebies. I often wonder why folks sometimes go for the 'quick and dirty' solution -- c'mon, we've all learned that most apps end up being long-term. And with the proper templates, it really doesn't take too long to develop robust applications vs the time it takes to slop something out. I especially like your suggestion to *start* with the comments. Good stuff :-)Joe Litton#
  5. ben,

    joe recommended this article to me over cafe con leche the other day and i must say i'm not sorry i read it! was meaning to anyway of course… good stuff sir! thanks for sharing it.jonvon#
  6. Hi

    How do you use the GetThreadInfo(LSI_THREAD_MODULE) for something useful? It returns a HEX number, but how can you use that?

    Best regards
    Hans-JørgenHans-Jørgen Pagh#
  7. Ben, great article. Thanks for sharing it. I'd recommend Ben's suggestions here to anyone doing Notes development. Just because Notes isn't your standard everyday object oriented IDE, that doesn't mean we should code like script kiddies. Well done.Keil Wilson#

Comments on this post are now closed.

About

I’m a software architect / developer / general IT wrangler specialising in web, mobile web and middleware using things like node.js, Java, C#, PHP, HTML5 and more.

Best described as a simpleton, but kindly. You can read more here.

";