PreviousNext…

Fun with SOAP

Introduction

So, in addition to yer common-or-garden Notes and Domino development (joking!) of late, I’ve also been tinkering with consuming unusual web services. This has been fun in a twisted kind of way, although there has also been a fair amount of frustration. I think it’s fair to say that the technology behind web services, both provision and consumption, still has a long way to go when it comes to doing stuff with SOAP. It isn’t simple inter-operability, not by a long chalk. In a Skype conversation today, Julian coined the perfect name for a user-group of IT types frustrated with some of the later developments in the wonderful world of software—but I digress, perhaps more on that later!

Rather than an editorial rant (far too many of those out there, and I’m as guilty as the next man), I thought it would be useful to piece together some resources for anyone else looking to branch out into the wonderful world of web service development. This list is random, by no means even remotely exhaustive, and has an OS X bent. It is not specifically aimed at Dominophiles, but should be as useful to them as anyone else. Note also that I am focussing on the development of web service clients, in Java.

Tools: web service libraries

I started off coding clients by having MyEclipse generate stubs for me, and going from there. This is useful, and saves stacks of time, as you can imagine (assuming a valid WSDL file is at your disposal of course). MyEclipse makes use of the XFire libraries for all its web service-related whizziness. I’m told that XFire has some useful stuff over and above the functionality found in Apache Axis, but I remain to be convinced. To that end, I have also twisted MyEclipse into generating stub classes using the venerable WSDL2Java tool, found in the Axis package (more on that in a bit. It’s actually very easy).

(Whilst we’re on the subject of Axis, I should note that I’m using an older version, 1.2, which I know plays nice with Domino 7. Don’t bother with any of the Apache mirrors, they all throw 404s when you ask; I had success with http://archive.apache.org/dist/ws/axis/1_2/).

Tools: XML

When you’re generating clients from WSDL files, you might want to give the WSDL itself a bit of a going-over. All sorts of gremlisn can creep in, and whilst MyEclipse and other IDEs have some fairly decent validators, they still miss stuff. The lowest common denominator is to ensure that the XML itself is well-formed. I was surprised to note that the MyEclipse validator (itself driven by XFire) missed out on some stuff that really broke things further down the line—to wit, the XML developer’s sworn enemy, white space. Being a Mac head, I drop into Terminal at times like this, and run the excellent UN*X utility, xmlwf. As its name implies, this tool tests XML for Well-Formedness. It picked out a few things for me that I simply missed; (1) trailing spaces in some WSDL element names (which in turn throw errors in IBM’s Web Service Explorer tool, and kill any test SOAP responses that are generated from the elements) and; (2) duplicate prologs.

Eh? Prologs? (We call them “prologues” here in Blighty don’t you know…) Well, you will find reference to these all over the shop when it comes to Axis and co. breaking into a sweat over SOAP messages. Basically, a prolog is the bit of XML that says this:

<?xml version="1.0" encoding="utf-8"?>

If you have more than one of these in your SOAP message, or you have characters in the message that appear before the prolog (even whitespace), you could well start hitting a shed-load of errors in your code. What’s more, these whitespace-related errors often don’t arise in some of the usual web service tools (I still haven’t worked that one out).

So, be warned, take nothing for granted, and test everything that you possibly can, no matter how obvious it might seem. For example, how, you might ask, could anyone end up with duplicate prologs in their SOAP message? Well, that’s easier than you’d think (happened today actually). Some web server set-ups, in a bid to be helpful when rejecting posted messages, decided to include the body of the original request in their error report. If that error report is itself a SOAP message, BANG, you’re stuffed.

Web service client errors

To summarise, and for the benefit of Google, if you have whitespace or characters preceding your XML prolog, you may well encounter a SAXParseException in your client code, which reports The Processing Instruction Target Matching "[xX][mM][lL]" is Not Allowed. If you have a doubled-up prolog in your message, then expect to encounter errors like XML or text declaration not at start of entity or Content is not allowed in prolog.

If you have trailing spaces in one or more element names in your WSDL, then expect to get DOMException reports telling you that you have an INVALID_CHARACTER_ERR.

Tools: getting SOAPy

So this is all well and good, but how does one go about actually seeing what one is posting, SOAP-wise? Well, this is where two types of tool come into play, both of them invaluable. For one thing, you will want a web service test tool of some description. There’s no point hacking together client code if the source WSDL is inherently buggered (technical term—stop me if my language becomes too high-falutin’). Secondly, you will want some kind of packet sniffer to really track down the requests and responses going on behind the scenes. Here’s what I’m using:

soapUI is a splendid, free Java-based tool for testing web services, WSDL, and lots more. You can also monitor your SOAP requests and responses directly with it. I like it very much, and the Java / Swing UI means it runs on pretty much any platform. I downloaded the distro, unzipped it, and placed the resulting folder in my /Developer/Applications directory. Rather than bash out a long command in Terminal to run it each time, I just type soapui. Easy. To do this, set up an alias in your bash profile++ thus:

alias soapui='/Developer/Applications/soapui-2.0.2/bin/soapui.sh'

Note on the Mac / Linux you’re in UN*X land, so once you’ve put the soapUI folder where you want, be sure to chmod +x the soapui.sh script to make it executable (UN*X page on wiki for more!). If you’re a WIndows user, you just need to run the supplied batch file.

I won’t go into how you use soapUI here. The tool is pretty self-explanatory, and its site is excellent. If you want to know more though, drop me a line. Right, next…

Tools: sniffing

There are loads of utilities out there for packet sniffing. Ethereal is probably the best-known, and is certainly very complete. I’ve used it in the past, and like it a lot. The tool of choice on Kinky at the moment though is a splendid wee application that goes by the name of Eavesdrop, by Eric Shore Baur. It’s very easy to use: simply click “Start capture”, enter your password, and away you go. The tool supports the use of simple filters. In my case, I was tracking communication between my machine, and a remote Apache server on port 20700. The filter text for this in Eavesdrop is straightforward: port 20700. Nice!

So, the app is listening, you invoke your code, and there you go: a packet you can open and inspect showing (hopefully) an HTTP Post request, followed by a lovely HTTP response. Smashing.

Code generation

Earlier on, I mentioned simple code generation, i.e. stub classes with which to consume your web service as defined in a WSDL file, and how you can generate these in MyEclipse without relying on the in-built XFire wizards—in fact, you can trigger this stuff in normal Eclipse too. So long as the axis jar files are in your project classpath, this will work.

Some time back, someone came up with an Ecliupse plug-in which invokes WSDL2Java. It doesn’t work in modern versions of Eclipse, but to be honest, you really don’t need it. Instead, simple create a run configuration in Eclipse as follows:

Main class:
org.apache.axis.wsdl.WSDL2Java

Program arguments:
/<PATH TO YOUR WSDL FILE>/YOURFILE.wsdl -v -o src

-v can be omitted, it just means you get verbose output in your Eclipse console. -o src means, in this instance, that the generated files should be output to your project’s src folder. And that’s it!

Conclusion

This has been a somewhat rambling document—I may shift it to the wiki at some point—but I hope it has been useful.

Further reading

Comments

  1. hi there,

    eavesdrop. is it only OS X ? I tried downloading it -- has a .DMG extension -- as a windows/linux user, don't understand this file format.

    is there something equivalent on windows ?

    thank you,

    BR,
    ~Aanjan bacchu#
  2. I have done 2 web services projects with domino as a consumer and SAP workflow and Factiva.com services as producers. Ok. Only SAP workflow actually uses SOAP.
    I had quite a good experience with going a bit more low level.
    XMLSpy has good features to generate sample input/output message files from a WSDL-URL.
    This has been a huge time saver, because figuring out the structure of the soap messages from the wsdl is truely complex.
    Then I've used my own framework with jakarta-commons HTTP Client and a self written framework based on SAX and some introspection to convert the response from the producer into java beans. Using xpath capabilities of JDom or XOM would have been the better solution than my own framework, but it hasn't been that hard anyway.
    This strategy can be used for REST style webservices also. There are lots of webservices out there, which does not use SOAP. Factiva.com for example.

    People say that Axis2 and XFire offer HUGE performance gains. Thats certainly an important point. Axel#
  3. Entering Sosno in search field of http://www.parleys.com gives 3 interesting results of a SOAP critical point of view. SOAP has advantages and disadvantages when compared with other webservices architectures. I've found those talks very informative. Axel Janssen#
  4. I have made several webservice projects, and after using several other applications to read the WDSL and trying to create requests I found an application that did everything I needed.

    SoapUI automatically reads the WDSL, and you can create soap requests from the application, and it reveals the soap request and response from the server.

    This saves alot of time generating soap requests, since I can just ask the app. to update the WDSL etc. :)

    The app is available at http://www.soapui.org/

    Sincerly MaxMax#
  5. Thanks Max. I mention soapUI in the article. It’s very good, I agree.Ben Poole#

Comments on this post are now closed.

About

I’m a software architect / developer / general IT wrangler specialising in web, mobile web and middleware using things like node.js, Java, C#, PHP, HTML5 and more.

Best described as a simpleton, but kindly. You can read more here.

";