Another WS-Star candidate for Event Driven Architecture - WS-Eventing

In my last post I had a look at Web Services Notification (WSN). In this post I'll see what Web Services Eventing (WS-Eventing) has to offer. Where WSN is a formal OASIS Standard Web Services Eventing is currently only a Member Submission to W3C back in March 2006. The list of vendors behind the specification are most of the big players like for WSN with the big difference that Microsoft is participating.

The specification tries to meet the following requirements:

• Define means to create and delete event subscriptions.
• Define expiration for subscriptions and allow them to be renewed.
• Define how one Web service can subscribe on behalf of another.
• Define how an event source delegates subscription management to another Web service.
• Allow subscribers to specify how event messages should be delivered.
• Leverage other Web service specifications for secure, reliable, transacted message delivery.
• Support complex eventing topologies that allow the originating event source and the final event sink to be decoupled.
• Provide extensibility for more sophisticated and/or currently unanticipated subscription scenarios.
• Support a variety of encoding formats, including (but not limited to) both SOAP 1.1 [SOAP 1.1] and SOAP 1.2 [SOAP 1.2] Envelopes.

I haven't done a formal comparison but there seems to be quite a overlap between Web Services Notification and Web Services Eventing.

An attempt for convergense

It isn't the first time that we see two competing standards in the WS-Star stack, not that it makes it any better. I found that at the same time that the specification was submitted to W3C, the four companies Intel, Microsoft, IBM and HP put out a a letter of intent with the title "Toward Converging Web Service Standards for Resources, Events, and Management":

HP, IBM, Intel and Microsoft plan to develop a common set of specifications for resources, events, and management that can be broadly supported across multiple platforms. The parties will do this by building on existing specifications and defining a set of enhancements that enable this convergence. In many scenarios, vendors and customers building solutions using Web services will find that the existing specifications support their scenarios. Vendors and customers may use the new specifications and functions when needing the common capabilities.

The full text [PDF] can be found on intels website or in googles cache from Microsofts website. To my knowledge this never materialized, but maybe this is what turned up as a new working group at W3C.

Web Services Resource Access (WS-RA)

Just as I had concluded that WS-Eventing had stranded at W3C I found that W3C in november 2008 launched the Web Services Resource Access Working Group, and from the charter the scope is:

The Web Services Resource Access Working Group is chartered to standardize a general mechanism for accessing and updating the XML representation of a resource-oriented Web Service and metadata of a Web Service, as well as a mechanism to subscribe to events from a Web Service. The following list of features is intended to focus the work of the Working Group and ensure its timely completion:

1. The definition of basic Create, Read, Update, Delete (CRUD) operations that provide capabilities to create, read, update and delete a Web Services XML representation and the binding of these operations to SOAP.
2. A mechanism to allow a requester to retrieve metadata, or references to metadata, related to a Web Service and a mechanism to allow embedding of such metadata, or references to such metadata, in an Endpoint Reference (EPR). The defined capability must provide for retrieval and embedding of specific items of metadata such as WSDL, XML Schema and WS-Policy. The referencing of metadata must allow for both EPR or URL style references.
3. The definition of Web service operations (e.g. Enumerate, Pull, etc…) to enable a consumer to request and manage an enumeration context, and retrieve data items from an enumeration context for a data source. These operations must define details of SOAP messages for the request and response as well as one or more filter dialects to select the data that would be sent to the consumer. The ability to work efficiently with large datasets is important.
4. The definition of Web Services operations (e.g. Subscribe, Unsubscribe, etc…) to create and manage subscriptions to events that are delivered via Web Services.
   1. The definition of one or more filter dialects, such as XPath, that can be used in subscriptions to indicate interest in messages with specific content.
   2. Mechanisms to allow a subscriber to specify the means by which an event is delivered and the definition of a push-based delivery mode.
5. A mechanism by which a Web Service can advertise, through its metadata, such as WSDL and WS-Policy, the capabilities it supports from among the features defined by the Web Service Resource Access specifications.

The thing that jumps into my eyes is how RESTfull this sounds!

This high speed WG i supposed to deliver the recommendations in little more than a year (June 2010), which in quite fast in terms of standarization speed. The work will build on the following submission:

• Web Services Transfer (WS-Transfer)
• Web Services Resource Transfer (WS-RT)
• Web Services Enumeration (WS-Enumeration)
• Web Services Metadata Exchange 1.1 (WS-MetadataExchange)
• Web Services Eventing (WS-Eventing)

The current editors draft of WS-Eventing is available online as promised. Judging from the issues related to WS-Eventing convergence doesn't seem to be on the agenda.

Interesting enough IBM has part in both specifications and their standpoint can be read in a mail from Steve Holbrook to the W3C maillist to show their support for WS-RA, "Re: Standardization of WS-Eventing":

IBM remains committed to use of the OASIS Standard WS-Notification 1.3 as the interoperability standard for enabling a broad range of applications that include publish/subscribe and event notification in the context of business and complex event processing scenarios. However, we are supportive of the W3C standardization of WS-Eventing because we appreciate that some in the Web Services community are using it in certain domains such as device management and we are interested in ensuring that such a standard compose effectively with other Web services specifications including WS-Notification.

Conclusion

I'm not in a position to really conclude anything except from the obvious that I would have preferred to have one widely supported specification instead of two lightly adopted ones. Ih this I agree with Gartners conclusion from back in 2006, "WS-Notification Standard Ratified by OASIS Still Needs Work";

The delay in reconciling WS-Notification and WS-Eventing shows that the Web services standardization process has lost momentum. Standards have remained static for the past year, permitting only primitive interoperability.

A WS-* offering for Event Driven Architecture - Web Services Notification (WSN)

From my point of view Event Driven Architecture (EDA) is part of SOA and typically run with push(like) setups (pub/sub). Most of the SOA talk has been on request/response oriented setups, probably cause by the common usage of HTTP and no agreed upon interfaces for subscriptions/topics. In this blogpost I'll have a quick look at the OASIS standard Web Services Notification (WSN).

Web Services Notification (WSN) setup

The committee was started back in 2004 when the proposed standard was submitted to OASIS:

The OASIS WSN Technical Committee will coordinate efforts with the OASIS Web Services Resource Framework (WSRF) Technical Committee (also announced today). Members of the OASIS WSN Technical Committee plan to conform their work to the implied resource pattern specified by WSRF and will make use of WSRF specifications concerning lifetime and properties.

So it has a close relation with Web Services Resource Framework (WSRF). The homepage for the OASIS Web Services Notification (WSN) TC has the following overview:

The purpose of the Web Services Notification (WSN) TC is to define a set of specifications that standardise the way Web services interact using "Notifications" or "Events". They form the foundation for Event Driven Architectures built using Web services.

These specifications provide a standardized way for a Web service, or other entity, to disseminate information to a set of other Web services, without having to have prior knowledge of these other Web Services. They can be thought of as defining "Publish/Subscribe for Web services".

Two years later in October 2006 they announced that its members had approved the WS-BaseNotification, WS-BrokeredNotification and WS-Topics specifications as OASIS standards. A simpel and in my opion relevant indicator of chance for succes (for a ws-* standard) is which companies approved the standard, and here it was:

• AmberPoint
• CA
• Fujitsu
• HP
• Hitachi
• IBM
• Oracle
• SAP
• Sonic Software
• Tibco
• webMethods

There are certainly a lot of big companies supporting (with a degree of holeheartness), but in my view Microsoft is missing and to some degree Sun.

Implementation

If a standard is to be used theres a need for (widespread) implementation. OASIS has as part of the standardization process a requiment of several practical implementations (like ex. the w3c). At the time when the standard was approved is was (taken from the announcement):

Amberpoint, Fujitsu, and IBM have all implemented WS-Notification either internally or externally (in keeping with the consortium's requirement that at least three parties assert successful implementation prior to OASIS Standard balloting). Fujitsu's implementation is available as open source tool through unicore.forge.net.

The reference to the Fujitsu open source implementation isn't active anymore but the Apache Muse project does:

The Apache Muse Project is a Java-based implementation of the WS-ResourceFramework (WSRF), WS-BaseNotification (WSN), and WS-DistributedManagement (WSDM) specifications. It is a framework upon which users can build web service interfaces for manageable resources without having to implement all of the "plumbing" described by the aforementioned standards. Applications built with Muse can be deployed in both Apache Axis2 and OSGi environments, and the project includes a set of command line tools that can generate the proper artifacts for your deployment scenario.

The implementation of WSN is only partial, with only WS-BaseNotification of the three specifications that make up the standard.

Update: IBM has support for WSN in their WebSphere suite.

The standard for Web Services Notification (WSN)

To get a more detailed view of the features offered by the three specifications:

• WS-BaseNotification 1.3 OASIS Standard (In PDF it's 68 pages)
• WS-BrokeredNotification 1.3 OASIS Standard (In PDF it's 43 pages)
• WS-Topics 1.3 OASIS Standard (In PDF it's 40 pages)

let's look at their goals and requirements.

WS-BaseNotification

Goal

The goal of WS-BaseNotification is to standardize the terminology, concepts, operations, WSDL and XML needed to express the basic roles involved in Web services publish and subscribe for notification message exchange.

Requirements

Must support resource-constrained devices.

The specifications must be factored in a way that allows resource-constrained devices to participate in the Notification pattern. Such devices will be able to send information to, and receive information from Web services, without having to implement all the features of the specifications.

Must provide runtime metadata

There must be a mechanism that lets a potential Subscriber discover what elements are provided for subscription by a NotificationProducer, and in what formats the subscription for notification can be made.

WS-BaseNotification must be independent of binding-level details

Transport protocol details must be orthogonal to the subscription and the delivery of the notifications, so that the specification can be used over a variety of different transports.

Must allow for Message Oriented Middleware implementations.

The design of the WS-BaseNotification must allow a service that is acting as a NotificationProducer to delegate its implementation of WS-BaseNotification semantics to a Message Oriented Middleware provider.

Relationship to other WS-* specifications

WS-BaseNotification must be composable with other Web services specifications.

WS-BrokeredNotificatio

Goal

The goal of WS-BrokeredNotification is to standardize message exchanges involved in Web services publish and subscribe of a message broker. The overall requirements of WS-Notification are presented in [WS-BaseNotification]. The following section lists the specific subset of those objectives realized by WS-BrokeredNotification.

Requirements

Must allow for a notification broker as an intermediary.

A NotificationBroker is an intermediary Web service that decouples NotificationConsumers from Publishers. A notification broker can relieve a Publisher from having to implement message exchanges associated with NotificationProducer; the NotificationBroker takes on the duties of subscription management and distributing Notifications on behalf of the Publisher. It implements NotificationProducer interface. It may implement SubscriptionManager or may delegate the subscription management work to another component.

Must allow for federation of brokers.

It must be possible to build configurations with multiple intermediary broker services in a dynamic fashion. This specification must allow for a variety of broker topology usage patterns. Among other things, these allow for greater scalability and permit sharing of administrative workload.

Must provide runtime metadata.

There must be a mechanism that lets a potential Subscriber discover what elements available for a subscription are provided by a NotificationBroker, and in what formats the subscription for a notification can be made.

Must conform to WS-BaseNotification.

A NotificationBroker must support required message exchanges defined by the [WS-BaseNotification] specification. It must conform to the NotificationProducer and the NotificationConsumer interfaces defined in WS-BaseNotification.

WS-BrokeredNotification must be independent of binding-level details.

Transport protocol details must be orthogonal to the subscription and the delivery of the notifications, so that the specification can be used over a variety of different transports.

Must not exclude non-service producers and subscribers.

WS-BrokeredNotification design must not exclude a non-service entity to deliver a notification message to a NotificationBroker. It must not exclude a NotificationBroker to send a notification message to a non-service consumer.

Must provide publisher registration.

WS-BrokeredNotification must define standard message exchanges for registering a NotificationPublisher with a NotificationBroker.

WS-Topics

Goal

The goal of the WS-Topics specification is to define a mechanism to organize and categorize items of interest for subscription known as “topics”. It defines a set of topic expression dialects that can be used as subscription expressions in subscribe request messages and other parts of the WS-Notification system.

Requirements

Must support resource-constrained devices.

The specifications must be factored in a way that allows resource-constrained devices to participate in the Notification pattern. Such devices will be able to send information to, and receive information from Web services, without having to implement all the features of the specifications.

Must permit transformation and aggregation of Topics.

It must be possible to construct configurations (using intermediary brokers) where the Topic subscribed to by the NotificationConsumer differs from the Topic published to by the NotificationProducer, yet Notifications from the NotificationProducer are routed to the NotificationConsumer by a broker that is acting according to administratively-defined rules.

Must permit non-centralized development of a topic tree.

It must be possible for actors to define additional topics based on existing topics without requiring coordination with the actor responsible for creating the topics that are being built on.

Conclusion

If I were to spot the "Achilles' heel" of the WS-* stack, it must be the lack of a standard to really replace the classic messaging setups like ex. JMS, though that was what many set out for. It took an incredible long time to finally get something like WSN and even more importantly WS-RX. And that is just the standards - then you have to wait for at product near you to support it! In the meantime some has lost confidence (and maybe gained productivity) but using more propriotairy standards making it even harder to gain critical attraction.

Doing a web search for wsn gives far fewer interesting hits than I would have liked to see. This indicates to me that though WSN has alot to offer the real usage is strongly limited.

A Demo WSDL for part of OIOUBL Simpel Ordering

I want to get back on track with my WCF certification and the best thing to do seems to be rolling a service myself. I've had this post in draft for almost two years now so it's about time i finalize it! To save myself from yet another uninspiring hello world example I've decided to base it on a simple document flow that's part of a basic OIOUBL process " OIOUBL BASPRO - Basic procurement Cycle [PDF]. This will not only give my webservice a touch of the real world but I'll also update my knowledge of OIOUBL slightly.

Here's a couple of diagrams:

There's a complete package to download from oioubl.info and this one is called Basic procurement Cycle (Ref. S03) [BASPRO.zip].

I'll develop this WSDL based on the Contract First principle - that's based on the schemas (and not generate it from code), but where do I find the XML Schemas for OIOUBL? The answer is on the introduction page to OIOUBL in the section "The relationship between OIOUBL and UBL 2.0" (my strong):

As mentioned above, OIOUBL is a subset of UBL 2.0. This means that UBL 2.0 contains a number of business documents in addition to the ones in OIOUBL. For example international transportation documents. These additional business documents are fully compatible with and may be used with the OIOUBL documents. However, no Danish language instructions or guidelines are provided for these documents.

The OIOUBL specifications are based directly upon the UBL 2.0 XML schemas. No additional XML schemas have been developed or are required for OIOUBL.

It should be noted that, for business reasons, a few elements from the UBL 2.0 schemas have been excluded in the OIOUBL customization. These elements have been judged as not applicable in Denmark, either because of specific Danish legislation, or because they are of not relevant in a Danish context. These omitted elements will not be present in the OIOUBL specifications, and if these fields were to be used they will not pass the document validator published by the National IT and Telecom Agency.

So contrary to the first Danish adoption to UBL, here no special schemas so I can use the one from the OASIS website UBL-Order-2.0.xsd. The rest is just boilerplate WSDL.

I've uploaded the WSDL SX_OIOUBLorderDemo.wsdl to digitaliser.dk (the successor of the InfoStructureBase). It's still open to anyone, and the biggest change I've found is that once you've made anything public it can't be edited! Not the file (artifact) nor the resource (metadata), the only way is a new version of the resource.

My current C# books wishlist

Since I've picked up C#/.NET again I've found that not only do I need to practice some but also lend on some of the knowledge others have built up all ready, which in one form is books. Here's my current list of C# books I would like to own (and have time to read), that I've found on Amazon.

Pro C# 2008 and the .NET 3.5 Platform, Fourth Edition (Windows.Net) (Hardcover)

C# in Depth: What you need to master C# 2 and 3 [ILLUSTRATED] (Paperback)

Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries (2nd Edition) (Microsoft .NET Development Series) (Hardcover)

CLR via C#, Second Edition (Pro Developer) (Paperback)

Effective C#: 50 Specific Ways to Improve Your C# (Effective Software Development Series) (Paperback)

More Effective C#: 50 Specific Ways to Improve Your C# (Effective Software Development Series) (Paperback)

Programming WCF Services [ILLUSTRATED] (Paperback)

Interfaces in System.Collections

In preparing for 70-536 I've been looking at the Interfaces in System.Collections (that's quite dated since the generic variants been here for a while). Looking at the Exam objective:

Developing applications that use system types and collections (15 percent)

• Manage a group of associated data in a .NET Framework application by using collections. May include but is not limited to: ArrayList class; Collection interfaces; Iterators; Hashtable class; CollectionBase class and ReadOnlyCollectionBase class; DictionaryBase class and DictionaryEntry class; Comparer class; Queue class; SortedList class; BitArray class; Stack class

I've bought Tony Northup's Self-Paced Training Kit and it doesn't even mention the Interfaces! If it's for a god reason I guess it's since in practical use the Collection Classes are used and not the Interfaces. I don't really think it's a god idea, since hopefully there's a reason for the Interfaces and one of these should be implementing own collection and Classes with collection functionality.

The MSDN Library on System.Collections has god coverage but I'm missing two things - the Interfaces as code and a class diagram. I've looked and searched but not found none, not even at the Mono Project. Here I'll present my own Class Diagram and reconstruct the 8 Interfaces in System.Collections from the available information. One last hiccup is that the MSDN Library declares Interfaces that are extended redundantly, fx. IList implements ICollection but IEnumerable is also shown in the snip like:

public interface IList : ICollection, IEnumerable

This reduandancy confuses me in trying to understand the hierarchy, it sort of disrespects the hierarchy and suggest some form of composite pattern instead (or I just missing a point). It seems like most of the documentation for C# is used this way, but I'm not totally alone with this view, at least pdavila on the Manning maillist has raised the same concern.

A Class Diagram for System.Collections

With the help of Argouml and my missing skills with I was able to create something that resembles a correct Class Diagram for System.Collections:

The one thing that strikes me the most is that the two Base Classes CollectionBase and DictionaryBase isn't used by the framework itself so why should I use it? Perhaps there is a god reason for it and maybe I'll even find it as i read thorugh Krzysztof Cwalina's blog.

The Interfaces as C# code

• IEnumerable
• Icollection
• IList
• IDictionary
• IEnumerator
• IDictionaryEnumerator
• IComparer
• IEqualityComparer

IEnumerable

The root interface carries the following description:

Exposes the enumerator, which supports a simple iteration over a non-generic collection.

    1 namespace System.Collections
    2 {
    3     [ComVisibleAttribute(true)]
    4     [GuidAttribute("496B0ABE-CDEE-11d3-88E8-00902754C43A")]
    5     public interface IEnumerable
    6     {
    7         /* Methods */
    8 
    9         // Returns an enumerator that iterates through a collection.
   10         IEnumerator GetEnumerator();
   11     }
   12 }

ICollection

Next in the Hierarchy has:

Defines size, enumerators, and synchronization methods for all nongeneric collections.

    1 namespace System.Collections
    2 {
    3     public interface ICollection : IEnumerable
    4     {
    5         /* Properties */
    6 
    7         // Gets the number of elements contained in the ICollection.
    8         int Count { get; }
    9 
   10         // Gets a value indicating whether access to the ICollection is synchronized (thread safe).
   11         bool IsSynchronized { get; }
   12 
   13         // Gets an object that can be used to synchronize access to the ICollection.
   14         Object SyncRoot { get; }
   15 
   16 
   17         /* Methods */
   18 
   19         // Copies the elements of the ICollection to an Array, starting at a particular Array index.
   20         void CopyTo(Array array, int index);
   21     }
   22 }

IList

Here the class diagram splits in IList that:

Represents a non-generic collection of objects that can be individually accessed by index.

    1 namespace System.Collections
    2 {
    3     [ComVisibleAttribute(true)]
    4     public interface IList : ICollection
    5     {
    6         /* Properties */
    7 
    8         // Gets a value indicating whether the IList has a fixed size.
    9         bool IsFixedSize { get; }
   10 
   11         // Gets a value indicating whether the IList is read-only.
   12         bool IsReadOnly { get; }
   13 
   14         //   Gets or sets the element at the specified index.
   15         Object Item[int index] { get; set; }
   16 
   17 
   18         /* Methods */
   19 
   20         // Adds an item to the IList.
   21         int Add(Object value);
   22 
   23         // Removes all items from the IList.
   24         void Clear();
   25 
   26         // Determines whether the IList contains a specific value.
   27         bool Contains(Object value);
   28 
   29         // Determines the index of a specific item in the IList.
   30         int IndexOf(Object value);
   31 
   32         // Inserts an item to the IList at the specified index.
   33         void Insert(int index, Object value);
   34 
   35         // Removes the first occurrence of a specific object from the IList.
   36         void Remove(Object value);
   37 
   38         // Removes the IList item at the specified index.
   39         void RemoveAt(int index);
   40     }
   41 }

IDictionary

The other branch is IDictionary that:

Represents a nongeneric collection of key/value pairs.

    1 namespace System.Collections
    2 {
    3     [ComVisibleAttribute(true)]
    4     public interface IDictionary : ICollection
    5     {
    6         /* Properties */
    7 
    8         //Gets a value indicating whether the IDictionary object has a fixed size.
    9         bool IsFixedSize { get; }
   10 
   11         // Gets a value indicating whether the IDictionary object is read-only.
   12         bool IsReadOnly { get; }
   13 
   14         // Gets or sets the element with the specified key.
   15         Object Item[Object key] { get; set; }
   16 
   17         // Gets an ICollection object containing the keys of the IDictionary object.
   18         ICollection Keys { get; }
   19 
   20         // Gets an ICollection object containing the values in the IDictionary object.
   21         ICollection Values { get; }
   22 
   23 
   24         /* Methods */
   25 
   26         // Adds an element with the provided key and value to the IDictionary object.
   27         void Add(Object key, Object value);
   28 
   29         //   Removes all elements from the IDictionary object.
   30         void Clear();
   31 
   32         // Determines whether the IDictionary object contains an element with the specified key.
   33         bool Contains(Object key);
   34 
   35         // Overloaded. Returns an IDictionaryEnumerator object for the IDictionary object.
   36         IDictionaryEnumerator GetEnumerator();
   37 
   38         // Removes the element with the specified key from the IDictionary object.
   39         void Remove(Object key);
   40     }
   41 }

IEnumerator

To iterate through a Collection there's an IEnumerator similar to the Java Iterator

Supports a simple iteration over a nongeneric collection.

    1 namespace System.Collections
    2 {
    3     [ComVisibleAttribute(true)]
    4     [GuidAttribute("496B0ABF-CDEE-11d3-88E8-00902754C43A")]
    5     public interface IEnumerator
    6     {
    7         /* Properties */
    8 
    9         //Gets the current element in the collection.
   10         Object Current { get; }
   11 
   12 
   13         /* Methods */
   14 
   15         //   Advances the enumerator to the next element of the collection.
   16         bool MoveNext();
   17 
   18         // Sets the enumerator to its initial position, which is before the first element in the collection.
   19         void Reset();
   20     }
   21 }

IDictionaryEnumerator

The pendant for Dictionaries (Java Map) are an IDictionaryEnumerator:

Enumerates the elements of a nongeneric dictionary.

    1 namespace System.Collections
    2 {
    3     [ComVisibleAttribute(true)]
    4     public interface IDictionaryEnumerator : IEnumerator
    5     {
    6         /* Properties */
    7 
    8         // Gets both the key and the value of the current dictionary entry.
    9         DictionaryEntry Entry { get; }
   10 
   11         // Gets the key of the current dictionary entry.
   12         Object Key { get; }
   13 
   14         // Gets the value of the current dictionary entry.
   15         Object Value { get; }
   16     }
   17 }

IComparer

There's two utility Interface where the first is for sorting:

Exposes a method that compares two objects.

    1 namespace System.Collections
    2 {
    3     // Exposes a method that compares two objects.
    4     [ComVisibleAttribute(true)]
    5     public interface IComparer
    6     {
    7         /* Methods */
    8 
    9         // Compares two objects and returns a value indicating whether one is less than, equal to, or greater than the other.
   10         int Compare(Object x, Object y);
   11     }
   12 }

IEqualityComparer

And lastly the IEqualityComparer for searching:

Defines methods to support the comparison of objects for equality.

    1 namespace System.Collections
    2 {
    3     //Defines methods to support the comparison of objects for equality.
    4     [ComVisibleAttribute(true)]
    5     public interface IEqualityComparer
    6     {
    7         /* Methods */
    8 
    9         // Determines whether the specified objects are equal.
   10         bool Equals(Object x, Object y);
   11 
   12         // Returns a hash code for the specified object.
   13         int GetHashCode(Object obj);
   14     }
   15 }