Financial Cryptography

The Baby Back Ribbed Theory of Architecture

Somebody asked me how I do Architecture, and it wasn't a question I could easily answer. Darn! I am one, but what is one, and how do I do it?

Since then, I've been doing a bit of reading, and I think I have the answer. Here it is, by way of an anecdote called Rafi's, which was a project, and also a ribs & beer place. Crack open a beer, order in some ribs, and settle in.

In 2000 or thereabouts, my team and I were sitting on the most sophisticated and working Internet payments system around. Having got it in place, we were then faced with the problem that bedevilled most all payment systems: people don't do payments, so much as, they do trade. This was a conundrum, one I wrote about in FC7, in where I claim that a financial cryptography system without a "financial application" is approximately worthless.

Some slight context. To answer the obvious question here, we already had our application, being the swapping of financial instruments for cash, and back again. Good as it was, I was looking for more. A wider complication is that many obvious applications have beartraps in them lurking for unwitting payment systems, so there wasn't an easy answer such as copy amazon or google or expedia or Rafi's ribs place. And, finally, luckily, no customer was demanding this, it was an internally-generated strategic demand, so I could take my time.

In trying to do more payments, then, the problem evolved into figuring out what trade people did, and trying to participate in more of that. In order to get a grounding in that, I surveyed how payments integrated into life. Initially, I thought about invoice cycles, because a payment is generated out of such a cycle. This seemed fairly tractable, but I wasn't comfortable with the variations. I went wider, and thought about trading of all form, the wider business exchange that goes on that generates an invoice cycle, and finally a payment.

In principle, as we'd already done financial trades, and it was simply a matter of substituting finance with some other business, walking the transaction, architecting it and implementing it.

Or so I initially thought, but it wasn't to be. Take a hotel check-in, an example I sometimes used to convince people *not* to get into the retail payments business. When you finally get to the hotel, and walk in the door, you start the process, something like this: "I'm after a room ..."

"We have singles, doubles, and the presidential suite ..." How much is the double? "100 per night" OK, I'd like three nights. "We only have doubles available for 2 nights, but we have the singles." OK, and do you have a room at the back, not facing the highway? "Ah, yes you'll be wanting the presidential suite then ..."

And on and on it goes. The point is that, while invoicing cycles include some variability, trade cycles are all variability, to the point of arbitrary, unpredictable chaos.

Examining such a process and trying to automate it presents rather special challenges. It is somewhat clear that we can create an ontology to capture all the probable paths of the hotel reception cycle. And indeed, this is what many projects tried to do: define the process, automate the use-cases. Consider flight booking systems. It's also possible to define an exceptions protocol, to catch those tricky "backside room," special meal requests or the more prosaic desire for a welcoming Carib.

But it's hard. And risky, as having built it, how do we know the customers will agree with the ontology? Building such things only makes sense if they work, guaranteed, and that's unlikely in this case.

But the real killer is that having done all that, ones grand structure is almost useless for the next business problem. That is, a flight booking system won't translate so easily to a hotel process, even though we want to offer both of them together (and many do). And it certainly doesn't have much relationship to a book drop-shipping business. Unless, that is, one believes that there is all-powerful super-meta-hyper methodology that can capture all interactions between all humans. An AP-SMH-UML, if you like, or perhaps the über-SAP. If you believe in that, just let me know how many decades you'll finance the building of it, and I'm willing to take your money :) if I can get to the head of the queue...

In the alternate, it is possible to understand the essence of trade, approximate, and find some benefits. And this is where my months of thinking started to pay out some small glimmers of inspiration.

The first thing I realised is that, a payment is but a tiny part. A bigger part is the invoicing process, which often involves a bill being delivered, checked, paid for, and confirmed. On both sides. This is an application in and of itself, it is probably 5 times bigger than a payment.

And we still haven't given it any semantics, any meaning for the end-user. Intuitively, we have to deliver meaning in order to make it reach a customer (or, academically, recall the FC7 hypothesis above). But as soon as I tried to add some sort of semantics around the invoicing, I ended up with the killer issue above: a mess of interactions of no real structure surrounding an already challenging invoicing protocol, with a payment or three tacked on. What started out as simple modelling revealed an intractable mess, which by size dominated the original mission. My finger-in-the-air estimate is this: 1% payment, 4% invoice, 95% the messages of chaos.

Logic therefore said to me that I if I could improve the 95% and reduce the chaos as it were, then this would likely dominate any efforts to improve the 4%, at a ratio of around 20 to 1! And, following this logic, the payment was now almost optional, almost vestigial. The less I thought about payments, the better.

My next glimmer was to treat trade as a series of requests and responses, back and forth. But, that didn't quite work out because many of the so-called requests were unanticipated, and unresponded. Think of advertising's part in trade, and the request-response model is dead, which may explain why so many billing systems layered over HTTP look so ghoulish. So eventually I was led to treating interaction as a somewhat arbitrary series of messages, showing patterns of no particular import to us.

The inspiration was then to flip the architecture system around: trade is messages, and to improve trade, improve the messaging capability. I didn't want semantics, I wanted freedom from semantics. I wanted a messaging system, with payments tacked on, rather than a payments system based on messaging. Indeed, I wanted a messaging system that could support arbitrary messages, and payments were just a trivial example of a message, within the full set of possible messages. (Trivial, because we already had them, and trivial, to focus the mind on the full possible set. Not trivial just from being payments, as these are anything but trivial.)

So the mission became to convert our payments-system-built-over-messaging into a messaging-system-with-payments. It seemed elegant enough, so over several nights at Rafi's, over ribs and beer, I outlined the story for my team:

1. we want more transactions,
2. payments business derives from trade ...
3. trade is really messages, with a payment tacked on,
4. we have a payment system, built on great messaging principles,
5. we just need to switch the emphasis of our system architecture a little, to:

   messaging-with-payments, not payments-over-messaging.

That's how Project Rafi's was born. I sold it to my team through beer & ribs, toasted the name, and commissioned a demo.

Of course, when people talk of messaging as an application, they think chat or Instant Messaging or SMS. So I seized that metaphor, and turned it into the popular expression of the mission. We were adding IM to our payment system, or so we said. Which was a subtly different subset to what I wanted, but close enough to be easy to sell, easy to discuss, and easy to tune in the detailed requirements and later implementation.

Sounds simple! Let's look back to the meat on the ribs of the original question: how did I do the architectural part? Looking at the above, the serial or static view is like this:

1. define the business problem that I am trying to solve
2. research the business context
3. extract requirements
4. build a virtual or model solution using some random classical technique or tools to hand

But this assumes too much omniscience, reality is far rougher, full of errors, omissions, unknowns. So, we need an error-correcting overlay, a feedback cycle. The dynamic view is then cyclical:

e. at each step, I test my results & conclusions against the earlier parts. E.g., test the solution against the requirements .. then test against the known business variations ... then similar business problems ... then the business statement.
e-bis. something breaks.
f. Out of which breach, identify the broken part (solution, requirements, research, problem).
f-bis. Jump back to a,b,c or d, depending. Evolve and re-state the broken input.
g. keep doing it until elegance strikes me on the forehead and I can't shake it off!

Now, this might be special, but it's not unique, Indeed, you can find this on the net if you look around. Here's one from Malan and Bredemeyer:

A 9 word summary might be set the mission, start high, go deeper, test, iterate. Or, if you want the 1 word secret, it is Iterate! Those with a sense of military history will see Boyd's OODA loop in there, with the problem being the enemy, the enemy being the problem, and the challenge to spin faster than the problem spins you :) And those with appetite will appreciate now why there are always so many ribs on the plate, and why architecture and fast food don't go well together.

What might offend some people is that it is so simple. I think the response to simplicity is that this easy model hides a lot, but that which it hides is not the essence of Architecture. Rather, the hidden part is the essence of something else, which the Architect is able to successfully integrate without spoiling the secret. And as this post is about Architecture, and is already long enough, I'll stop right there.

THE
the secret
the secret of software
                        the secret of software architecture

the secret of software architecture is . . .
i t e r a t i o n !

Hence that age-old complaint of the frustrated architect, "there aren't enough ribs on my plate!"

Posted by iang at June 5, 2010 10:28 PM | TrackBack