August 07, 2007

Security can only be message-based?

Reading this post from Robert Watson:

I presented, “Exploiting Concurrency Vulnerabilities in System Call Wrappers,” a paper on the topic of compromising system call interposition-based protection systems, such as COTS virus scanners, OpenBSD and NetBSD’s Systrace, the TIS Generic Software Wrappers Toolkit (GSWTK), and CerbNG. The key insight here is that the historic assumption of “atomicity” of system calls is falacious, and that on both uniprocessor and multiprocessing systems, it is trivial to construct a race between system call wrappers and malicious user processes to bypass protections. ...

The moral, for those unwilling to read the paper, is that system call wrappers are a bad idea, unless of course, you’re willing to rewrite the OS to be message-passing.

And it sparked a thought that systems can only be secure if message-based. Maybe it's a principle, maybe it's a hypothesis, or maybe it's a law.

(My underlining.) To put this in context, if a system was built using TCP/IP, it's not built with security as its number one, overriding goal. That might be surprising to some, as pretty much all systems are built that way; which is the point, such an aggressive statement partly explains where we are, and partly explains how tricky this stuff is.

(To name names: SSH is built that way. Sorry, guys. Skype is a hybrid, with both message-passing and connections. Whether it is internally message-passing or connection-oriented I don't know, but I can speculate from my experiences with Ricardo. That latter started out message-passing over connection-oriented, and required complete client-side rewrites to remove the poison. AADS talks as if it is message-passing, and that might be because it is from the payments world, where there is much better understanding of these things.)

Back to theory. We know from the coordination problem, or the Two Generals problem, that protocols cannot be reliable about what they have sent to the other side. We also know from cryptography that we can create a reliable message on the receive-side (by means of an integrated digsig). We know that reliabile connections are not.

Also, above, read that last underlined sentence again. Operating systems guys have known for the longest time that the cleanest OS design was message passing (which they didn't push because of the speed issues):

Concurrency issues have been discussed before in computer security, especially relating to races between applications when accessing /tmp, unexpected signal interruption of socket operations, and distributed systems races, but this paper starts to explore the far more sordid area of OS kernel concurrency and security. Given that even notebook computers are multiprocessor these days, emphasizing the importance of correct synchronization and reasoning about high concurrency is critical to thinking about security correctly. As someone with strong interests in both OS parallelism and security, the parallels (no pun intended) seem obvious: in both cases, the details really matter, and it requires thinking about a proverbial Cartesian Evil Genius. Anyone who’s done serious work with concurrent systems knows that they are actively malicious, so a good alignment for the infamous malicious attacker in security research!

But none of that says what I asserted above: that if security is your goal, you must choose message-passing.

Is this intuition? Is there a theory out there? Where are we on the doh!-to-gosh scale?

Posted by iang at August 7, 2007 11:09 AM | TrackBack

i would put it slightly differently, KISS api with strongly enforced boundaries and partitioning.

message-passing tends to imply various kinds of partitioning.

however, there was a lot done in the 60s (disclaimer, i was involved) with virtual machine paradigm as a partitioning mechanism ... with message-passing between the virtual machines.

this was work at the cambridge science center ... 4th
flr, 545 tech sq.

which traces some of its heritage back to CTSS. recent mention of upcoming 40th anniversary of (cp67, as well as 35th anniversary of vm370) product announce

also reference to archeology in this recent post Doom and Gloom spreads, security revision suggests "H6.5: Be an adept!"
and this reference

Of course, Multics on the 5th flr, also traces back to CTSS, recent post mentioning multics If your CSO lacks an MBA, fire one of you

the science center was also responsible for the technology for the internal network

which was larger than arpanet/internet from just about the beginning until sometime mid-85.

also gml (precursor to sgml, html, xml, etc) was invented at the science center in '69

of other historical reference .... the compare&swap instruction was invented by CAS at the science center (note that mnemonic for compare&swap instruction are charlie's initials) as part of work on fine-grain multiprocessor serialization related to cp67. With some difficulty managed to get CAS added to the 370 architecture in the early 70s ... lots of past postings on multiprocessor support and/or compare&swap instruction

for lots & lots of historical drift ... the original sql/relational work was implemented in virtual machine (vm370) environment at SJR ... lots of past posts mentioning system/r

some amount of the AADS work drew on background

having done operating concurrency/integrity with cp67 and vm370 ... but also with system/r. Also as part of doing the high-availability product

we had to work out loads of failure modes ... not just security related ones ... and provide for correct operation in distributed environment. if fact, during the ha/cmp work we had coined the terms "disaster survivability" and "geographic survivability"

another influence was having to do the distributed lock manager for distributed scaleup operation; some old references in this series of old email

as frequently mentioned ... we were later called in to consult for a small client/server startup that wanted to do payment transactions on their server ... which then required this thing called a payment gateway

and two of the people that were responsible for what they were calling a commerce server had earlier worked with us on distributed database transaction scaleup ... and, in fact, were at the meeting mentioned in these posts

that whole sequence of having done mainframe multiprocessor scaleup and concurrency, database concurrency and scaleup, distributed database operation and scaleup ... and then having done the availability product before working on what has since come to be called electronic commerce ... also influenced the later work on AADS.

Posted by: Lynn Wheeler at August 7, 2007 03:54 PM
Post a comment

Remember personal info?

Hit preview to see your comment as it would be displayed.