Financial Cryptography

User education: worse than useless

Cypherpunk askes a) why has phishing gone beyond "don't click that link" and b) why we can't educate the users?

A lot of what I wrote in The Year of the Snail is apropos to that first question. In economic terms, we would say that Phishing is now institutionalised. In more general parlance, and in criminal terms, it would be better expressed as organised crime. Phishing is now a factory approach, if you like, with lots of different phases, and different actors all working together. Which is to say that it is now very serious, it's not a simple net bug like viruses or spam, and that generally means telling people to avoid it will be inadequate.

We can look at the second question much more scientifically. The notion of teaching people not to click has been tried for so long now that we have a lot of experience just how effective the approach of 'user education' is. For example, see the research by Ye and Smith and also Herzberg and Gbara, who tested users in user interface security questions. Bottom line: education is worse than useless.

Users defy every effort to be educated. They use common sense and their own eyes: and they click a link that has been sent to them. If they didn't do that, then we wouldn't have all these darn viruses and all this phishing! But viruses spread, and users get phished, so we know that they don't follow any instructions that we might give them.

So why does this silly notion of user education persist? Why is every security expert out there recommending that 'users be educated' with not the least blush of embarrassment at the inadequacy of their words?

I think it's a case of complexity, feedback and some fairly normal cognitive dissonance. It tends to work like this: a security expert obviously receives his training from some place, which we'll call received wisdom. Let's call him Trent, because he is trusted. He then goes out and employs this wisdom on users. Our user, Alice, hears the words of "don't click that link" and because of the presence of Trent, our trusted teacher, she decides to follow this advice.

Then, Alice goes out into the world and ... well, does productive work, something us Internet geeks know very little about. In her office every day she dutifully does not click, until she notices two thing. Firstly, everyone else is clicking away like mad, and indeed sending lots of Word documents and photos of kids and those corny jokes that swill around the office environment.

And, secondly, she notices that nobody else seems to suffer. So she starts clicking and enjoying the life that Microsoft taught her: this stuff is good, click here to see my special message. It all becomes a blur and some time later she has totally forgotten *why* she shouldn't click, and cannot work out what the problem is anyway.

(Then of course a virus sweeps the whole office into the seas ...)

So what's going on here? Well, several factors.

1. Trent successfully educated Alice. So he runs around with the notion that user education is a fine tool and it works A-OK! Yes, he believes that we can achieve results this way. He promotes user education as the answer to our security needs.
2. Alice more or less left her education session successfully educated. But it doesn't last, because everyone else doesn't do it, and she can't see the sense in it anyway. That is to say, in systems theory, Alice cannot close the loop. She cannot see the action she is asked for as causing any additional security. As there is no reinforcing feedback, after a while it breaks, and never gets renewed.
3. Trent only managed to impress a small group of users. The vast majority of users out there however are 'the masses'. Many people don't actually know any computer experts, and they go into a retail store like Walmart to buy their computer. They have enough trouble coping with the fact that the parts need to be connected together in order to work ... and esoteric notions of clicks and security and viruses just go right over their heads.
4. When a virus or other plague like phishing does sweep through and wash the users into the sea, it's very unclear who to blame. Was it the bank? Was it Microsoft? Was it Sally-Anne who clicked on too many links? Or the Simon the systems administrator who installed a new Virus filter? Or, maybe it was Hamisch the terrible eastern European hacker again? Or ... , or ..... Just exactly why it happens is totally difficult to determine, and *that* means that the distance between the crime and any adequate defences is so great that there is no real feedback to reinforce the defences. Why defend when it is not doing anything?
5. If only a small subset of users are 'educated' this achieves nothing other than the minor effect of them potentially being protected. That's because if a virus sweeps through, we all suffer, even if we have protected ourselves. Yet, to stop the virus, we *all* have to follow the discipline. So we have a sort of free-rider problem: we have no way to make sure that all the users are 'educated' and thus there is no real incentive for anyone to educate.
6. Getting back to Trent, of course, he is secure. Everyone reading this blog is secure. Unless they are real users, in which case they didn't understand a word of it and didn't get this far (sorry about that!). So for Trent and our sort of person, we don't see the problem because it doesn't directly effect us. We are shut off from the real world of users.

Hence, cognitive dissonance. In this case, the security industry has an unfounded view that education is a critical component of a security system. Out in the real world, though, that doesn't happen. Not only doesn't the education happen, but when it does happen, it isn't effective.

Perhaps a better way to look at this is to use Microsoft as a barometer. What they do is generally what the user asks for. The user wants to click on mail coming in, so that's what Microsoft gives them, regardless of the wider consequences.

And, the user does not want to be educated, so eventually, Microsoft took away that awful bloody paperclip. Which leaves us with the lesson of inbuilt, intiutive, as-delivered security. If you want a system to be secure, you have to build it so that it is so intiutively to the user. Each obvious action should be secure. And you have to deliver it so that it operates out of the box, securely. (Mozilla have recently made some important steps in this direction by establishing a policy of delivery to the average user. It's a first welcome step which will eventually lead them to delivering a secure browser.)

If these steps aren't taken, then it doesn't help to say to the user, don't click there. Which brings me to the last point: why is user education *worse* than useless? Well, every time a so-called security expert calls for the users to be educated, he is avoiding the real problems, and he is shifting the blame away from the software to the users. In this sense, he is the problem, and until we can get him out of the way, we can't start thinking of the solutions.

Posted by iang at December 27, 2004 02:17 PM | TrackBack