Computer Security Isn't an Afterthought

Computer security must be built in to every application, it is almost impossible to add it on at the end.
For a brief moment we deceived ourselves into believing that firewalls and defined interfaces would allow us to ensure arbitrary applications could be made “secure”. In this case I define secure to mean behave in a defined way and in particular not leak information. Computing theory can be used to supports this, but with such strong caveats and in such limited cases, that in practice it is only a dangerous dream. In theory, this form of security is achievable if the full state of the system is known, and exactly how the system transforms its state based on input data is also known. In practice, for even a moderately complex systems this rapidly grows unachievable.

A simple thought experiment can show the limits of knowability - assume we had a perfect model of the system being protected
1. A quick demonstration that even this perfect model is insufficient for an arbitrary system is to answer the question, how many inputs would you have to check before you would be certain that the state transformation would not result in an information leakage in the future? This is computationally explosive for simple systems and unknowable for arbitrary systems.

The end result of this thought process is that arbitrary systems cannot be protected by slapping on a layer of security after the event. Furthermore, systems of systems are even worse, as even if the parts behave in a secure way, the whole must be considered (cross site scripting attacks are a class of attacks on systems of systems).

Well defined interfaces have gone a long way towards helping us build secure systems, but they are ineffective if the developers of those systems are not conscious and conscientious about protecting against attacks.

1. Surprisingly perfect models are easily available in the days of virtualisation - we could snapshot the system being protected, duplicate it and expose the “model” system in an isolated environment to the inputs we wish to check.