Corporate Computing » Observations of an Edinburgh Chap

Left Hand Fines Right Hand After Human Interface Tragedy

Corporate Computing, Life in General No Responses »

Jan 132012

I read that the Information Commissioner has proposed to fine Brighton and Sussex University Hospitals NHS Trust £375,000 for a breach of patient confidentiality whereby hard drives containing patient data were sold on eBay. Digging a little further into the story, it seems that the NHS Trust hired a contractor to remove the drives, delete their contents and then dispose of them. The contractor sold the drives on, but didn’t do the deletion.

This is actually quite normal. Asset disposal companies take care of the owner’s legal obligations in disposing of kit, and they then offset the management charges by selling the kit on for whatever they can get. Obviously, the disposal company must correctly discharge its data cleansing duty, but the practice of an external contractor removing the kit and then selling it is completely standard, and returns the asset’s value to the owner.

The problem is that the patient data was not erased. And that’s why the Information Commissioner’s Office (ICO) has got involved.

But the ICO’s response, of fining the NHS Trust, is completely wrong.

Firstly, any government organisation fining any other government organisation is like me moving a £10 note from the front to the back of my wallet. It’s still money in the system. Except that it doesn’t cost me anything to move notes around in my wallet. There are loads and loads of people in the NHS and ICO who will be involved in the levying, paying and processing of the fine. These people will contribute nothing at all to the country by carrying out this transaction.

Secondly, NHS Trusts need all the money they can get. Reducing the money supply to an organisation that is already hard-up will only have the effect of increasing the pressure on staff, and therefore the likelihood of similar mistakes being made again.

Thirdly, this data breach is a human error. Somebody didn’t do their job properly. Was it negligence? It certainly appears that way. I’m not really a witch-hunt kind of guy, but why is the employing organisation being fined? It does not create a link between the punishment and the perpetrator of the crime/error. What should be done in this case is a period of non-criminal community service. The error affects the general public, so the perpetrator of that error should make reparations to the general public.

Fourthly, it doesn’t even appear as if this error was made by an employee of the Brighton and Sussex University Hospitals NHS Trust, so why are they being fined? Surely the error was made by the disposals company that tried to sell the unwiped hard drives. They are the ones who should be being fined. (Although, B&SUH should really have had a flow-down clause in its contract with the disposals company so that they could pass the liability on)

These points expose my real feeling of disgruntlement. Certainly, a privacy breach occurred, and a lesson needs to be learned. But that lesson has to be targeted and appropriate. The people involved need to be aware of the implications of their failure to correctly discharge their duty, and – where appropriate – must carry out some form of penitent action to make up for that mistake.

One government department stealing money from another teaches nobody anything, except for teaching the public at large that the whole system is FUBAR.

More Thinking on the PSN Hack

Corporate Computing, Internet, Technology 2 Responses »

May 112011

On 27th April, Sony sent me a fairly long, simplistic, patronising e-mail that completely failed to in any way apologise or offer recompense for their inability to secure my data. It was so bad in fact that Outlook tried to protect me from it, throwing into the Junk Mail folder.

I haven’t pored and pored over it. Fortunately, being of reasonable intelligence, I can read something once, and often retain much of that content in my head.

Some things have struck me in the intervening time:

Why didn’t Sony systems see the data leaving its network?

Sony has said that all PSN users’ accounts have been compromised. Accordingly, someone, presumably external to the PSN, managed to pull 77 million records from the database, without the data type being analysed by the firewalls through which it passed.

To reliably deliver large quantities of content over the internet, where some of the content itself is in large chunks (e.g. downloadable game patches, at between 20 and 600MB each), the network is split into sectors. The user interactions with the store front (PlayStation Store) are handled by one set of servers. The delivery of the game data (patches, downloadable content etc) is managed by a separate “content-delivery” network. The PlayStation Store instructs the CDN server to send the file to the customer, and in that way the customer interactions with the store (or the interactions of other customers) are not affected by the large download in progress.

Separation of PSN account services from content delivery services (thanks to Sykonist for the PS3 icon used under Creative Commons)

The assertion I’m making here is that the PSN should expect to be serving large files from CDN servers, but that the user data in question (the 77m records) would not be on those CDN servers – or at least should not be. This would make the delivery of those user records – which would be a substantial file transfer – an unusual event that intrusion prevention systems can be configured to look for in the traffic flow of standard operations. Was an intrusion prevention system (IPS) in place, operational, and properly configured? (Note: in the recent hack of Barracuda, the company ‘fessed up to the fact that their IPS was in place, and was configured, but had accidentally been left in “watch & learn” mode).

Were attackers hitting the content delivery servers, not the Store servers?

The content delivery servers shouldn’t really have any interaction with the Store servers, except to confirm that a request coming in from a user is to be accepted or declined. This can be secured quite simplistically, with a challenge/response request between the CDN server and the Store server. Firewalls between the two networks can be configured to allow a maximum conversation size, and cut anything longer than that.

This allows a lesser level of security to be put in place to defend the CDN servers, as they’re fairly dumb boxes configured to dump large quantities of data to the internet. And economising on security can – if done correctly – save real money.

How were the attackers able to get access to all the database records?

One of the requirements I have for securing web database systems is that the web servers are only allowed very limited access to the database. The web server acts as a proxy for the user (or attacker), passing the user’s request for information to the database server. By correctly configuring the database user account that the web server uses, security can be engineered into the data store.

PSN Dialogue: How not to get user information

Fundamentally, there is no requirement for the web servers’ user accounts to be able to retrieve all database records. The web servers need to be able to query the entire database, but at no point should the content of the database be returned to that server. It is enough to query for the existence of a record, and then return only that record, instead of grabbing all the records, then checking to see if the data grabbed contained the right information.

Let's talk about this guy

This is also a design function that should be being used for performance and scalability reasons, rather than as a pure-play security measure. Once again, it requires the security consultant to understand the size of a valid conversation between servers, and then to configure the security systems to block outsize traffic streams.

So, how was the security landscape at Sony?

Well, there was the revelation that one of the attacked servers was running software at least ten years old… This is unforgivable. It points to an underlying security attitude of “if it ain’t broke, don’t fix it”, which was the prevailing attitude to software versioning in around 1998. The critical flaw with this attitude is that it assumes that the threat landscape does not change with time. An assumption so ridiculous that even as I type it out I want to flame it until it’s dead.

That aside, drawing conclusions from such a limited information set is always risky. However, it seems certain that the PSN was not secured on a behavioural level.

The fact that hackers were able to extract large quantities of data from systems that should only be serving data fragments suggests that no systems were in place to check that servers were doing things they should not be. It seems that the network layers being secured were low-level, and that there was little – if indeed any at all – application intelligence being applied. Consequently, there is little to support the existence of Intrusion Prevention System (IPS) devices in the Sony network. Making accusations is easy, but really, it’s difficult to conside the omission of IPS as anything other that professional negligence either by a security architect not specifying this equipment or by a manager deleting it for budgetary reasons.

Moving on…

Sony has talked of “adding” firewalls, implying that they may have been scrambling to introduce additional network tiers that add abstraction between the frontline servers that PS3s talk to and the back-end database that powers the network intelligence. Perhaps they’ve been implementing these IPS devices, but “simplifying for the masses” the accounts of their actions by just calling them firewalls.

Sony must also be scrambling to bring up to date the web servers that were running the aged software. This isn’t something that should be rushed: it’s likely that many programming methodologies used in developing software circa 2000 are now prohibited, and will have to be recoded, retested, and, well, generally pretty much rewritten from scratch.

They should be shifting network security to be both at the firewall (request/accept/deny) level, and at the behavioural (inspect/review/allow/block) level. Database servers configured to only provide very limited data sets. No account information in the content delivery network. Different access levels for PS3/PSP devices from other devices. All things that should have been implemented before, and should be relatively simple to implement now.

In the meantime, the irony is that my 21st Century internetworked high definition console sits idle, while my need for a gaming fix is satiated by an organisation that is proudly “Steampowered“.

Lessons from PSN: Would You Bet Your Business on a Cloud?

Corporate Computing, Internet, Technology 3 Responses »

Apr 262011

“Cloud Computing” is the ICT industry buzzword du jour, and it’s causing a lot of discussions in IT offices and boardrooms across the globe. It is, as TechMarketView‘s Richard Holway would say, a “disruptive” technology. Cloud has the power to rip up the rulebook on IT service provision, both on an in-house departmental basis, and on an IT Outsourced Service basis.

But what is cloud computing?

Put simply, cloud computing is any combination of systems that provides IT services using infrastructure disconnected from the consuming organisation, or paid for on an ongoing service basis. Typically, line of business applications connected to through secure web sites, paid for on a per-user-per-month basis. Naturally, as the term was coined so it fragmented into things like “software as a service”, “service oriented architecture”, and others.

PSN Sign-In

The big commonality between cloud services, is that they almost always use a distributed architecture, of anything from tens to millions of servers, spread across the internet. The users see a logon page, and connect to it, entering their credentials and connecting to their session on their instance of the service.

Think Gmail, Facebook, Skype. These are all cloud services. All services where consumers of the service are not concerned with the number of servers, the connection types, what database is on the back end, or whether a new version is around the corner. The service is paid for (perhaps at zero cost) on an ongoing basis, rather than the usual capital+depreciation model of traditional IT.

Sony’s PlayStation Network is just such a cloud service. Just before Easter 2011, PSN came under attack from malicious outside forces. Disruption to service ensued, culminating, on Good Friday, with Sony taking the entire service offline. Four days later, it’s still offline, and Sony has stated that it could be offline “indefinitely” while repairs to the infrastructure are made. PSN is the mechanism by which Sony PlayStation 3 consoles connect users to other users and to the games manufacturers. Online gaming, profile storage and software updates are all served through PSN. And as gaming has moved more and more online, so much of the functionality has moved to the cloud. The PS3 is itself moving back to the more traditional computing understanding of the term “console”, being a device that manages the user input and the screen generation. Obviously, in the case of ultra-advanced games such as Guerilla’s Killzone 3, this is an extraordinarily computing-intensive task, but the fact remains that with PSN offline, the enduring appeal of the game is neutered.

PSN Maintenance Message

Naturally, the gaming community is going a bit mental about this.

It came to a head on 26^th April however, when Sony admitted that, not only had PSN come under attack from outside, but that sensitive user information – including credit card details – had been obtained during the action. Rightly, in my view, users are outraged that it took Sony so long to admit this to be the case. PlayStation Network has 75 million users.

“How does this free-to-access games console support network relate to my business?” you may be wondering. Well, it’s the paradigm of access. When your key business system is hosted inside of your premises, you have far greater control over it. You can hire people to look after it. You interview them, vet them, follow up on their references. You firewall your kit from the outside world. You control it.

I have frequently been asked whether “we should move <service> to the cloud”. Not once, yet, has the organisation asking me been truly, culturally, ready to make that move. Sure, from an IT perspective, it’s easy. Migrate the data and the account details, tell users where the service is now, and away you go. But culturally it’s a far greater leap than simply switching from a planned programme of capital investment to a monthly charge. You, the client organisation, have to cede control to a faceless corporate, whose service levels will mean jack when the service falls over. Whose financial restitutions won’t recover your lost client records. Assuming that there isn’t some weasel-wording in the contract that lets them away with it.

Back to PSN: Service offline for seven days. Key user data lost. No comeback for the users against the provider. Can your business stand to lose its key business application for multiple days at a time? Can you, as the person responsible for IT, hand over the keys to your service, to your data, to your customers’ data? Because only when these things do not matter can you class yourself as cloud-ready.

Either then, or when cloud providers sort their acts out, of course.

Older Entries