How to scale-out cloud for the future of finance

Financial services are increasingly reliant on data to survive. To survive in a fast shifting environment they need systems which can instantly grow to deal with shifting demand the business places on those systems. This might come direct from customers accessing an application or website or from systems which still have to deal with monthly, quarterly and yearly peaks in demand created by processing wages or dealing with regulators.

Financial services have additional burdens in making this work. They must prove their systems meet an ever changing, and ever growing, set of regulations covering everything from data protection to insider trading. They must ensure all activity fits within risk management profiles.

The cloud promised to end these problems. It promised endless, instant scalability at the touch of a button at lower cost than scaling up your own data centre. But the cloud’s dirty little secret is that there is a ceiling to this scalability both in terms of what is actually possible and at which point it becomes far cheaper to run your own private cloud instead of relying on a third party.

Even if the scalability is physically possible it often comes with a set of management headaches which make it effectively unworkable in practise.

That is why the majority of the big vendors are backing the hybrid cloud rather than one uniform environment. This allows organisations to exploit the benefits of different platforms while avoiding the pitfalls. Running your own cloud in-house guarantees information security for developers and let’s them optimise hardware without interference. Equally keeping infrastructure applications running on your own kit guarantees availability where demand is fixed or predictable.

But the public cloud can still be used for new releases and products where demand is harder to quantify and it can scale up to deal with peak demands. And it can do all this on a ‘pay-as-you-go’ basis without draining capital expenditure.

But building such a platform without it becoming a management nightmare of incompatible systems needs expert help.

Building a scalable cloud-based infrastructure does more than guarantee service levels for existing applications and business services.

It allows the organisation to be ready to exploit emerging technologies like Artificial Intelligence and machine learning which are increasingly available as cloud-based services rather than on-premises solutions.

This gives the obvious advantages of cost and scalability but also means that systems can be built which can exchange information and lessons learned from peer companies facing and solving similar challenges. Of course competition is not going away but collaboration is a better way to fix technical problems than attempting to re-invent the wheel. For financial services the challenge is not creating new services it is in scaling up to provide those services fast to everyone who wants to access them. The challenge from start-up fintech companies is that they have the potential to scale without being encumbered with legacy technology portfolios.

But despite this scaling up still proves an insurmountable barrier for many financial start-ups.

Arguably the single most disruptive fintech phenomenon of the last ten years, Bitcoin, is suffering from its own scalability crisis. The digital currency is struggling to increase the number of transactions per second which the network can cope with and arguments about possible solutions are ongoing.

For anyone building a more conventional scalable platform in the next few years it will likely be a hybrid infrastructure.

The secret is getting the glue right – a hybrid infrastructure is not just a random, mixed bag of platforms. It needs to be built on shared standards so that it can interoperate seamlessly and applications and services can moved around with the minimum of pain.

Finding the right partner to provide that glue is just as important as choosing the right providers for the actual platforms.