How to Save £120k on Your Cloud Bill
December 20, 2022
Zain Daniyal
Head of Data & Infrastructure, MTP
With the looming recession increasing the importance for businesses to examine every expense, optimising your cloud infrastructure can be an easy way to optimise performance and save money on your cloud bill. MTP’s head of Data and Infrastructure, Zain Daniyal, helps companies manage their data strategy, data governance and cloud hosting infrastructure. He explores how a major energy supplier cut their cloud costs by £120k.
Why do companies tend to overspend on their cloud infrastructure?
In most cases, their cloud infrastructure is inefficient.
Three mistakes are typically made that cause overinflated cloud bills:
1) Defunct cloud services that could be turned off or deleted are left running.
2) Some cloud services are run 24/7 despite only being required during working hours.
3) The code base is not optimised to run on the cloud and is therefore inefficient.
How can I optimise my cloud infrastructure?
Cloud cost reductions are achieved through an efficient code base and optimisation of cloud resource usage.
By examining the billing report, the most expensive services can be identified. It is likely that one or two services will represent ca.80% of the bill total.
Once we understand which cloud services are the most expensive, we can begin debugging these services to achieve efficiency.
During the debugging process, the data volume and processing frequency are identified. This allows us to compare a cost estimation according to the data requirements against the real-world cost.
To streamline cloud costs, the optimum amount of cloud computing resources is required and code base refactoring can be used to ensure the code can efficiently run on cloud services
How did you save £120k on the cloud bill?
We explored their cloud cost breakdown and identified the most expensive cloud services being used. Typically, one or two services represent the majority of billing cost. In our client’s case, £10k a month was being spent on cloud dataflow on GCP.
We began to debug the dataflow and identify how often specific functions ran, which was complex. By comparing against other cloud services, it was evident that an insufficient volume of data was passing through to account for the £10k monthly cost.
We identified a relationship between the number of computers available and the backlog. Dataflow, as the name suggests, is a flow. Pausing a job on Dataflow acts like a dam and the backlog builds up.
When the job is resumed or a new one is deployed, the dam is opened up. The backlog is programmed to rapidly clear itself following the removal of the blockade.
The simultaneous release of the backlog and increase in the number of cloud computing resources required to process the dataflow was causing the overinflated cloud spend.
We made a recommendation to our client that, by allowing the dataflow to run and avoiding pausing jobs, they would prevent backlogs and such a large monthly cloud bill.
As a result of our recommendation, the client reduced their cloud dataflow bill from £10k per month to roughly £50 per month.
In this case, code refactoring was not required to reduce the cloud bill; however, MTP’s in-house technical team can support the necessary code refactoring to optimise your cloud infrastructure and save you money on your cloud bill.