Working with Imagicloud
I thought I’d share some information on what it’s like working with us, Imagicloud.
I’ll talk about the types of clients we have worked with as Imagicloud, but also share some information about the clients I worked with before forming Imagicloud as an independent consultant.
The birth of Imagicloud
Primarily our clients have been in the startup space. Our first client was a global web analytics company who had an eye watering six figure monthly AWS bill, software which wasn’t designed to run in the cloud, but was running in the cloud, and a team of devops engineers who had abandoned ship. I joined this organisation as an independent contractor with the understanding that it was a “disaster recovery” piece of work and needed someone to start urgently. I was told that I would have very little handover and I’d just need to figure it out, and also informed that if I couldn’t reduce the AWS bill by at least $10,000 within the first month the company would likely be going under anyway.
I work well in high pressure situations, and this sounded fun – at the time, I had a choice of contracts, one which was paying far more in London but sounded mind numbing as the requirements were very vanilla – EC2, ALB’s, RDS, Lambda etc etc, the kind of stuff you find in how to guides which isn’t that interesting once you’ve done it a few times, and at this point I’d just spent 4 years doing that and was in the mood for something more challenging, so this contract, which sounded like walking into a building on fire to put out the flames, sounded ideal.
I’ll write an entire piece on the work we did for this client at some point in future, it was one of the most challenging projects I’ve ever come across and included complex database migrations, AWS-AWS migrations, a last minute GCP replatform and much more.
After six months of working with this client I had reduced their hosting bill by nearly 50%, implemented tools to help me manage everything such as Ansible, Terraform and Jenkins, introduced monitoring and locked down the most critical security issues which were present. I had reached a point where the company was able to survive, we had reached something which loosely resembled stability, and I was now comfortable managing everything on my own.
I was now faced with the challenge of re-platforming… everything, which was not going to be easy. I was asked if I knew anyone else who could help, and I did, which is when Matt joined us and it wasn’t long before they asked if I could setup a whole team for them, so I obliged and formed a full team of engineers to fix all of the remaining issues and take the platform as close to fully well architected as we possibly could, and Imagicloud was born.
Growing
It wasn’t long before the investment company who were the primary investors of our first client caught wind of the work we had been doing and asked if it would be okay if they referred the rest of their portfolio to us for any DevOps needs in future. They shared that escalating cloud hosting costs and scalability pains were a common theme across most of their startup ventures and were impressed at the savings we had achieved when their previous team had told them it was impossible to reduce costs further, and had also told them that auto scaling would be impossible – we had successfully applied auto scaling to every component in their infrastructure by the time our engagement had concluded.
Our next client came quite soon after our initial expansion, on the referral of our first client. Their needs were entirely different – the building was not on fire, the engineers had not jumped ship, but they needed to grow faster than their current DevOps team knew how, and so we came in to help out.
I started by conducted a well architected review so that we could understand their current infrastructure, application and the team’s current knowledge. I produced an extensive report of this review on the request of the client so that the wider team could understand, and so a detailed analysis was written and delivered based.
We then proposed a refactored version of their current Terraform, which we had identified was the main thing preventing rapid expansion of their platform and delivered this as a set piece to be handed over to their existing engineers. Whilst doing this we also implemented a multi account and multi region architecture ensuring separations of concern which had been established in our initial meetings, we included billing, iam, transit, dev, test, stage and production environments and also configured VPCs globally ready to accept their infrastructure with a secure transit VPC to cater for the future security and connectivity needs which had also been identified in our initial meetings.
Whilst doing the refactoring work, we brought what we could up to well architected standards and made plans for/logged remaining pieces which weren’t considered essential to the business. We allocated a full time resource from our internal team to help with delivery, and I would attend architecture meetings to monitor progress of delivery and ensure we remain on track.
We worked with this client for more than 12 months and it was great to work with some very talented developers and a very interesting piece of software.
Security
Our next client, who again were referred to us by existing clients, called asking if we could help bring their existing platform up to higher standards as one of their new clients, who happened to be in the banking space, had indicated that there was no way they would enter into an agreement unless they brought their architecture up to a much higher standard of security.
As usual, we started by conducting a free well architected review and produced a basic report. As transpired, this one wasn’t going to take much – we needed to improve their security group definitions, introduce IAM, add some monitoring and create some terraform to help manage it all.
The bulk of the service was operating in an ECS cluster and so not many components were involved outside of this making it a relatively small piece of work.
We produced full documentation for what we were about to hand over, and spent some time with their existing engineer to ensure they understood everything and could manage it longer term, and we remained in their slack channel for a few months just in case they had any questions for us.
Cloud-Cloud Migration
Most larger organisations that I have worked with always have a desire to be cross-platform, but very few actually achieve this.
For one of our clients, they found themselves with no choice – they wanted to sign a customer who stated they categorically could not have their website integrated with AWS in any way, but GCP would be acceptable. The reasons were due to competition, and at the time I was entirely unfamiliar with GCP – I’d tried out some of the services when it first launched but not gone back since.
It’s at this point I realised how much better GCP had become compared to AWS – the speed at which we were able to deliver was significantly quicker, instances would launch faster and many other time saving elements exist.
We had recently completed the task of fully automating the AWS version of their deployment only to find we were now at step 1 for GCP… there was one limitation we found with GCP which was very specific to our clients software, the auto scaling and load balancing services would not work in the way we needed them to, this was also the case in AWS but we were able to get around this by using lifecycle actions.
On GCP, this was not an option and so we found ourselves in a position where we would have to write auto scaling ourselves… which actually ended up saving a lot in cost and was a better architecture as we were now in full control of the scaling and traffic management and opted to use DNS load balancing instead of application based load balancing. Having full control of the auto scaling elements allowed us to be quite clever when it came to failure handling, we could use custom nginx lua code and apply some logic – something which is possible on AWS, but is expensive – especially when dealing with high volumes of traffic which is what we were dealing with.
How much traffic?
Dealing with high volumes of traffic is always entertaining.
Doing this whenÂ