In partnership with

Hey — It's Govardhana MK 👋

I always get a little excited putting together each edition. Writing five times a week can be intense, but it’s worth it.

There’s no formula for that spark, so I keep refining until it feels just right. Hope you enjoy the content.

Before we begin... a big thank you to today's sponsor.

Good day. It's Tuesday, Sep. 24, and in this issue, we're covering:

• Why Cloud isn’t Always the Answer

• DataDog - State of Cloud Costs Report

• Clever 'GitHub Scanner' campaign abusing repos to push malware

• 5 Steps to Set up Vault for Widespread Adoption at Your Org

• Cloud Build Unleashed: Expert Techniques for CI/CD Optimization

• Top 10 Must-Know AI Terms for Developers, SREs, and DevOps Engineers

Use Case

Why Cloud isn’t Always the Answer

Basecamp, a well-known project management software company, made headlines with the statement:

AND they pressed 👇️ 

Initially, it sounded like an exaggerated reciprocation – even some cloud gurus saw this isn’t possible, even if cost is saved, performance may be a trade-off for cost.

But soon, Basecamp silenced the doubts by publishing astonishing evidence of performance improvement after their cloud exit:

Here’s the summary:

• Median Request Time: 19ms (from 67ms)

• Mean Request Time: 95ms (from 138ms)

• Median Query Time: 50% reduction

• 95% of Requests: Below 300ms

• Compute (vCPUs): 122 vCPUs (before), 196 vCPUs available now

• Hardware Cost: Less than $20,000 per machine, amortized to $333/month over 5 years

This is no longer a surprise news; we keep hearing about companies either completely or partially exiting from the cloud:

My Hero in this context is definitely “Hivekit”

They not only saved 98% in cloud costs but also ended up writing their own database.

HiveKit is a platform that tracks thousands of vehicles and people, each sending location updates every second. Imagine having 13,000 vehicles constantly pinging you with data. Now multiply that by a full month, and you’ve got 3.5 billion updates.

They were using AWS Aurora with the PostGIS extension to store all this geospatial data. Aurora worked well initially, handling the load, but as HiveKit’s data grew, so did their AWS bills. They were burning through $10k a month just on the database, and it was only going to get worse.

At this point, most companies would either negotiate with their cloud provider or maybe try to optimize their queries. Not HiveKit. They took a different approach—they built their own.

Why Build Their Own Database?

So why go through the pain of building a database from scratch when there are so many good ones out there? Well, HiveKit had very specific needs. They didn’t just want a regular database; they needed something that could:

1. Handle insane write performance – Their system needed to support 30,000 location updates per second across multiple nodes.

2. Parallelism without limits – They wanted as many nodes as possible to write simultaneously without locking issues or performance degradation.

3. Minimal storage footprint – With 3.5 billion updates each month, storage had to be extremely efficient.

HiveKit chose to control their data destiny. They built an in-process storage engine that writes data in a delta-based binary format, perfectly optimized for their needs.

How Did HiveKit Do It?

• Delta-Based Format: Every 200 writes, HiveKit stores the full state of an object; between those, only changes (deltas) are recorded. This reduces a location update to just 34 bytes, fitting 30 million updates into 1GB of storage.

• Smart Indexing: A separate index file converts object IDs into 4-byte identifiers, speeding up lookups from 2 seconds to 13 milliseconds.

• Batching Updates: Instead of writing updates continuously, they batch and write once per second, reducing write operations and cutting storage costs.

The results? 98% savings on their cloud costs, plus faster queries and reduced overhead. It’s not just about cutting costs; they now have a system built exactly for their needs.

Closing Thoughts:

My 2 Cents 🪙 🪙 

• Future is going to be multi-cloud and hybrid cloud.

• Don’t throw it away when you get a chance to work on on-prem.

• Don’t stick to one cloud expertise; become an expert in one and grow in at least two more.

Want SOC 2 compliance without the Security Theater?

Question 🤔 does your SOC 2 program feel like Security Theater? Just checking pointless boxes, not actually building security?

In an industry filled with security theater vendors, Oneleet is the only security-first compliance platform that provides an “all in one” solution for SOC 2.

We’ll build you a real-world Security Program, perform the Penetration Test, integrate with a 3rd Party Auditor, and provide the Compliance Software … all within one platform.

Schedule a demo to get a quote! 🛡️

2024 is 73.22% complete.

Tool Of The Day

Try it out !

Trends & Updates

Resources & Tutorials

P.S. Want help with technical consultation? If you're a business, you can book time with me here

With that, it's time to wrap up. Got to get ready for tomorrow!

If you like what you read, please add me to your address book or reply (it helps with deliverability).