Choosing the Right Database in 2025: A Category-First Guide

 A practical map for faster system-design decisions

Intro

The database landscape has exploded—in a good way. Instead of hunting for a single “best” engine, modern teams compose the right categories for the job: transactional integrity, low-latency reads, high-throughput writes, full-text search, batch analytics, and durable object storage. This post summarizes the trade-space I use in design reviews and includes a one-page visual you can share with your team.

---

The Category Map (quick reference)

1. Relational (RDBMS) — Strong consistency, complex queries, auditability.
   Examples: PostgreSQL, MySQL, Aurora (AWS), SQL Server, Oracle.
   
   
2. NoSQL: Key-Value & Wide Column — Massive scale, predictable keys, high write throughput.
   Examples: DynamoDB, Cosmos DB, FoundationDB; Cassandra/ScyllaDB/HBase; Bigtable/InfluxDB/VictoriaMetrics.
   
   
3. Document Stores — Flexible schema, nested JSON, fast iteration.
   Examples: MongoDB, Couchbase, Amazon DocumentDB; Firebase Firestore, Realm; CouchDB/PouchDB; RavenDB/ArangoDB.
   
   
4. Caches & In-Memory — Ultra-low latency, pub/sub, rate-limiting.
   Examples: Redis, Memcached, Azure Cache for Redis, Hazelcast, Apache Ignite, NCache.
5. Search & Real-Time Analytics — Full-text search, fast aggregations, time-series dashboards.
   Examples: Elasticsearch/OpenSearch, ClickHouse, Apache Druid, Typesense, Meilisearch.
   
   
6. Data Warehouses & Lakes — Petabyte-scale analytics, BI, ELT/ETL, lakehouse.
   Examples: Redshift/BigQuery/Snowflake; Athena/Trino(Presto)/Starburst; Databricks, Delta Lake, Apache Iceberg.
   
   
7. Object Storage (Blob) — Cheap, durable storage for media, logs, ML artifacts, data lakes.
   Examples: Amazon S3, Google Cloud Storage, Azure Blob Storage; MinIO/Ceph/SeaweedFS; Wasabi/Backblaze B2/Cloudflare R2.

---

How to Choose Quickly (three prompts)

1. Access pattern first: keys, reads vs writes, latency, joins/aggregations, query flexibility.

2. Failure and scale model: single-region vs multi-region, consistency needs, hot keys, TTLs.

3. Ops reality: managed vs self-hosted, cost model (throughput vs storage vs egress), team skills.

---

Common, Proven Combos

1. Low-latency read API: KV (DynamoDB/Cassandra) + Redis cache + object storage for blobs.

2. Search-heavy product: RDBMS or document for source-of-truth + Elasticsearch/OpenSearch for text + Redis for hot paths.

3. Analytics stack: Object storage (S3/GCS/Blob) + warehouse/lakehouse (Redshift/BigQuery/Snowflake/Databricks) + Trino/Athena for ad-hoc.

---

Few Pitfalls to Avoid

1. One size fits all: forcing a single DB to do everything increases cost and fragility.

2. Ignoring cache invalidation: fast until stale—then expensive. Design TTL/keys up front.

3. Unbounded growth: plan TTL/archival to object storage; watch hot partitions and fan-out.

4. Ops mismatch: great tech, wrong team fit—prefer managed where it accelerates delivery.

---

Closing
The win in 2025 isn’t a silver bullet database—it’s the ability to compose the right categories for each workload and evolve the stack as needs change. I’ll follow up with short decision flows for specific scenarios.

Which scenario should I tackle first? Comment on LinkedIn.

Flawless Is The Enemy of Good

When tackling large-scale, complex initiatives, the notion of “flawless execution” sets up leaders and teams for disappointment. Complexity guarantees surprises: dependencies shift, risks emerge, and trade-offs appear late in the game. From my experience leading programs at Amazon, Citrix, Philips and beyond, I’ve learned that the right aspiration isn’t flawless execution, but fearless execution.

Fearless execution doesn’t mean chasing perfection. It means leading with clarity, transparency, and adaptability, while creating an environment where teams feel safe and empowered to do their best work. Or as I like to say: flawless is the enemy of good.

 Here are five key drivers that consistently define fearless execution:

1. Clear Vision
A compelling north star aligns decision-making. Without it, teams drift. Vision isn’t just a tagline—it’s a narrative that helps individuals understand how their day-to-day work maps to long-term outcomes. Fearless leaders repeat the vision often and tie every milestone to that broader story.

2. Detailed Plans
Planning is a continuous discipline, not a one-time artifact. The best leaders break ambiguity into structured components: clear scopes, aligned dependencies, risk registers, and iterative delivery cadences. And critically—plans must include all teams involved, not just your own. Engineering, product, design, security, operations—when everyone sees the same field, execution accelerates with fewer surprises and more ownership.

3. Risk Mitigation
Hope is not a strategy. Fearless execution requires early identification of failure points—technical, process, organizational—and building real fallback plans. In my team, we paired launch plans with rollback and customer impact scenarios, so we never froze when something failed. Speed is important—but responsible speed is built on risk awareness.

4. Psychological Safety
No team performs at its best in fear. I’ve seen this firsthand: motivated, capable engineers under-deliver when they feel their voice doesn’t matter or mistakes are punished. Fearless leaders create environments where dissent is safe, ideas are welcomed, and failures are treated as learning moments. This isn’t soft—it’s strategy. Engagement, creativity, and ownership all stem from feeling safe and supported.

5. Adaptability
Even the most meticulous plans collide with reality. Fearless leaders prepare teams for this: to reevaluate priorities, shift roadmaps, and reset expectations without panic. What matters is how quickly and thoughtfully we respond—not that we saw everything coming. Adaptability is the muscle that keeps momentum intact.

Closing Thought
Flawless execution is impossible in large-scale programs. But fearless execution—anchored in vision, planning, risk management, psychological safety, and adaptability—turns complexity into impact. In the end, it’s not about control—it’s about creating the conditions for people to succeed, even when everything doesn’t go as planned.

14 Essential Software Engineering Concepts for Engineers and Managers

There are many terms and concepts that are important for an engineer to be familiar with, in order to effectively build software. This post includes some of those terms. I will continually add to or update this list.

1. Agile. A flexible and iterative approach to software development that emphasizes collaboration, customer feedback, and adaptive planning. My experience and success with agile development was the inspiration behind starting this blog.

2. DevOps. A set of practices and tools that improve efficiency, speed, and reliability of the product through automation and optimization of the software development and delivery process where operational efficiency is part of the development process.

3. Continuous Integration and Continuous Delivery/Deployment (CI/CD). A set of practices and tools that result in faster and more frequent releases, through automation of building, testing, and deployment of software. A key part of CI/CD is to deliver software to production frequently and using techniques such as Test Driven Development, can significantly improve the results that teams get through CI/CD.

4. Test Driven Development (TDD). With TDD, developers write tests before writing code. This helps ensure that code meets the desired requirements and improves overall code quality. This practice encourages developers to match the requirements to the behavior of their code before writing it, which can lead to fewer bugs and a generally more robust software.

5. Scalability. The ability of a system to handle increasing load and demand, either by adding more resources or by improving its architecture. Cloud computing technologies, such as AWS provide various options to scale a system's load-bearing capabilities.

6. Performance Optimization. The practice of making software run faster, consume fewer resources, and handle more concurrent users, in order to improve the user experience. Here again, cloud computing technologies like AWS provide several tools that can help you in monitoring and optimizing your system's performance.

7. Resilience. The ability of a system to recover from failures and continue to function properly, even in the face of adverse conditions or circumstances.

8. Security. The practice of protecting software and its users from unauthorized access, exploitation, and data breaches. Security was initially a big concern for organizations to adopt cloud computing, though now all cloud computing providers offer robust set of tools to manage permissions and credentials to address those concerns.

9. Cloud Computing refers to the delivery of computing resources, such as servers, storage, databases, and software, over the internet. With cloud computing, organizations can scale their computing resources up or down as needed, and only pay for what they use, rather than having to make large upfront investments in hardware and software.

10. Infrastructure as Code (IaC). The practice of using code to automate the provisioning and management of infrastructure, allowing for faster and more reliable deployment, scaling, and maintenance. AWS's CDK is a great example of IaC.

11. Model-View-Controller (MVC) is a design pattern that separates an application into the Model, the View, and the Controller components that are inter-connected. The Model stores & manages the data, the View displays the data, and the Controller handles the interactions between the Model and the View.

12. Microservices is an architectural pattern where a large application is broken down into small, independent, and loosely coupled services. This approach enables developers to build, test, and deploy each service independently, leading to faster time to market and reduced risk of failure. Medium to large applications often use a combination of MVC and back-end microservices.

13. Software as a Service (SaaS) is a model where software is made available to customers over the internet, often for a subscription fee. SaaS providers host and manage the software, taking care of maintenance, upgrades, and security, and usually charge a fee for their solution. SaaS products help business customers to focus on their business processes rather than building software for needs that they share with other businesses. There are also several SaaS products for individuals and families.

14. Artificial Intelligence (AI) and Machine Learning (ML). Technologies that allow computers to learn from data and make predictions or decisions without being explicitly programmed to do so.

It's important to have a general understanding of these concepts and they impact the delivery successful products.

Being OPEN about Sprint demos

Consider This

Someone is presenting the work that they have just completed. Compare and contrast the two scenarios below:

ONE.

"This is freaking awesome!", you think. At one moment, everyone in the room was listening intently - not a word was spoken by the audience. At another, multiple hands went up for questions. Then, there was debate, many questions answered, some actions noted. Finally, the whole room erupted with applause and appreciation for the presenter and for each other.

TWO.

"Is that all you got?", you can't help but wonder. The room is low energy - the audience is either trying hard to listen, is distracted - only pretending to listen, or just busy chatting with someone else. There are no questions asked, and of course, none answered.

 

Which of the above scenarios would you like be a part of - in either role, as presenter or as audience?

If you have worked in software development for few years, with different teams or organizations where they demo their work every couple of weeks, then chances are that (like me), you have come across both of these scenarios.

OPEN Rules

Below are the OPEN rules for presenting compelling Sprint Demos that rock:

1. Ownership - Present it like you own it and acknowledge others who contributed. That will engage everyone who you acknowledge immediately, increase your credibility and tune in anyone who is interested in learning about what you and those you mentioned did.


2. Purpose - Delve into questions like: What are you demonstrating, what problem does it solve, how far along you are in solving the problem, and what is unique about your solution?


3. Engagement - As a presenter, it is up to you to make or break your demo. You can decide how you want it to be like - interesting, nerdy, funny, or anything else. Most importantly, show enthusiasm for the topic, as without it you will sap the energy out of the whole experience.  Some things to think about: What is your presentation style? When you speak, can they hear you, understand you? Are you okay being interrupted with questions during the demo or do you want everyone to wait till you have completed your demo? If you do mind being interrupted during the demo, then letting the audience know when you start is a good idea.


4. Next steps - A demo is a way to present your progress. So, you might have next steps related to what you presented in the following sprint or later. Letting everyone know what those are and what to expect reduces the chances of surprise in the next demo. Plus, you should expect feedback during the demo - most good demos receive new ideas worth exploring or specific things to work on. Make sure you capture it all (better yet, have someone on the team do so) and mention what you will do about the feedback. As you close out, thank everyone for their participation.

Hope you have wonderful demos!

Related Reading

(1)

https://georginahughes.com/2017/10/16/power-start-your-meetings/

(2) I'm Not Coming to Your Bad Meeting

Create change... then get rid of it!

Someone said to me, just a few months ago - "There are two ways to deal with something you don't like. One is to try to change it and another is to accept it". The advice came from someone I had come to respect, yet the notion of accepting something that I don't like made me uncomfortable.

After thinking about it, and naturally falling back to the ethos of agile software development, I argue that the advice is sane and is closely related to managing change.

Here is what I think:
1. Responding to change sometimes requires creating the change
2. Any change requires prioritization
3. Some changes need to be shortlived
4. Some changes are not worth it
5. Some changes leave a mark behind

Creating the change

It is unnatural to me to accept something that I don't like straight away. So I try my best to make a change and make things better. If I don't even try, would I be doing my duty towards making the situation better? To me, this is about thinking about the change that should be made and how will I know that I have succeeded. This is the first step.

Prioritization

What if we treat all changes with an identical sense of importance and urgency? When everything becomes urgent and important, we become perfectionists and lose sight of what is valuable. The perfectionist approach usually results in too much work in progress that reduces focus and we get nothing done.

Shortlived changes

I used to think that changes are meant to persist. Now, that is hardly true! Too many changes are just transient. Their purpose, for example, maybe to trigger further changes or to be quick fixes so we can tackle an immediate problem at the moment.

Changes Not worth it (or Worth being patient about)

There are many changes that are just not worth trying. Maybe there are alternatives, or maybe it is just better to wait for the situation to just evolve. Jumping the gun doesn't solve a thing, in these cases. This is about accepting how things are at this moment, and not about giving up.

Changes that leave a mark behind

There are changes that we should really care about. Involve people who will be our partners, chart a course that will lead us to our desired state. Doesn't matter whether the change is small or big, although I am biased towards small changes as of now (it is 2017 and I am still about short Do/Build, Test and Learn cycles). Eventually, the change should not feel like a change... it should blend so much with the environment that it should feel like you've gotten rid of it!

Would love to hear your thoughts, what do you think?

Six steps to unlock your team's true potential

What's true potential?

My definition of true potential is a state of being where performance is maximized even when the situation is hard, or even too hard. When someone or something reaches its true potential it reaches the state of hyper-performance without fear, not hyper-performance with stress.

Why is it hard to achieve?

I have often seen fear cast a shadow over our existing potential, making it hard to see. Fear can manifest itself anywhere at anytime. At other times, it may be quite evident. At times, it may be invisible but it is there. When we can't see our existing potential due to fear, it is even harder to see what true potential might look like.

This is true not only for individuals but also for teams. It is easy to see that a team is made up of individuals, who have their own fears. In the worst case, the team's fear could be compounded by multiplying the fears of rest of the teams. Yes, we definitely don't want that!

Addressing the Ecosystem

The ecosystem in which teams operate have a huge impact on their performance and can deter or improve their chance of unlocking their true potential. Here are some ways of addressing the ecosystem.

Noise cancellation

Team members' fears can act as noise cancellation mechanisms for other team members' fears. How?

Let us take an example. Say that a QA on the team is afraid of receiving bad delivery from developers. Now assume that a developer on the team is afraid of poorly groomed stories. These fears when looked upon by themselves can cause a lot of pain and strife. Now put their fears together. Poorly groomed stories often mean that it is not clear what the expectation downstream is, so it makes harder for the developer to code which results in poor delivery to the QA. OTOH, if the whole team was aware of these fears, they could all get together to discuss how the story needs to be tested (the T in INVEST) before starting to work on it.

In other words, certain fears can help guide positive collaboration between the team members and give them the courage to deal with problems. I find retrospectives are an essential noise cancellation tool in any team's arsenal. So if you find your team skipping retrospectives or having them only for namesake, then hopefully you know what to do.

The Role of Leadership

Some researchers at Google through their project Aristotle found that high performing teams have a psychological safety. Modern Agile also lays it down as one of the four cornerstones. I believe psychological safety is everyone's responsibility but the lion's share of the responsibility falls on the leaders of the organization. Why?

Leaders of the organization heavily influence how it operates. They influence the structure; their decisions and behaviors either create or destroy walls between people based on their departments or functional groups. Even at the team level, leaders influence relationships between people. Negative influence can create mistrust, and positive influence can help the team soar even in most difficult situations.

Unfortunately, there are always some things that cannot be discussed openly even in the most open of all organizations and teams. Sometimes that creates fear. It is the leaders' job to sense what those things are and how they impact the safety of their people. Leadership's role cannot be downplayed.

Understanding True limits

When you have canceled the noise and you have leadership support, look into what you can do to learn about your current limitations. A useful conversation to have is around - To what extent can you go today without getting into trouble?

Take an example, a team perceives that it is delivering slowly. However, it does not know how fast it goes today. Now, the team may try different approaches to go faster, like slicing user stories or reducing work in progress. It may even work but if it is afraid of measuring its progress objectively and often, it may easily fall back to old ways. This would create more fear for change and would be counter productive.

The SIX STEPS: What to do?

Here are six steps to unlock your team's true potential and become the best team ever:

1. Find out what your limits are. Only if you know your limits, you can challenge them.


2. Identify what to do to get as close as possible to the limits.


3. Get permission to fail, and give yourself the permission to fail.


4. Do what you need to in order to get close to the limits.


5. Fail or not - Retrospect, learn from the experience, what can you do to push the limits? Push the limits.


6. Repeat from step 1

Unlocking your team's potential can be easier than you think. The one hard thing about it is that it requires an open culture based on trust and respect. Even then, these six steps should help take things from where they are, towards where they should be.

Image courtesy:https://pixabay.com/en/youth-active-jump-happy-sunrise-570881/

5 Agile Quotes: Kung-Fu-Panda-3

I love the Kung Fu Panda movies, hope you enjoy them too. Apart from the funny awesomeness of Po the panda, I love the subtle and not so subtle messages and quotes these movies contain. Here are few gems from Kung Fu Panda 3. I only saw it this week, so am sharing these now.

[caption id="attachment_283" align="aligncenter" width="595"]

 Rock like Po the panda![/caption]

Image courtesy: https://www.flickr.com/photos/27474697@N08/2556889334/

1. When will you realize the more you take, the less you have? - Master Oogway to Kai (the beast)

When Kai tells Master Oogway that he has taken the chi of every master there is, Master Oogway pleads to Kai to realize this. There is a virtue in giving; always give when you can! So how is this connected to agile? There are examples. It could be giving help to a struggling team member, or giving credit to someone else who deserves it, or giving someone encouragement for their effort, or giving someone any other type of support they need - mentally, physically and emotionally.

Look beyond yourself. There is great power in humility and kindness, it helps build strong relationships that help teams thrive.

2. If you only do what you can do, you'll never be better than what you are. - Master Shifu to Po

This one is straightforward. A core tenet of agile is to keep challenging oneself to do better than today. Never cease to challenge the status quo, it is a key to customer and team happiness.

3. I have so much to learn! - Po after the party at the secret Panda village

Learning is essential for an agile mindset. Learning manifests itself everywhere. There is always more to it than meets the eye, but one should be willing to see beyond what is visible.

4. You gotta let the hill tell you where to roll. - Li, Po's panda dad during Po's training

This one is interesting. Agile mindset encourages to be courageous and to experiment. Let the results tell you what to do next. Surely there can be failures along the way. Accept successes and failures as normal events, but don't let the causes become your excuses.

5. Feeling relaxed? Just let yourself fall into it - Li, Po's panda dad during Po's training

This is a natural deduction for the previous quote. Immerse yourself into the environment and let it guide your actions thoughtfully. Not only will you increase your chances of success, you will also tremendously increase the enjoyment you get from your experience.

That's all for now! If you have more quotes to share from Kung Fu Panda or anywhere else, I would love to hear :)