This post by Maria Vechtomova really resonates:

Do you have the scale of YouTube, Netflix, Meta?

If not, do not use their architecture as a reference. You might be overcomplicating things.

Solve your own problems that fit your scale!

I'm reminded of when I had just published Parallel R and was getting lots of Hadoop-related inbound. Every company wanted Hadoop but none could explain why. And when I checked, hardly any of them needed it.

(The same scenario has played out for cloud, for mobile, for data science and ML, and now for genAI/LLMs. It will probably go the same way for the next iteration of the hype cycle. It's all fad diets, all the way down.)

The takeaway lesson? Don't worry about what others are doing. You have to find out what works for you.

Start with your company, your business model, and your challenges, then see how The Shiny New Thing might help. While you're at it, ask whether another, less-shiny tool will work.

That's it.

(Longtime readers will recognize this as the core of developing a data strategy. Something I've been talking about for … a while now. Probably since 2010-2011, after which I included it in a conference talk in 2013, then finally wrote it down in a blog post in 2016. Being practical stands the test of time.)