• Hungry Minds
  • Posts
  • 🍔🧠 Sharding 101: How to Scale Databases

🍔🧠 Sharding 101: How to Scale Databases

PLUS: Meta with 1B TPS ⚡, a free prompt generator 🤖, all the news from Google+OpenAI 🗞️

Author

Alex Zajac
May 20th, 2024

In partnership with

Happy Monday! ☀️

Welcome to the 1513 new hungry minds who have joined us since last Monday!
If you aren’t subscribed yet, join smart, curious, and hungry folks by subscribing here.

🍔 THIS WEEK’S MENU 🥗

  • 📚 Razorpay handles 1500 requests per second, database sharding scales systems for large databases, books that don't change over time

  • 🗞️ OpenAI introduces GPT-4o, Google unveils AI innovations at I/O 2024, Meta working on AI earphones with cameras

  • 👨🏻‍💻 Quick byte: Currying and partial application in Javascript

Reading time: 5 minutes

A few words from our sponsor this week:

Understand AI’s Risk and Opportunities

With the rapid growth of AI, it's crucial for businesses to understand both the risks and opportunities it presents. Discover how you can utilize AI to drive success while minimizing potential pitfalls.

In the MIT Artificial Intelligence: Implications for Business Strategy online short course you’ll gain:

  • Practical knowledge and a foundational understanding of AI's current state

  • The ability to identify and leverage AI opportunities for organizational growth

  • A focus on the managerial rather than technical aspects of AI to prepare you for strategic decision making

Food for Thought
A mindset, an example, and an action item to start the week
❝

"The only way to achieve the impossible is to believe it is possible."

Charles Kingsleigh

Mindset: Believing in possibilities unlocks our potential, empowering us to transcend self-imposed limitations and achieve remarkable feats.

Example: Netflix, once a modest DVD rental service, disrupted the entertainment industry by believing in the possibility of streaming movies over the internet.

Action Item: Identify one seemingly impossible goal and write down three reasons why achieving it might be possible.

The Rabbit Hole
Deep dives, trends, and resources curated to stay ahead

What is Database Sharding?

#11 System Design - Database Sharding

How Razorpay Scaled to Handle Flash Sales at 1500 Requests per Second

#46: A Case Study on Payment Gateway Scalability (5 minutes)

How Stripe synchronizes time across their distributed system

How Stripe uses physical and logical clocks for keeping time. Plus, a deep dive on caching in system design, how eBay uses LLMs to improve developer productivity and more.

LUC #53: Embracing Simplicity: A Look Into Stateless Design

Plus, tokenization explained, how event-driven architecture works, and how the TCP handshake works

Learn things that don't change

And take a look into books I recommend you to read in 2024. (34 books)

Staying Sharp

Why Interview Even When You’re Happy

Show, Don’t Tell

How to multiply your chances of influencing others

How Facebook served billions of requests per second Using Memcached

The largest Memcached system in the world

The 3 Keys to Working Remotely

After working remotely with multiple teams over the past four years, I’ve learned a lot about remaining effective and contributing meaningfully while not being colocated with my teammates. Throughout these experiences, I’ve learned three things that have made a world of a difference in how I work.

Communicate like a Senior: Use clear deltas

Level up your performance reviews and influence. Get clear expectations to the next level.

💾 SIDE DISHES 💾

ARTICLE (ddocs not gdocs)
Design Docs at Google

ESSENTIAL (free-wisdom)
10 lessons from 12 years at Google

The Weekly Digest
Software, AI, and startup news worth your time

Brief: GPT-4o is the newest flagship model from Open AI that provides GPT-4-level intelligence but is much faster and improves on its capabilities across text, voice, and vision: it’s free!

Brief: Google's I/O 2024 showcases new security features for Android, geospatial AR content on Google Maps, and the introduction of LearnLM for generative AI learning, among other cutting-edge AI announcements.

Brief: Meta, Facebook's parent company, is reportedly innovating AI-powered earphones with cameras named "Camerabuds," envisioned to offer live translation and real-time object identification, aiming to revolutionize wearable technology.

Brief: As part of the partnership, OpenAI will gain access to Reddit’s Data API, granting the AI company real-time, structured, and exclusive access to the vast array of content generated by Reddit’s vibrant communities

Brief: Google launches "AI Overviews" to provide users with AI-generated summaries at the top of search results, integrating Gemini AI for search accuracy.

Brief: GameStop and AMC experience significant rallies as Keith Gill, also known as "Roaring Kitty," resurfaces online after a prolonged absence, sparking trading volatility and renewed interest in meme stocks.

The Quick Byte
One coding tip because you’re technical after all

This week’s coding challenge:

Coding Challenge #62 - Git Contributions Visualisation Tool

This challenge is to build your own version of the GitHub contributions visualisation.

This week’s tip:

Type hints are a way to specify the expected types of variables, function parameters, and return values in Python. They provide static type checking and improve code readability and maintainability.

Wen?

  • Large and Complex Codebases: Type hints are particularly useful in large and complex codebases, where it becomes increasingly difficult to keep track of the expected types of variables, function parameters, and return values.

  • Collaborative Development: When working on a project with multiple contributors, type hints help ensure that everyone adheres to the same type conventions, reducing the likelihood of type-related bugs and misunderstandings.

  • Library and Framework Development: If you are developing a library or framework intended for use by others, type hints provide valuable documentation and ensure that your code is used correctly by consumers.

Why?

  • Static Type Checking: Type hints enable static type checking tools like mypy to analyze your code and catch type-related errors before running the code, improving overall code quality and reducing runtime errors.

  • Better Code Documentation: Type hints act as self-documenting code, making it easier for developers to understand the expected types of variables, function parameters, and return values without needing to dig through separate documentation.

  • Improved Tooling Support: Modern IDEs and code editors can leverage type hints to provide better autocompletion, code navigation, and refactoring capabilities, enhancing the overall development experience.

Burp-A-Laugh
The most important meal of your day

That’s it for today! ☀️

Enjoyed this issue? Send it to your friends here to sign up, or share it on Twitter!

If you want to submit a section to the newsletter or tell us what you think about today’s issue, reply to this email or DM me on Twitter! 🐦

Thanks for spending part of your Monday morning with Hungry Minds.
See you in a week — Alex.

Icons by Icons8.

*I may earn a commission if you get a subscription through the links marked with “aff.” (at no extra cost to you).