New Series — Baseline System Design
Starting Resources For A System Design Interview
For reasons that are unclear to me, DesignGurus has apparently severed its connection to Educative and Grokking The System Design Interview is now only available through the designgurus.io site. Educative now has a different course they call “Grokking MODERN System Design Interview FOR ENGINEERS & MANAGERS,” and I would love to hear the story behind that. I have had a couple interactions with Arslan Ahmad, as I contributed a couple of times to a Medium publication he curates, but I certainly have not had enough interactions to ask him about it.
Though it is not necessarily related, I see that Yangshun Tay has removed his Educative sponsorship links in Grind75.
So, as it currently stands today, if you want to study for a system design interview you can:
- Pay $16 a month to use Grokking The System Design Interview
- Pay $59 for lifetime access to Grokking The System Design Interview, with that exact same link
- Pay $16, $24, or $30 a month for Educative (at a glance, the $16 a month option looks fine), which has lots of courses but is something I explicitly purchased (and no longer have) for the two Grokking courses it no longer has. I cannot speak to their replacement one
Or, you can
- Look at System Deisgn Primer for free
- Watch Guaguarav Sen for free
- Read Medium blog posts with a $5-a-month subscription, targeting reputable people like the creator of Grokking The System Design Interview
…or you could do a mix of the two.
I have seen these two GitHub resources recommended, as well:
Some Of My Random Thoughts
I thought that I had written quite a bit about system design on this blog, but I was mistaken. The only thing I wrote was this, which provided just a little bit of information about how Twitter works:
I had a series I was starting called “Software Engineer Failures,” which allowed me to read some interesting papers on the Therac-25 disaster, but I think I speak for everyone when I say that just because you know how to crash a car does not mean you know how to design one.
Twitter, I think, could serve as a quick “resource comparison” because so much of its design is up in the air now. Instead of discussing its strengths and shortcomings in some sort of private company email, Twitter’s internal engineer arguments have taken place on Twitter.
In spite of my confusion with the Educative/DesignGurus split, I did enjoy Fahim ul Haq’s blog post about Twitter’s new design.
This lays out how Twitter actually works, including names such as Manhattan, their key-value store, Gizmoduck, which stores all user information in profiles, and Tweety Pie, an object that represents a tweet in the Tweet API. I wanted to find something from Twitter itself to corroborate the use of Gizmoduck, and…there it is.
It is near the bottom, right below Manhattan. This was Twitter’s design nine months ago.
Here is the Twitter technical blog, writing about Manhattan ten years ago:
In contrast, System Design Primer and “Grokking The System Design Interview” both focus on “Toy Twitter,” a hypothetical sketch you might make of how Twitter MIGHT work if it were designed in 45 minutes. “System Design Primer” and “Grokking” are not carbon copies by any means — I find “Grokking” to be much easier to navigate, with “System Design Primer” often dropping links and, in this case, covering just one specific part of Twitter.
Both emphasize, even in “Toy Twitter,” that Twitter is read-heavy. From there, the two diverge.
- Only “Grokking” mentions an emphasis on availability over consistency
- Only “Grokking” goes into some detail about sharding, whereas “System Design Primer” simply references the word
- “System Design Primer” explicitly mentions “Fanning out” tweets, which is what Twitter really does
The Fanout Service is a potential bottleneck. Twitter users with millions of followers could take several minutes to have their tweets go through the fanout process. This could lead to race conditions with @replies to the tweet, which we could mitigate by re-ordering the tweets at serve time.
We could also avoid fanning out tweets from highly-followed users. Instead, we could search to find tweets for highly-followed users, merge the search results with the user’s home timeline results, then re-order the tweets at serve time.
— From System Design Primer https://github.com/donnemartin/system-design-primer/blob/master/solutions/system_design/twitter/README.md
SQL vs. NoSQL makes for a more interesting discussion here. You can read about Twitter’s “chronological evolution of scale” here, in that section, but it is a little more complicated than simply choosing SQL or NoSQL. They originally used MySQL, then moved to many replicated MySQL clusters with Gizzard, then…for lack of much more description…FlockDB, Snowflake, and Cassandra entered the mix. Cassandra did not fully replace MySQL, and in 2014 they launched Manhattan.
Here is more about how Twitter actually does sharding.
Closing Thoughts
Going over the Twitter blog links could probably be its own blog post, making this one a two-part miniseries.
The YouTube channel below argues that system design interviews are not a great indicator of skill
…but it’s such a “me” thing to link a video like that prematurely. The truth is, at least in my opinion, everything a person studies for a system design interview is potentially useful in the field.
A wise engineer may hypothetically argue, however, that we should genuinely study system design by reading books like this:
