How Apple built iCloud to store billions of databases (10 minute read)
Apple's iCloud and CloudKit were built using a combination of Cassandra and FoundationDB. Cassandra, which is being actively improved by Apple engineers, handles exabytes of data and millions of queries. FoundationDB powers CloudKit, Apple's cloud backend service. It provides key features like transactional guarantees and solves scalability issues that Cassandra alone could not address. Each iCloud user gets their own database, resulting in billions of databases. This is made possible by the Record Layer's stateless architecture and resource allocation using a “record store” abstraction.
|
How Meta animates AI-generated images at scale (12 minute read)
Meta's animate feature, which allows users to generate short animations from AI-generated images, had big challenges in scalability and efficiency. To overcome these challenges, the team implemented a series of optimizations, including halving floating-point precision, improving temporal-attention expansion, leveraging DPM-Solver to reduce sampling steps, and combining guidance and step distillation. They also deployed a traffic management system for better routing and less latency, as well as a retry system with optimized settings to reduce errors.
|
Fixing a Bug in Google Chrome as a First-Time Contributor (17 minute read)
This developer fixed a bug in the Chromium/Google Chrome web browser, their first contribution to an open-source project of this scale. They detail the process of building Chromium from source, finding the bug in the Devtools network inspector, debugging the code, and finally creating a fix. They also talk through the process of submitting the code, getting it reviewed, and the final impact of their contribution.
|
|
Predicting the Future of Distributed Systems (18 minute read)
Object storage is becoming more popular for both transactional and analytical workloads. Adopting new programming models presents a greater challenge due to the lack of clear migration paths and the perceived high investment risk. Object storage in the future may become a ubiquitous component of distributed systems as new programming models emerge, shifting code from applications to infrastructure.
|
Dokku: my favorite personal serverless platform (6 minute read)
Dokku is an open-source platform-as-a-service that allows users to create a Heroku-like environment on their own servers. It comes with features like automatic SSL certificate management, basic authentication, and flexible scaling. This developer uses Dokku to deploy various applications for their LLM consulting work, especially for non-GPU workloads.
|
|
av (GitHub Repo)
av is a command-line tool that helps you manage your stacked PRs on GitHub. It allows you to create a PR stacked on top of another PR, and it will automatically update the dependent PR when the base PR is updated.
|
Protobuf-ES 2.0 Released (8 minute read)
Protobuf-ES 2.0 is a significant update to the Protobuf runtime library for JavaScript and TypeScript. The release brings major improvements, including full support for Protobuf Editions, new APIs for field presence and default values, and a complete runtime reflection API.
|
Tach (GitHub Repo)
Tach is a Python tool that enforces dependencies using modular architecture.
|
|
What the interns have wrought, 2024 edition (15 minute read)
Every year, Jane Street presents what its interns worked on that summer. Arya Maheshwari developed Camels, an OCaml dataframe library, while another intern, Arvin Ding, designed a faster binary serialization protocol, resulting in improved tail latencies in production systems. Alex Li enhanced Limshare's time-travel debugger by integrating snapshotting, making it easier to investigate production incidents and understand the history of state changes.
|
Stripe Data vs Open‐Source Alternatives: an MRR example (11 minute read)
Stripe makes it difficult to calculate crucial metrics like MRR through its API, forcing companies to use add-ons like Stripe Data Pipeline and Stripe Sigma. This strategy creates a barrier for businesses, especially those with usage-based billing models, making data analysis challenging. As a result, third-party tools and open-source billing solutions have emerged to fill the gap.
|
Erasure Coding for Distributed Systems (20 minute read)
This blog post explores the use of erasure coding in distributed systems through practical examples, explaining its benefits in terms of storage efficiency, fault tolerance, and tail latency reduction while discussing the trade-offs involved, such as increased IOPS and decoding costs.
|
|
Linux Pipes are Slow (12 minute read)
Linux pipes are significantly slower than expected due to the kernel's inefficient memory management and lack of vectorization when copying data, leading to a 10x slowdown compared to writing directly to memory.
|
avante.nvim (GitHub Repo)
avante.nvim is a Neovim plugin that provides users with AI-driven code suggestions and the ability to apply these recommendations directly to their source files with minimal effort.
|
|
|
Love TLDR? Tell your friends and get rewards!
|
|
Share your referral link below with friends to get free TLDR swag!
|
|
|
|
| Track your referrals here. |
|
Want to advertise in TLDR? 📰
|
If your company is interested in reaching an audience of web developers and engineering decision makers, you may want to advertise with us.
If you have any comments or feedback, just respond to this email!
Thanks for reading,
Priyam Mohanty, Jenny Xu & Ceora Ford
|
|
|
|
