Stripe’s rate limiters are built on top of Redis, and until recently, they ran on a single very hot instance of Redis. The server had followers in place for failover, but at any given time, one node was handling every operation.
Catalog
Quotes tagged ‘Scalability’
Anycast is the use of routing and addressing policies to affect the most efficient path between a single source and several geographically dispersed targets that "listen" to a service within a receiver group.
Network providers use the Border Gateway Protocol to tell each other which networks they can reach and how many hops away they are. The end user ISP then, in most cases, takes the shortest route to reach the destination.
In pursuit of answers, I set up almost 60 different sysbench fileIO test configurations with different kernels, filesystems, mount options and RAID block sizes. Once the best fit configuration was chosen from these experiments, I ran another 20 or so sysbench OLTP runs with other system permutations…
For example, over 2,000 new repos are being created during peak hours, and CI runners are requesting new builds 3,000,000 times per hour. It's an interesting problem to have. We have to store this information somewhere and make sure that while we're gaining data and users, GitLab.com keeps working f…
The approach above is bad in several different ways. There is no way to control how many go routines we are spawning. And since we were getting 1 million POST requests per minute of course this code crashed and burned very quickly.
A query that used to take over an hour can run in seconds on the cache.
Finally, we sped up our application from more than 2.5 seconds to less than 250 milliseconds for longest request. These times occur just in our use case. We are confident that for a larger number of writes or longer eviction period, access to standard cache can take much more time, but with bigcache…
MySQL replication is the process by which a single data set, stored in a MySQL database, will be live-copied to a second server. This configuration, called "master-slave" replication, is a typical setup. Our setup will be better than that, because master-master replication allows data to be copied f…
MySQL cluster works best in a shared-nothing environment. Ideally, no two components should share the same hardware. For simplicity, and demonstration purposes, we'll limit ourselves to using only three Droplets. There will be two Droplets acting as data nodes which are syncing data between themselv…
We use Tornado with Python, but Go’s native support for concurrency is ideal for most new performance-critical services.
The choice to use a NoSQL database is often based on hype, or a wrong assumption that relational databases cannot perform as well as a NoSQL database. Operational costs, as well as other stability and maturity concerns, are often overlooked by engineers when it comes to selecting a database.
Our team knew the dangers and distractions of premature optimization, and figured the “easiest” way to shard was to avoid it for as long as possible.
Building an email system requires efficiently managing a huge amount of information essentially from day one. Our sync system has more data for a single user than most startup’s database of everything. To date, nearly 150k accounts have been synced by the Nylas Cloud system, and our datastore grows…