index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
      <title>rafaelcapucho.com</title>
      <generator uri="https://hugo.spf13.com">Hugo</generator>
    <link>https://rafaelcapucho.github.io/</link>
    <language>en-us</language>
    <author>Rafael Capucho</author>
    <copyright>2017 Rafael Capucho</copyright>
    <updated>Wed, 18 Jan 2017 00:00:00 UTC</updated>
    
    
    
    
    <item>
      <title>Enhance the quality of your API calls with Client-Side Throttling</title>
      <link>https://rafaelcapucho.github.io/2016/10/enhance-the-quality-of-your-api-calls-with-client-side-throttling/</link>
      <pubDate>Sun, 02 Oct 2016 00:00:00 UTC</pubDate>
      <author>Rafael Capucho</author>
      <guid>https://rafaelcapucho.github.io/2016/10/enhance-the-quality-of-your-api-calls-with-client-side-throttling/</guid>
      <description>

&lt;p&gt;If you are a backend engineer and work with high load environment you &lt;strong&gt;should&lt;/strong&gt; be familiarized with the old Server-Side Rate Limit, mainly for security concerns.&lt;/p&gt;

&lt;p&gt;Didn&amp;rsquo;t you ever heard about Server-Side Rate Limit? It is a technique to limit the amount of requests that the server or API can handle in a defined period of time, and that limit can be set per user or not.&lt;/p&gt;

&lt;p&gt;Throttling is often used to limiting traffic on the server, mainly as a way to protect against &lt;a href=&#34;https://en.wikipedia.org/wiki/Denial-of-service_attack&#34;&gt;DOS Attack&lt;/a&gt;. Additionally, it can also be used on the client-side, as this article describes.&lt;/p&gt;

&lt;p&gt;When a rate limit is exceeded, all requests are throttled and fail for a brief period of time or until the activities back to normal. Once throttled, the caller usually get the HTTP 403 (forbidden) status as response.&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://rafaelcapucho.github.io/img/client-side-throttling/Rate-Limit-and-Throttling-Graph.jpg&#34; alt=&#34;Rate Limit &amp;amp; Throttling Graph&#34; /&gt;&lt;/p&gt;

&lt;p&gt;In the above picture, the user #3 are exceeding the global or individual rate limiting, being throttled to guarantee the quality of service and security.&lt;/p&gt;

&lt;h2 id=&#34;just-reject-the-requests-is-enough&#34;&gt;Just reject the requests is enough?&lt;/h2&gt;

&lt;p&gt;When a user is throttled, his next requests calls will be rejected. This behavior consumes significantly fewer resources than actually processing the requests and serving back correct responses.&lt;/p&gt;

&lt;p&gt;If a request were simple, like only retrieving information from local memory, it is almost equally expensive to reject it as it is to accept it and return the expected result.&lt;/p&gt;

&lt;p&gt;Additionally, we should consider that the server also allocates resources to receive the requests, establishing a complete TCP connection to finally reject them. Helping DDoS attackers.&lt;/p&gt;

&lt;h2 id=&#34;big-mistake-when-consuming-apis&#34;&gt;Big mistake when consuming APIs.&lt;/h2&gt;

&lt;p&gt;Consider that you are creating the new version of Facebook client, that makes periodics requests to Facebook API to collect news feed, comments, likes, online users etc.&lt;/p&gt;

&lt;p&gt;Almost everytime happens that unprepared clients, once throttled and getting rejected requests from API, keep making the same amount of requests, trying to reach the data.&lt;/p&gt;

&lt;p&gt;As result, the API keep blocking the user to maintain the overall quality of the service and your client could never retrieve the data as desired.&lt;/p&gt;

&lt;h2 id=&#34;how-to-apply-client-side-throttling-with-adaptive-throttling&#34;&gt;How to apply Client-Side Throttling with Adaptive Throttling?&lt;/h2&gt;

&lt;p&gt;When a client detects that a significant portion of its recent requests have been rejected, it starts self-regulating and caps the amount of outgoing traffic it generates.&lt;/p&gt;

&lt;p&gt;Requests above the cap should fail locally without even reaching the network.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;em&gt;Adaptive throttling&lt;/em&gt;&lt;/strong&gt; is the key to solve the problem. To implement this technique the client needs to keep the following information for at least two minutes of its history:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Requests&lt;/strong&gt;: The number of requests attempted by the application.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Accepts&lt;/strong&gt;: The number of requests accepted by the backend.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;When everything is fine at the client, the two values are equal. Once the API start rejecting the client requests the number of &lt;em&gt;Accepts&lt;/em&gt; becomes smaller than the number of &lt;em&gt;Requests&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;The client app should still be able to issue requests to the API until the &lt;strong&gt;Requests&lt;/strong&gt; be &lt;strong&gt;K&lt;/strong&gt; times grather than &lt;strong&gt;Accepts&lt;/strong&gt;, assuming K &amp;gt; 1.&lt;/p&gt;

&lt;p&gt;For example, &lt;strong&gt;K = 1.1&lt;/strong&gt; means that the client will allow one &lt;em&gt;Reject&lt;/em&gt; for each 10 &lt;em&gt;Accepts&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Once the threshold is surpassed, which will happen when the state of the system meet the criteria &lt;strong&gt;Requests &amp;gt; k * Accepts&lt;/strong&gt;, the app should be able to self-regulate and new requests are now reject locally based on the probability P&lt;sub&gt;0&lt;/sub&gt; described below.&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://rafaelcapucho.github.io/img/client-side-throttling/adaptative-throttling.jpg&#34; alt=&#34;Adaptive throttling Probability&#34; /&gt;&lt;/p&gt;

&lt;p&gt;Example, the client app performed &lt;strong&gt;1000 Requests&lt;/strong&gt; in the last 2 minutes, with &lt;strong&gt;600 of them Accepted&lt;/strong&gt;. Also, our system is designed to allow 4 rejects for each 10 requests which means that our &lt;strong&gt;K = 1.4&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In the previous scenario the app should start regulating new request based on P&lt;sub&gt;0&lt;/sub&gt; since the initial criteria is meet:&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://rafaelcapucho.github.io/img/client-side-throttling/initial-criteria.jpg&#34; alt=&#34;Initial Criteria&#34; /&gt;&lt;/p&gt;

&lt;p&gt;Since the client app is issuing more requests than our sample &lt;strong&gt;K&lt;/strong&gt; allow, the app should reject new requests locally to the server with probability equal to 0.16, rejecting 16 requests in 100:&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://rafaelcapucho.github.io/img/client-side-throttling/probability-answer.jpg&#34; alt=&#34;Probability Answer&#34; /&gt;&lt;/p&gt;

&lt;h2 id=&#34;the-problem-with-that-approach&#34;&gt;The problem with that approach&lt;/h2&gt;

&lt;p&gt;The P&lt;sub&gt;0&lt;/sub&gt; probability is originally defined in the Chapter 21 of the book &lt;a href=&#34;https://www.amazon.com/Site-Reliability-Engineering-Production-Systems/dp/149192912X&#34; target=&#34;_blank&#34;&gt;Site Reliability Engineering: How Google Runs Production Systems&lt;/a&gt; by, guess who? Google guys.&lt;/p&gt;

&lt;p&gt;The main goal is to increase the probability of dropping new requests, as soon as the server appears to be overloaded (assuming a small value  of &lt;strong&gt;K&lt;/strong&gt;). Doing this, every client instance will help the server recover faster from the outage, by using less resources.&lt;/p&gt;

&lt;p&gt;Also, this approach is used in some Google services and they suggest K = 2, based on their own experiences.&lt;/p&gt;

&lt;p&gt;My concerns about this probability is that, if the server goes down for more than 2 minutes, the P&lt;sub&gt;0&lt;/sub&gt; value will stand in 1, rejecting locally every new request to the server, so the client app won&amp;rsquo;t be able to set up a new conection. As the result of it, the client app will never have another request reaching the server.&lt;/p&gt;

&lt;p&gt;I would like to suggest a new probability measure, P&lt;sub&gt;1&lt;/sub&gt;, that upper limits the chance of rejecting a new request to 90%, allowing the client to recover even in that worst scenario, when the service is down more than 2 minutes.&lt;/p&gt;

&lt;p&gt;&lt;img src=&#34;https://rafaelcapucho.github.io/img/client-side-throttling/alternative-probability-measure.jpg&#34; alt=&#34;Alternative Probability Measure&#34; /&gt;&lt;/p&gt;

&lt;p&gt;I have noted that this technique can benefit client libraries and apps to enhance their own quality. Also, companies that offer comunication through APIs, in order to improve their own quality, should encourage client designers to use techniques like these.&lt;/p&gt;

&lt;p&gt;Cheers,&lt;br&gt;
Rafael.&lt;/p&gt;
</description>
    </item>
    
    
    
    <item>
      <title>Speeding up your MongoDB queries up to 30 times with Tornado</title>
      <link>https://rafaelcapucho.github.io/2016/09/speeding-up-your-mongodb-queries-up-to-30-times-with-tornado/</link>
      <pubDate>Mon, 05 Sep 2016 00:00:00 UTC</pubDate>
      <author>Rafael Capucho</author>
      <guid>https://rafaelcapucho.github.io/2016/09/speeding-up-your-mongodb-queries-up-to-30-times-with-tornado/</guid>
      <description>

&lt;p&gt;Sometime ago, I was facing a &lt;em&gt;problem&lt;/em&gt; using MongoDB to perform heavy operations like Aggregations over a considerable amount of documents.&lt;/p&gt;

&lt;p&gt;Fortunately, most of them were repeating, like when your projects are fetching contents from database to make the same menu structure, it doesn&amp;rsquo;t change every second.&lt;/p&gt;

&lt;p&gt;We already know how is the recipe to solve that: &lt;strong&gt;Cache it&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;I was using &lt;a href=&#34;http://www.tornadoweb.org&#34;&gt;Tornado&lt;/a&gt; and it&amp;rsquo;s common to use &lt;a href=&#34;https://github.com/mongodb/motor&#34;&gt;Motor&lt;/a&gt; to perform async database operations over MongoDB. The idea was to implement a Mixin that can be used in all handlers and support the three most common operations to get data:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Find&lt;/li&gt;
&lt;li&gt;Find One&lt;/li&gt;
&lt;li&gt;Aggregate&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Ok, let&amp;rsquo;s do it.&lt;/p&gt;

&lt;h2 id=&#34;establishing-the-connection&#34;&gt;Establishing the connection&lt;/h2&gt;

&lt;p&gt;We won&amp;rsquo;t need a lot of stuffs here, just the boilerplate.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;import logging
import motor

from pymongo.errors import ConnectionFailure
from tornado.options import define, options

define(&amp;quot;mongo_host&amp;quot;, default=&amp;quot;mongodb://localhost:27017&amp;quot;, help=&amp;quot;mongodb://user:pass@localhost:27017&amp;quot;)
define(&amp;quot;mongo_db&amp;quot;, default=&amp;quot;database_name&amp;quot;)

def create_connection():
    try:
        return motor.MotorClient(options.mongo_host)[options.mongo_db]

    except ConnectionFailure:
        logging.error(&#39;Could not connect to Mongo DB. Exit&#39;)
        exit(1)
        return False
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now we&amp;rsquo;re able to establish the connection and the object will be available through &lt;code&gt;self.application.db&lt;/code&gt; in every handler. Other aspects of the code like imports are omitted.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;class Application(tornado.web.Application):
    def __init__(self):
        handlers = [
            (r&amp;quot;/?&amp;quot;, MainHandler),
        ]

        self.db = create_connection()

        tornado.web.Application.__init__(self, handlers)

http_server = tornado.httpserver.HTTPServer(Application())
http_server.bind(options.port)
http_server.start(1)

main_loop = tornado.ioloop.IOLoop.instance()
main_loop.start()
&lt;/code&gt;&lt;/pre&gt;

&lt;h2 id=&#34;the-mixin-code&#34;&gt;The Mixin Code&lt;/h2&gt;

&lt;p&gt;With the connection available, it&amp;rsquo;s our Mixin time.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import print_function
from __future__ import unicode_literals

from hashlib import md5
import datetime

import tornado.web
from tornado import gen
from tornado.options import define, options

from handlers import DBMixin

define(&amp;quot;cache&amp;quot;, default=&amp;quot;True&amp;quot;, type=bool, help=&amp;quot;Enable/Disable the internal cache&amp;quot;)

_cache = {}

class DBMixin(object):
    @property
    def db(self): return self.application.db

class CacheHandler(tornado.web.RequestHandler, DBMixin):
    def __init__(self, *args, **kwargs):
        super(CacheHandler, self).__init__(*args, **kwargs)
        self.FIND_ONE = 1
        self.FIND = 2
        self.AGGREGATE = 3

    @gen.coroutine
    def cache(self, type, col, *args, **kwargs):

        memory = kwargs.pop(&#39;memory&#39;, True) # Local Memory (Ram)
        timeout = kwargs.pop(&#39;timeout&#39;, 60)
        sort = kwargs.pop(&#39;sort&#39;, None)

        # Union of every input
        signature = str(type)+col+str(args)+str(kwargs)

        # Generate a unique key 
        key = md5(signature.encode(&#39;utf-8&#39;)).hexdigest()

        @gen.coroutine
        def get_key(key):

            if not options.cache:
                raise gen.Return(False)

            if memory:
                if _cache.get(key, False):
                    raise gen.Return(_cache[key])
                else:
                    raise gen.Return(False)

            else:
                data = yield self.db[&#39;_cache&#39;].find_one({&#39;key&#39;: key})
                raise gen.Return(data)

        @gen.coroutine
        def set_key(key, value):

            delta = datetime.datetime.now() + datetime.timedelta(seconds=timeout)

            if memory:
                _cache[key] = {
                    &#39;d&#39;: value,
                    &#39;t&#39;: delta
                }

            else:
                yield self.db[&#39;_cache&#39;].insert({
                    &#39;key&#39;: key,
                    &#39;d&#39;: value,
                    &#39;t&#39;: delta
                })

        @gen.coroutine
        def del_key(key):
            if memory:
                if _cache.get(key, False): del _cache[key]
            else:
                yield self.db[&#39;_cache&#39;].remove({&#39;key&#39;: key})

        _key = yield get_key(key)

        if _key:

            # If the time in the future is still bigger than now
            if _key[&#39;t&#39;] &amp;gt;= datetime.datetime.now():
                raise gen.Return(_key[&#39;d&#39;])
            else: # Invalid
                yield del_key(key)

        # otherwise (key not exist)
        if type == self.FIND_ONE:

            data = yield self.db[col].find_one(*args, **kwargs)

        elif type == self.FIND:

            if sort:
                cursor = self.db[col].find(*args, **kwargs).sort(sort)
            else:
                cursor = self.db[col].find(*args, **kwargs)

            data = yield cursor.to_list(kwargs.pop(&#39;to_list&#39;, None))

        elif type == self.AGGREGATE:

            cursor = self.db[col].aggregate(
                kwargs.pop(&#39;pipeline&#39;, []),
                *args,
                cursor = kwargs.pop(&#39;cursor&#39;, {}),
                **kwargs
            )

            data = yield cursor.to_list(kwargs.pop(&#39;to_list&#39;, None))

        if options.cache:
            # Persist the key
            yield set_key(key, data)

        raise gen.Return(data)
&lt;/code&gt;&lt;/pre&gt;

&lt;h2 id=&#34;how-it-works&#34;&gt;How it works?&lt;/h2&gt;

&lt;p&gt;The code is simple, basically we can use Local Memory or a MongoDB table as cache, and we are able to choose which one of them by setting the argument &lt;code&gt;memory&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Also, your handler should be using our Mixin, as in the example.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;class MainHandler(tornado.web.RequestHandler, CacheHandler):

    @gen.coroutine
    def get(self):

        category = yield find_category_by_slug(&#39;electronic-gadgets&#39;)

        self.render(&#39;index.html&#39;, {&#39;category&#39;: category})

    @gen.coroutine
    def find_category_by_slug(self, slug, filters={}):

        defaults = {&#39;slug&#39;: slug}
        defaults.update(filters)

        category = yield self.cache(
            self.FIND_ONE, # Type of query
            &#39;categories&#39;,  # Name of the document
            defaults,      # Conditions
            memory=True,   # Store on python memory 
            timeout=3600   # Expire in 1 hour
        )

        raise gen.Return(category)

    @gen.coroutine
    def find_categories(self, slug, filters={}):

        categories = yield self.cache(
            self.FIND,              # Type of query
            &#39;categories&#39;,           # Name of the document
            filters,                # Conditions
            sort = [(&#39;order&#39;, 1)],  # Sort Criteria
            memory = False,         # Store on db table
            timeout = 86400,        # Expire in 1 day
            to_list=50              # Return up to 50 items
        )

        raise gen.Return(categories)

    @gen.coroutine
    def complex_aggregation(self, category, sort, limit):

        pipeline = [
            {&#39;$match&#39;:
                {&#39;cats_ids&#39;: {&#39;$all&#39;: [ ObjectId(category[&#39;_id&#39;]) ]}}
            },
            {&#39;$project&#39;: 
                {&#39;products&#39;: 1, &#39;min_price&#39;: 1, &#39;max_price&#39;: 1, &#39;n_products&#39;: 1, &#39;_id&#39;: 1}
            },
            {&#39;$sort&#39;: sort },
            {&#39;$skip&#39;: self.calculate_page_skip(limit=limit)},
            {&#39;$limit&#39;: limit}
        ]

        groups = yield self.cache(
            self.AGGREGATE,      # Type of query
            &#39;products_groups&#39;,   # Name of the document
            memory=False,        # Store on db table
            pipeline=pipeline    # Operations
        )

        raise gen.Return(groups)
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;As you can see, there are always an opportunity to enhance the performance.&lt;/p&gt;

&lt;p&gt;&lt;a href=&#34;https://gist.github.com/rafaelcapucho/571a5ce63bc0fb0d4c43b445324ae58c&#34;&gt;Download the Gist&lt;/a&gt;&lt;/p&gt;

&lt;h3 id=&#34;how-can-i-know-if-it-worth-the-price&#34;&gt;How can I know if it worth the price?&lt;/h3&gt;

&lt;p&gt;It doesn&amp;rsquo;t count if you don&amp;rsquo;t see it, right? Ok, lets measure it.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import print_function
from __future__ import unicode_literals

import time

class Timer(object):
    def __init__(self, verbose=False, desc=&amp;quot;&amp;quot;):
        self.verbose = verbose
        self.desc = desc

    def __enter__(self):
        self.start = time.time()
        return self

    def __exit__(self, *args):
        self.end = time.time()
        self.secs = self.end - self.start
        self.msecs = self.secs * 1000  # millisecs
        if self.verbose:
            if self.desc:
                print(&#39;Time elapsed of {0} - &#39;.format(self.desc), end=&amp;quot;&amp;quot;)
            print(&#39;{0:f} ms&#39;.format(self.msecs))
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now we are able to test it.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;with Timer(True, desc=&#39;getting the category with cache&#39;) as t:
    category = yield self.cache(self.FIND_ONE, &#39;categories&#39;, {&#39;slug&#39;:&#39;cars&#39;}, memory=True)

with Timer(True, desc=&#39;getting the category without cache&#39;) as t:
    category = yield self.db.categories.find_one({&#39;slug&#39;: &#39;cars&#39;})
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;I&amp;rsquo;m pretty sure that you will see the spent time near to 0ms with the cache.&lt;/p&gt;

&lt;p&gt;Cheers,&lt;br /&gt;
Rafael.&lt;/p&gt;
</description>
    </item>
    
    
  </channel>
</rss>