There are a lot of scenarios where we want to reduce the number of requests we do either towards our database either towards a third party API. Those may include reducing the usage cost -either referring to the need of scaling our systems, either to the usage of an external API for a specific number of requests-, and improve performance.
One of the solutions you will hear when the discussions are going on for those problems is caching. This can be described as the process of storing some parts of our data in a temporary data-store, which will typically be in-memory storage. That allows us to access the data faster when we need it and it will limit the requests towards the operational data sources. We can implement our own cache mechanism since most times what we want to do with it is simply a PUT (insert something to the cache) and a GET (retrieve something from the cache). There are a lot of data structures in all the languages that will allow us to do so. For example, we could use a dictionary in Python or a HashMap in Java or JavaScript objects. This will add a slight overhead and there are a lot of ready, well-tested solutions which provided extra functionality in case we need it. You may have heard about Redis, Memcached or Riak.
In this example, we will use Redis as a cache mechanism in an attempt to limit the request we do for our application towards another API. Redis is an open-source (BSD licensed), in-memory data structure store, used as a database, cache and message broker and it supports all of the major data structures such as Hash, List, Sets and Strings. You can find a “cheat sheet” with the commands per data structure here. Also, have a look at Redis documentation itself, which is a great resource overall. It belongs to the Key-Value family of the NoSQL databases.
Use case
Let’s assume that we are building an application, where a part of it is responsible to show the current weather data and a forecast for the next hours. There are a ton of weather APIs out there to get this info and most of them are using a pricing model like, up to x requests y amount to pay. We have picked weatherbit for our examples which provides a free tier with limit 500 requests per day and 1 per second for non-commercial use. In a production environment, we would have used one of their paid plans and we would like to limit the requests towards them as much as possible to limit down the cost. But even in the free version, there are a couple of things to consider. First that we have to respect that they are giving us free access and not spam them with requests and secondly we need to make sure that we will make good use for our free requests without wasting them. We can tackle both with a cache mechanism.
For the part that we want to implement (current weather and forecast), we will consume two different endpoints from weatherbit:
- GET /current?city={city}&country={country}
- GET /forecast/hourly?city={city}&country={country}
The first one always provide the most recent data available -let us assume 15 minutes granularity- and the second one has 1-hour granularity. This is very useful information and we will see how to use it.
Implementation
There are several Python-Redis libraries, we will use redis (pip install redis). You will be needing a running Redis instance as well -we will implement against a local installation.
The first thing we want to do is to create a connection to the Redis server. Then on the connection, we can call Redis commands to interact with the server. For example, if we want to check if the connection is ok we can use the ping command for this.
import redis
def redis_connect():
try:
client = redis.Redis(
host="localhost",
port=6379,
)
ping = client.ping()
if ping is True:
return client
except redis.ConnectionError:
print("ConnectionError")
sys.exit(1)
except redis.AuthenticationError:
print("AuthenticationError")
sys.exit(1)
This is a trivial task, we attempt to connect to the local instance and to execute the ping command. If we get it correctly we return a client to work with. We also check for connection errors and then for authentication errors and if any of those occur we stop the application. Note that is a dummy error handling.
Let us now create two methods to interact with the external API. We will use the requests library for this and since the responses have a lot of data we will keep only what is useful for our application.
import requests
BASE_URL = "http://api.weatherbit.io/v2.0"
API_TOKEN = "YOUR-API-KEY"
def get_current_weather(city, country):
response = requests.get(
f"{BASE_URL}/current?city={city}&country={country}&key={API_TOKEN}"
)
if response.status_code != 200:
return {}
data = response.json()["data"][0]
data_to_return = {
"temperature": data["temp"],
"wind_speed": data["wind_spd"],
"wind_direction": data["wind_cdir"],
"description": data["weather"]["description"],
}
return data_to_return
def get_forecast(city, country):
response = requests.get(
f"{BASE_URL}/forecast/hourly?city={city}&country={country}&hours=5&key={API_TOKEN}"
)
if response.status_code != 200:
return {}
data = response.json()["data"]
data_to_return = {"forecast": []}
for forecast in data:
hourly_forecast = {
"time": forecast["timestamp_local"],
"temperature": forecast["temp"],
"wind_speed": forecast["wind_spd"],
"wind_direction": forecast["wind_cdir"],
"description": forecast["weather"]["description"],
}
data_to_return["forecast"].append(hourly_forecast)
return data_to_return
Both methods are accepting a city and a country in order to be able to complete a proper request. We check if got a response -if not return empty dictionary for example’s shake- and then we convert the response to JSON -or you can call it a Python dictionary- and from this, we extract what we need and return it.
What we want to do next is to create our caching logic. That means that we will only perform an actual request towards weatherbit API if we do not have the response stored in Redis. To be able to implement this logic we need to create two methods to interact with Redis itself. One for putting/setting a key-value in Redis and one for retrieving the value based on its key.
def get_data_from_cache(key):
return client.get(key)
def set_data_to_cache(key, value, is_forecast):
if is_forecast:
expiration_time = 60 * 60
else:
expiration_time = 60 * 15
set_data = client.setex(
key,
expiration_time,
value=value,
)
return set_data
The first one get_data_from_cache simply attempts to retrieve the value from Redis based on its key which is provided as parameter. The set_data_to_cache as the name implies accepts a key, a value and a boolean is_forecast. The key and the value are what we want to store as a pair in Redis. The boolean parameter allows us to calculate for how long we will store it. If it is forecast data we store it for 3600 seconds (1 hour) and if it is current weather data we will store it for 15 minutes -Those timers are based on the granularity of each endpoint we discussed earlier. Based on those finally, we are invoking the setex command with key, time to live and value. If we did not want to set a time to live we could simply use the set command.
Let us implement now the previously mentioned logic for caching for the current weather.
def get_weather_optima(city, country):
current_weather_data = get_data_from_cache(key=f"current:{city}:{country}")
if current_weather_data is not None:
data_to_return = json.loads(current_weather_data)
data_to_return["from_cache"] = True
return data_to_return
else:
data = get_current_weather(city, country)
if data:
data["from_cache"] = False
data = json.dumps(data)
state = set_data_to_cache(
key=f"current:{city}:{country}", value=data, is_forecast=False
)
if state is True:
return json.loads(data)
return data
The first step here is to attempt to retrieve the value from Redis. We are using the key pattern: {type}:{city}:{country}which is derived from our application needs. If we find the value in Redis we convert it to JSON, we add a new key to identify if we found the value we wanted in Redis and return it. If we do not find the data in cached we do a normal request to the external source, we set again the found in the cache identifier, we convert the response to a JSON string and then we store that JSON string with its relevant key in Redis. We do that step because we have set up our interaction with Redis for strings. If we have done the implementation based on Hashes it would be a bit different and we could skip this conversion. In the final part, we check if it is saved in Redis and return the data from the initial response directly.
Similar is the implementation of the forecast part.
def get_forecast_optima(city, country):
current_forecast_data = get_data_from_cache(key=f"forecast:{city}:{country}")
if current_forecast_data is not None:
data_to_return = json.loads(current_forecast_data)
data_to_return["from_cache"] = True
return data_to_return
else:
data = get_forecast(city, country)
if data:
data["from_cache"] = False
data = json.dumps(data)
state = set_data_to_cache(
key=f"forecast:{city}:{country}", value=data, is_forecast=False
)
if state is True:
return json.loads(data)
return data
Those could be one method since the coding is the same pretty much, but for clarity reasons for this example, we have split them. Just note how we use the same key pattern here.
Finally, let’s create an endpoint from our side to test what we have built so far. We will use Flask in a very simple way for this.
@app.route("/weather/<country>/<city>")
def fetch_weather(country, city):
return {
"weather": get_weather_optima(city, country),
"forecast": get_forecast_optima(city, country),
}
if __name__ == "__main__":
app.run(debug=True)
This is a simple GET endpoint where we also have in the path the country and the city. Those will be passed to our ‘optima’ methods and will return the response. For example, if we run it locally we can perform a request on http://127.0.0.1:5000/weather/it/milan. The first time it will return:
{
"forecast": {
"forecast": [{},{},{},{},{}],
"from_cache": false
},
"weather": {
"description": "Scattered Clouds",
"from_cache": false,
"temperature": 5.9,
"wind_direction": "SSW",
"wind_speed": 0.708325
}
}
Note the from_cache fields. If we do one more same request before 15 minutes we will get:
{
"forecast": {
"forecast": [{},{},{},{},{}],
"from_cache": true
},
"weather": {
"description": "Scattered Clouds",
"from_cache": true,
"temperature": 5.9,
"wind_direction": "SSW",
"wind_speed": 0.708325
}
}
Last, there is a chance to get one from cached and one from the external source
{
"forecast": {
"forecast": [{},{},{},{},{}],
"from_cache": true
},
"weather": {
"description": "Scattered Clouds",
"from_cache": false,
"temperature": 5.6,
"wind_direction": "SSW",
"wind_speed": 0.619446
}
}
As long as our cached key-value pairs are alive we are managing to save some requests to the external source and also relief it from the need to constantly serve us with the fresh data.
Performance
It is also very interesting to have a look at the performance. When we build an application like this we tend to depend a lot to the external APIs for our overall performance. A cache mechanism helps a lot on this aspect as well. Let us create a decorator to time our optima methods and see the difference between cached and live results.
from functools import wraps
from time import time
app = Flask(__name__)
def time_it(my_func):
@wraps(my_func)
def timed(*args, **kw):
tstart = time()
output = my_func(*args, **kw)
tend = time()
print(f"{my_func.__name__} took {(tend - tstart) * 1000} ms to execute")
return output
return timed
We are not gonna go in details about this since decorator discussion is out of scope, just note that all it does is to get a timer when a function is called and one more when the function completes what it does. Then subtracts them and returns the result.
Let’s add it where needed -so in our optima functions:
@time_it def get_weather_optima(city, country): @time_it def get_forecast_optima(city, country)
Now let us check our endpoint and console with a new city, Rome for example.
127.0.0.1 - - [12/Dec/2020 13:45:35] "GET /weather/it/rome HTTP/1.1" 200 - get_weather_optima took 739.7031784057617 ms to execute get_forecast_optima took 618.7915802001953 ms to execute
This is the first time we searched for Rome in Italy thus our application performed a request to weatherbit API in order to be able to provide a response. Let us do now again the same request
127.0.0.1 - - [12/Dec/2020 13:50:39] "GET /weather/it/rome HTTP/1.1" 200 - get_weather_optima took 0.7479190826416016 ms to execute get_forecast_optima took 0.6015300750732422 ms to execute
As you can see the difference is huge. From approximately 700 ms we went to below 1 when we used the cached results. The tradeoff to consider here is how tolerant can we be in ‘outdated’ data.
Hopefully, this will provide an overview, primary on when it is a good idea to implement a cache mechanism for your applications but also on how if this happens to be Redis as well. We only touched the surface of Redis capabilities (caching perspective, string data structure). Consider playing around with other data structures as well since simple strings most likely will not be good enough.
The code of the example can be found here.