Use monkey patching to continue developing applications depending on external API calls while waiting for access
Sometimes you don’t have the time to wait for API keys being validated and ready to be used. Although the documentation is open to the world, the whole proces of waiting on getting certified keys (mostly within business setting) can take ages. These situations could halt your own work and thus the general development speed. There is however, a way to ignore the use of ‘working’ API keys and still move on with developing your application.
This article will describe a “good practice” workaround combining the documentation of the external API, a method written to query that API call and monkey patching that same method in order to return a predefined result. All this by using a unit testing approach. So yes, using this practice will result in having unit tests in place for your application. Win-win.
Normally, unit test are used to make sure code is working as it should be. Logic is applied to test the output of the functions or methods. This article takes the unit test approach to develop faster, by mocking the API calls. The documentation of the external API is used to make sure the mocked call is returning the right structure. Making sure the right structure is returned, opens up the way to use the mocked API for other parts of the application. Whenever the API keys are certified and working, the application will work on live data. Who needs working keys anyway?
Prerequisites
For this example we will be using the Google Maps API and a python package called googlemaps. For this example we will use the find_place() method within the googlemaps.Client class to query the API. But as I don’t have any working keys yet, I will monkey patch this method in order to move on with the further development of the application.
For executing the unit tests I will be using pytest. Install using the command pip install pytest in your environment.
For monkey patching the method, I will use the pytest-mock module built for the pytest package. Monkey patching is part of the pytest-mock package. Install using pip install pytest-mock in your environment.
We are about to mimic the method find_place(), part of the Client class available in the googlemaps package. Install using pip install googlemaps in your environment.
My code is using the python-dotenv package to handle environment variables, configured in the .env file in root. The .env file contains a working key to initalise the googlemaps.Client class but without permissions to query any API from Google yet.
The goal
The custom function query_address() requires an address_code and a googlemaps.Client initialised with a working key, in order to retrieve a set of options and see which one fits the address best based on Google Places. The function first checks the address_code using a regex to verify whether the format is valid. When succesfully validated the function will call the find_place() method to call the API for searching on that set address. Whenever the key response in the JSON is set to OK return the candidates key to the code invoking the function.
The find_place method is coded by the guys from Google and by looking at the documentation I know beforehand that the input requires a string as the search query and another string named input_type telling the method which kind of search it should do. As the app wants to search for places on an address, the input_type is set to textquery.
The challenge
If this function would be executed with some valid customer_address_code the Client would throw an error as the key is not valid yet. Therefore this exact method is being monkey patched by mocking the function in a test.
Within a directory called tests, another file which tests the query_address.py is made. This file contains a function starting with the prefix test_ which will be picked up when running pytest .. In this example the function is named test_query_address.py which contains the tests for the function. The test does need to know that the original find_place method needs to be overwritten by another package.
Faking the method
Create a new function called find_place_fixed inside the test_query_address.py python file. Make sure it copies the exact same blueprint of the original method. This blueprint can be found in the original code of the Client class.
As this method is part of a class, the first argument should be the object itself. Normally the argument is called self but as it seems, the guys at google changed it to client. The original method contains the input and the input_type argument.
The fixed function should return my own desired output, in order to go on with the logic in other places of the application. This logic should however contain the exact same structure as when the working API returns. This function returns the exact example from the documentation, to make sure it would work in the future whenever the key is activated and ready to be used.
Overwriting the method
Using pytest-mock and fixtures (making sure the function is executed before the test are run) it is possible to overwrite the function only within the test file.
To activate the monkey patching functionality, the only argument for the find_place_patches function is monkeypatch. Within the function the package googlemaps is imported. The reason for putting the import in the function is to make sure that no other function with the same name is monkey patched (import order drama’s). The monkeypatch object from the argument is used to set the attribute, which overwrites the original code.
The target is the googlemaps.Client module, the name of the method is find_place and the value should be our new fixed function called find_place_fixed.
Running the test
Finally it is time to test the application code without actually having a working API key (yet). As stated before, a function called test_query_address is used to test the code.
The function has one argument, the name of the function find_place_patched which monkey patches the desired methods. As with every other pytest, the code is tested by using assert statements. Since it is known what the output for the API call should be, it easy to test if the code is working and the logic is correct.
Within the query_address code there is a check if the status key equals “OK”. This is the case for our monkey patched function. So whenever the monkey patch part works, it should return the JSON but only the candidates part. As in the example the candidates key consists of a list of one candidate, the length should equal 1. The namekey of the the first item in the list should equal the one in the monkey patch output.
Run the test using pytest . and make sure all tests pass. The full test file including other tests for validating using regex right is printed below.
As the full example consists of multiple test functions, having different scopes, having the need for using the monkey patched code it is possible to automatically use the monkey patched function. This can be achieved by setting the argument autouse to True in the fixture of the find_place_patched function. If you ever feel like reusing the monkey patched function in other test files, have a look at adding a scope to the fixture.
From here it is possible to move on with the programmer life and write code based on the fixed output of the query_address function. Keep on testing and thus debugging your code using the test combined with the monkey patch functionality and your application will be ready before the keys are. Cheers.
Discussion
Why use unit tests to debug the code? Good question. Of course it would be more straightforward to mock the code within the original python script and refactor whenever the keys are there. But, this approach directly tests the logic and does not need any altering of the code whenever the key is valid. If you don’t agree, please shout at me in the comment section for a good and fruitful discussion.
Don’t forget to give me a clap and a follow (in case you think I deserve one).
