I was working on a school project and I wanted to build an Azure Functions that runs Python code with a server-less consumption plan, and the function needs to do the following:
- Retrieve some data using the official library of a market data API
- Write the data into Cosmos DB
And one problem I encountered was that the function was too slow, because reading something over the network and writing something into the database are both I/O-intensive actions. And running I/O-intensive actions sequentially won’t do anything good to the application in terms of execution time, so it would be good to run these independent I/O-intensive actions in parallel.
Then I started looking at how to run my code in an async block, but to my knowledge, both the libraries of the market data provider and Cosmos DB are synchronous functions. Now the problem becomes:
- How to run an asynchronous block with synchronous function calls within Azure Functions
- How to run multiple copies of the synchronous functions, and I need to control the concurrency (in order not to flood the third-party API)
With the help of the Azure Functions documentation, and this Stack Overflow thread, I was able to wrap synchronous codes in an asynchronous function, and also control the concurrency of the application. And here is a sample that summarises what I have come up with.
This code intends to be run as __init__.py of an Azure Function. In the debug console of Visual Studio Code, we can check the verbose output of the above code. The variable max_concurrent_task was set to 3, so the function async_task() had waited for one of the three running tasks to finish before starting another.
This might not be a perfect solution, and I think it’s a little too convoluted to wrap a synchronous block in an asynchronous block in another asynchronous block. But as a newbie to Azure Functions and also asyncio, this is sufficient enough for me to continue prototyping my project.
