Debugging python programs running in docker
Work with docker, airflow and whatever hard to debug environments with ease.
This approach works with remote applications as well
You’ll probably come across dockerised applications at almost every codebase you work on, atleast professionally. Unfortunately they are a massive pain when it comes to debugging. At work we recently switched to an airflow project running in docker. Suffice to say, debugging that has been nearly impossible and I’ve only observed print debugging which has been a major step down.
I figured pdb (pythons built in debugger) must have offered some way to debug programs remotely. Well, it doesnt, but it turns out we have another great tool that does.
In comes epdb (extended python debugger)
Usage
The usage is simple:
install epdb via pip (make sure to install the correct one).
add a remote “breakpoint” in code that serves on a specified port.
def func_to_debug():
... code ...
import epdb
epdb.serve(port=6000)
# debugger running on port 6000
... code ...connect to that port via your cli.
python -c "import epdb; epdb.connect(host='app_ip', port=6000)"Here, app_ip will be the ip of your service. In case of a docker container, you can get this by running docker inspect ${container_id}
debug as if it was pdb.
Considerations
For a remote server you are trying to debug, provided you have setup a firewall (as one should) you’ll probably have to ssh into the server first.
For projects like airflow, you will have trouble debugging regardless, and would have to debug each step/operator independantly. I suspect this is due to airflows execution model. Still, it’s better than nothing.