The code may behave differently than developer expects. I mean not the correctness, but the performance. The performance bottleneck may be easy to found by running the profiler. cProfile is the way to do it in Python.


I am going to run my binary pipeline code twice using the profiler. First time in my local computer where the main bottleneck is my internet connection. The second time, on AWS ECS node, where the bottleneck should be CPU.


I ran locally the script as I used to do, but wrapped it around the cProfiler:

pipenv run python -m cProfile -s tottime
local run before any changes
ECS run before any changes
local run after JSON changes
ECS run after JSON changes


It is worth to invest a bit time to understand your bottlenecks instead of increasing the service tier. I am pretty sure the code might be even performing much better.

Software Developer, Data Engineer with solid knowledge of Business Intelligence. Passionate about programming.