Cleaning AWS S3 multipart leftovers

Many tools and frameworks perform uploading files to S3 in parts to gain performance or overcome 5GB limitation. If the process is terminated and not completed it may still occupy storage space.

CLI

You can list all multipart uploads in bash for each bucket:

aws s3api list-multipart-uploads --bucket=

You can even remove them manually by calling:

aws s3api abort-multipart-upload --bucket= --key= --upload-id=

Automation

AWS already provides solution for it which may be just enabled. I was always told that S3 Lifecycle Rules are for moving objects between storage classes. It turns out you can do much more with it. Let’s enable incomplete multipart upload deletion.

Outcome

If everything went right you will see in the Console newly added and enabled rule.