Cleaning AWS S3 multipart leftovers

Many tools and frameworks perform uploading files to S3 in parts to gain performance or overcome 5GB limitation. If the process is terminated and not completed it may still occupy storage space.

CLI

You can list all multipart uploads in bash for each bucket:

aws s3api list-multipart-uploads --bucket=
aws s3api abort-multipart-upload --bucket= --key= --upload-id=

Automation

AWS already provides solution for it which may be just enabled. I was always told that S3 Lifecycle Rules are for moving objects between storage classes. It turns out you can do much more with it. Let’s enable incomplete multipart upload deletion.

Outcome

If everything went right you will see in the Console newly added and enabled rule.

Software Developer, Data Engineer with solid knowledge of Business Intelligence. Passionate about programming.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store