Cleaning AWS S3 multipart leftovers
Many tools and frameworks perform uploading files to S3 in parts to gain performance or overcome 5GB limitation. If the process is terminated and not completed it may still occupy storage space.
You can list all multipart uploads in bash for each bucket:
aws s3api list-multipart-uploads --bucket=
You can even remove them manually by calling:
aws s3api abort-multipart-upload --bucket= --key= --upload-id=
AWS already provides solution for it which may be just enabled. I was always told that S3 Lifecycle Rules are for moving objects between storage classes. It turns out you can do much more with it. Let’s enable incomplete multipart upload deletion.
If everything went right you will see in the Console newly added and enabled rule.