I want to process/download .gz files from S3 bucket. There are more than 10,000 files on S3 so I am using
import boto3 s3 = boto3.resource('s3') bucket = s3.Bucket('my-bucket') objects = bucket.objects.all() for object in objects: print(object.key)
This lists .txt files which I want to avoid. How can I do that?
The easiest way to filter objects by name or suffix is to do it within Python, such as using
.endswith() to include/exclude objects.
Filter by Prefix, but not by suffix.