python-read-gzip-s3

要用 python 讀取儲存在 S3 中的 gzip 檔案資料
使用的過程中，boto3 有用 s3_client 也有用 s3 resource 取得 S3 object 的方式，需要找時間再研究這兩者的差異

code sample

def load_gzip(client, bucket, key): response = client.get_object(Bucket=bucket, Key=key) content = response['Body'].read() with gzip.GzipFile(fileobj=io.BytesIO(content), mode='rb') as fh: file_content = fh.read() return file_content.decode("utf-8")

讀多個 S3 檔案的方式

• python - How to read multiple files in a directory, all of which are csv.gzip with Airflow S3 Hook or boto3? - Stack Overflow

Reference

• amazon web services - Reading contents of a gzip file from a AWS S3 in Python - Stack Overflow
• To use gzip file between python application and S3 directly for Python3
• How to store and retrieve gzip-compressed objects in AWS S3
• Reading a Specific File from an S3 bucket Using Python – SQLServerCentral