[Python] boto3 multithreading

오도원공육사·2022년 12월 19일
0

파이썬

목록 보기
11/11
import concurrent.futures
import boto3
import json


files = ["path-to-file.json", "path-to-file2.json"] 

def download_from_s3(file_path):
    # setup a new session
    sess = boto3.session.Session()
    client = sess.client("s3")
    
    # download a file
    obj = client.get_object(Bucket="<your-bucket>", Key=file_path)
    resp = json.loads(obj["Body"].read())
    return resp

with concurrent.futures.ThreadPoolExecutor() as executor:
     executor.map(download_from_s3, files)

출처
https://stackoverflow.com/questions/52820971/is-boto3-client-thread-safe

profile
잘 먹고 잘살기

0개의 댓글