JPA Batch Insert๋ฅผ ์จ๋ณด์ž! ๐Ÿ‘

Telepathyยท2023๋…„ 5์›” 9์ผ
0
post-thumbnail

Batch Insert

Kotlin + Spring Boot๋กœ ์ž์ฃผ ๊ฐœ๋ฐœํ•˜๊ฒŒ ๋˜๋ฉด์„œ JPA๋ฅผ ๋งŽ์ด ์‚ฌ์šฉํ•˜๊ฒŒ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค.
๋งŽ์€ ์žฅ์ ์„ ๊ฐ€์ง€๊ณ  ์žˆ๋Š” JPA์ด์ง€๋งŒ ๋‹น์—ฐํžˆ ๋‹จ์ ๋„ ์กด์žฌํ•ฉ๋‹ˆ๋‹ค. ๋‹จ์ ์ค‘์—์„œ๋„ ์ œ๊ฐ€ ์ œ์ผ ๊ทนํ˜ํ•˜๋Š” ๋‹จ์ ์€ ์“ธ๋ฐ์—†์ด ์ฟผ๋ฆฌ๋ฅผ ๋งŽ์ด ๋‚ ๋ฆฌ๋Š” ๋ฌธ์ œ์ž…๋‹ˆ๋‹ค.

๐Ÿข Query Test

JPA๋ฅผ ์ด์šฉํ•˜์—ฌ 1000๊ฐœ์˜ ๋ฐ์ดํ„ฐ๋ฅผ PostgreSQL์— insertํ•˜๋Š” ํ…Œ์ŠคํŠธ๋ฅผ ์ง„ํ–‰ํ•˜์˜€์Šต๋‹ˆ๋‹ค.
1000๊ฐœ์˜ ๋ฐ์ดํ„ฐ๋ฅผ insert ํ•ด๋ณธ ๊ฒฐ๊ณผ ์–ด๋งˆ์–ด๋งˆํ•˜๊ฒŒ ๋Š๋ฆฐ ์†๋„๊ฐ€ ๋‚˜์™”์Šต๋‹ˆ๋‹ค.

๋ฌด๋ ค 40์ดˆ... ๋ผ๋Š” ์‹œ๊ฐ„์ด ๊ฑธ๋ ธ์Šต๋‹ˆ๋‹ค.

์šฐ๋ฆฌ ๊ฐ™์ด ์ด ์†๋„๋ฅผ ๋‹จ์ถ• ์‹œ์ผœ๋ด…์‹œ๋‹ค.


โ˜ ์ฒซ๋ฒˆ์งธ ๋ฐฉ๋ฒ•

application.yaml ํŒŒ์ผ์—

spring:
  jpa:
    properties:
      jakarta:
        persistence:
          sharedCache:
            mode: UNSPECIFIED
      hibernate:
        jdbc:
          order_inserts: true
          batch_size: 1000
์œ„์ฒ˜๋Ÿผ `hibernate.order_inserts=true`์™€ `hibernate.batch_size=1000`์ด๋ผ๊ณ  ๋„ฃ์–ด์ฃผ๋ฉด 1000๊ฐœ์”ฉ ๋ฌถ์–ด์„œ insert๋ฅผ ์‹œ์ผœ์ค๋‹ˆ๋‹ค. ๋‹จ DB์— autoincrement id ์—†์–ด์•ผ batch insert๋ฅผ ์‹œ์ผœ์ค„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. id๊ฐ€ autoincrement๋ผ๋ฉด ๋‹ค์Œ id๊ฐ’์„ ์•Œ์•„๋‚ด๊ธฐ ์œ„ํ•ด JPA๋Š” insertํ•˜๊ธฐ ์ „์— ๋งˆ์ง€๋ง‰ ์•„์ด๋””๋ฅผ select ํ•˜๋Š” ์ฟผ๋ฆฌ๋ฅผ ๋‚ ๋ฆฌ๊ฒŒ ๋˜์–ด batch insert๋ฅผ ํ•  ์ˆ˜ ์—†๋Š” ์ƒํƒœ๊ฐ€ ๋ฉ๋‹ˆ๋‹ค.

์ž ์œ„์ฒ˜๋Ÿผ ์„ค์ •์„ ์™„๋ฃŒํ•˜๋ฉด ์–ผ๋งˆ๋‚˜ ๋‹จ์ถ•์ด ๋˜์—ˆ์„๊นŒ์š”? ์ง€๊ธˆ ๋ฐ”๋กœ ํ™•์ธํ•ด๋ณด์‹œ์ฃ .

20์ดˆ ์ •๋„ ๊ฑธ๋ฆฝ๋‹ˆ๋‹ค. ๊ธฐ์กด ๋Œ€๋น„ 20์ดˆ ์ •๋„ ๊ฐ์†Œ ๋˜์—ˆ์ง€๋งŒ ๋งŒ์กฑํ•˜์ง€ ๋ชปํ•˜๊ฒ ๋„ค์š”.


โœŒ ๋‘๋ฒˆ์งธ ๋ฐฉ๋ฒ•

์šฐ์„  20์ดˆ๊ฐ€ ์ •๋„๊ฐ€ ๊ฑธ๋ฆฌ๋Š” ์ด์œ ๋ฅผ ์•Œ์•„์•ผ ๋ฉ๋‹ˆ๋‹ค. JPA ๋‚ด์žฅํ•จ์ˆ˜์ธ saveAll์„ ํ•˜๊ฒŒ ๋˜๋ฉด select๋ฌธ์ด 1000๊ฐœ ๋ฐœ์ƒ์ด ๋˜๊ณ  ๊ทธ๋‹ค์Œ insert๋ฅผ ์‹œ์ผœ์ฃผ๋Š”๊ฑธ ํ™•์ธ ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์™œ ๊ทธ๋Ÿฐ์ง€ ์ด์œ ๋ฅผ ์•Œ๊ธฐ ์œ„ํ•ด JPA ๋‚ด์žฅํ•จ์ˆ˜์— ์žˆ๋Š” saveAll ํ•จ์ˆ˜๋ฅผ ํ™•์ธํ•ด๋ณด๋‹ˆ insertํ•˜๋Š” ๋ฐ์ดํ„ฐ๊ฐ€ ์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ์ธ์ง€ ํ™•์ธํ•˜๊ณ  ์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ๋ผ๋ฉด ๊ทธ๋ƒฅ insert๋ฅผ ์‹œ์ผœ์ฃผ๊ณ  ์•„๋‹ˆ๋ผ๋ฉด ํ™•์ธ ํ›„ ์—…๋ฐ์ดํŠธ ํ•ด์ฃผ๋Š” ๊ธฐ๋Šฅ์ด ํฌํ•จ๋˜์–ด ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋ž˜์„œ insert์™€ update๋ฅผ ๊ฐ™์€ saveํ•จ์ˆ˜๋กœ ์ฒ˜๋ฆฌ๊ฐ€ ๊ฐ€๋Šฅํ–ˆ๋‹ค๋Š” ์‚ฌ์‹ค์„ ์•Œ๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

์ด๊ฑธ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด์„œ๋Š” Entity ํŒŒ์ผ์„ ์ˆ˜์ •ํ•˜์—ฌ์•ผ ํ•ฉ๋‹ˆ๋‹ค.

@Data
@AllArgsConstructor
@NoArgsConstructor
@Entity
@DynamicUpdate
@Table(name = "test_table")
data class TestEntity(
    @Id
    @Column(name = "key")
    var key: UUID? = null,

    @Column(name = "first_name")
    var firstName: String? = null,

    @Column(name = "last_name")
    var lastName: String? = null,

    @Column(name = "email")
    var email: String? = null,

    @Column(name = "phone")
    var phone: String? = null,
)
์ด ์ฝ”๋“œ๋Š” ์šฐ๋ฆฌ๊ฐ€ ํ”ํžˆ ๋ณด๋˜ Entity ํŒŒ์ผ์ž…๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๋Š” ์—ฌ๊ธฐ์— insertํ•˜๋Š” ๋ฐ์ดํ„ฐ๊ฐ€ ํ•ญ์ƒ ์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ๋ผ๊ณ  ์ธ์‹์„ ์‹œ์ผœ์ฃผ๋Š” ์ฝ”๋“œ๋ฅผ ๋„ฃ์–ด์ฃผ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
@Data
@AllArgsConstructor
@NoArgsConstructor
@Entity
@DynamicUpdate
@Table(name = "test_table")
data class TestEntity(
    @Id
    @Column(name = "key")
    var key: UUID? = null,

    @Column(name = "first_name")
    var firstName: String? = null,

    @Column(name = "last_name")
    var lastName: String? = null,

    @Column(name = "email")
    var email: String? = null,

    @Column(name = "phone")
    var phone: String? = null,
): Persistable<UUID> {
    override fun isNew(): Boolean {
        return true
    }
    override fun getId(): UUID? {
        return this.id
    }
}

TestEntity ํŒŒ์ผ์— ์œ„ ์ฝ”๋“œ์ฒ˜๋Ÿผ ์ˆ˜์ •์„ ํ•˜์—ฌ ํ•ญ์ƒ ์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ๋ผ๊ณ  ์ธ์‹์„ ์‹œ์ผœ์ฃผ๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

๐Ÿšจ ๋‹จ ์ฃผ์˜ํ•˜์‹ค ์ ์€ ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด Save ๋˜๋Š” SaveAll ํ•จ์ˆ˜๋กœ ์—…๋ฐ์ดํŠธ๋ฅผ ํ•˜์ง€ ๋ชปํ•˜๊ธฐ ๋•Œ๋ฌธ์— ํ•ญ์ƒ ์ƒˆ๋กœ์šด ๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•˜๋Š” EntityํŒŒ์ผ์—๋งŒ ์ ์šฉํ•˜์—ฌ์•ผ ํ•ฉ๋‹ˆ๋‹ค.

์ž ๊ทธ๋Ÿผ ์–ผ๋งˆ๋‚˜ ๋‹จ์ถ•์ด ๋˜์—ˆ๋‚˜ ํ™•์ธํ•ด๋ด…์‹œ๋‹ค.

2๊ฐœ์˜ ๋ฐฉ๋ฒ•์„ ์ ์šฉํ•ด๋ณด๋‹ˆ 1.6์ดˆ ์ •๋„ ๊ฑธ๋ฆฌ๋Š”๊ฑธ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.๐Ÿ˜œ

profile
๐Ÿ˜ ์œ ์ €๋“ค์ด ์‚ฌ๋ž‘ํ•˜๋Š” ๊ฒŒ์ž„์„ ๋งŒ๋“œ๋Š” ๊ฐœ๋ฐœ์ž

0๊ฐœ์˜ ๋Œ“๊ธ€