spark warehouse default location

/spark3/conf/hive-site.xml에서

<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://spark-worker-01:3306/metastore_db2?createDatabaseIfNotExist=true</value>
        <description>metadata is stored in a MySQL server</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>MySQL JDBC driver class</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
        <description>user name for connecting to mysql server </description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>root</value>
        <description>password for connecting to mysql server </description>
    </property>
    <property>
        <name>hive.metastore.warehouse.dir</name>
        <!--<name>spark.sql.warehouse.dir</name>--> <!-- hive-site.xml용 아님.... spark conf 용임.... -->
        <value>hdfs://spark-master-01:9000/skybluelee/skybluelee_warehouse_mysql_5.7</value>
        <description>location of default database for the warehouse</description>
    </property>
    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>
    <property>
        <name>datanucleus.autoCreateTables</name>
        <value>true</value>
    </property>
</configuration>

에서 value값을 수정해도 spark.sql.warehouse.dir 의 값만 수정되고
실제 수행되는 위치가 바뀌지 않는 문제 발생

문제 해결

첫 property에서 metasotre_db가 createDatabaseIfNotExist=true 이었기 때문에 첫 실행 이후 위치 변경 값이 metasore_db에 영향을 주지 못함
나는 metasotre_db2를 새로 생성했는데 metasotre_db를 삭제하고 다시 만들어도 정상적으로 작동할거라 생각.

default 바꾸기 실패 option

spark
    .sql("""
        SELECT 
            website, 
            avg(age) avg_age, 
            max(id) max_id
        FROM 
            people a 
            JOIN 
            projects b 
            ON a.name = b.manager 
        WHERE 
            a.age > 11 
        GROUP BY 
            b.website
        """)
    .write
    .option("path", "hdfs://spark-master-01:9000/skybluelee/skybluelee_warehouse_mysql_5.7")
    .mode("overwrite")  //--Overwrite mode....
    .saveAsTable("JoinedPeople")  //--saveAsTable(<warehouse_table_name>)....

option으로 경로 변경 가능