| 일 | 월 | 화 | 수 | 목 | 금 | 토 |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | ||
| 6 | 7 | 8 | 9 | 10 | 11 | 12 |
| 13 | 14 | 15 | 16 | 17 | 18 | 19 |
| 20 | 21 | 22 | 23 | 24 | 25 | 26 |
| 27 | 28 | 29 | 30 |
- 코엑스맛집
- 코딩테스트
- BFS
- 개발
- apache iceberg
- 자바
- HIVE
- hadoop
- 양평
- 영어
- bigdata engineering
- 여행
- java
- Iceberg
- 코테
- Data Engineer
- 프로그래머스
- 백준
- dfs
- 파이썬
- BigData
- 코딩
- 맛집
- 코엑스
- 삼성역맛집
- 알고리즘
- bigdata engineer
- Trino
- 용인맛집
- Data Engineering
- Today
- Total
목록지구정복과정 (492)
지구정복
[Hive] Error Table is marked as a managed table but is not transactional
Spark3로 managed table create하는데 아래와 같은 에러 발생 spark-sql> CREATE TABLE spark_catalog.db1_test.example_hive1 ( > id INT, > name STRING, > age INT > ) > STORED AS parquet; ErrorCaused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Table db1_test.example_hive1 failed strict managed table checks due to the following reas..
airflow db upgrade error발생하면 airflow db upgrade 진행 만약 위 명령어 에러나면 아래 airflow python package 설치해준다./usr/airflow/.pyenv/versions/venv/bin/pip install sqlalchemy==1.4.49/usr/airflow/.pyenv/versions/venv/bin/pip install apache-airflow-providers-common-sql 그리고 다시 airflow db upgrade 진행
[Iceberg] Iceberg Guide Book Summary | CHAPTER 6. Apache Spark
CHAPTER 6 Apache SparkConfigurationConfiguring Apache Iceberg and SparkConfiguring via the CLIAs a first step, you’ll need to specify the required packages to be installed and used with the Spark session. To do so, Spark provides the --packages option, which allows Spark to easily download the specified Maven-based packages and its dependencies to add them to the classpath of your application. ..
[Iceberg] Iceberg Guide Book Summary | CHAPTER 5. Iceberg Catalogs
CHAPTER 5 Iceberg Catalogs Requirements of an Iceberg CatalogIceberg provides a catalog interface that requires the implementation of a set of functions, primarily ones to list existing tables, create tables, drop tables, check whether a table exists, and rename tables. Hive Metastore, AWS Glue, and a filesystem catalog (Hadoop). with a filesystem as the catalog, there’s a file called version-hi..