Stack Overflow
Stack Overflow

Questions tagged [hive-table]

Homogeneous units of data which have the same schema in Hive

Apache Hive has two different types of tables namely Internal and External Tables.

18 questions
3
votes
1 answer

changing hive external table to internal table in the same database also drops the data from the another table

I have been trying to drop a hive external table using the statement alter table $tableName set tblproperties('EXTERNAL' = 'FALSE'); after I pass these properties and do a DROP TABLE TABLENAME; this command also drops the data from the…
Aditya Verma
  • 201
  • 4
  • 14
3
votes
4 answers

Spark DataFrame ORC Hive table reading issue

I am trying to read a Hive table in Spark. Below is the Hive Table format: # Storage Information SerDe Library: org.apache.hadoop.hive.ql.io.orc.OrcSerde InputFormat: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat OutputFormat: …
Subhasis
  • 65
  • 1
  • 7
2
votes
1 answer

Hive External table on AVRO file producing only NULL data for all columns

I am trying to create an Hive external table on top of some avro files which are generated using spark-scala. I am using CDH 5.16 which has hive 1.1, spark 1.6. I created hive external table, which ran successfully. But when i query the data i am…
Vaishak
  • 607
  • 3
  • 8
  • 30
2
votes
1 answer

Can we check the hive deleted table history or detailed information around it?

I have came across one scenario where someone dropped/deleted hive table from production environment by mistake. How can I track the history and get the details i.e.(who has deleted and when it got deleted). Any Suggestions?
code
  • 123
  • 3
  • 11
1
vote
1 answer

How to read a table which is saved by saveAsTable in Apache Spark?

I am trying to save a dataframe by using df.write.saveAsTable("table_name") but i am unable to retrieve this table again (not in the same run). I tried spark.read.table("table_name") and spark.sql("select * from table_name") but both didn't worked…
Mohit Kumar
  • 552
  • 9
  • 29
1
vote
0 answers

ALTER TABLE table ADD IF NOT EXISTS PARTITION (state = '34' , city = '123') is not adding the partition in temp/local folder

I am working on hive table partitioning and using spark client to trigger the request. I have created the table and inserted data with partition.When I execute select statement I can see the data, but when I add Partition spark.sql("ALTER TABLE…
madhusri mannuru
  • 41
  • 5
1
vote
1 answer

Hive External Table - Drop Partition

Facing a weird issue. Alter table command to drop partition works well for > or < or >= or <= signs but not for = check. Working command: ALTER TABLE XYZ DROP PARTITION(bizdate>'20231230'); Command that's not working and throwing an error stating…
Dasarathy D R
  • 335
  • 2
  • 7
  • 20
1
vote
2 answers

hive - how to automatically append data to hive table every day?

I have a directory in HDFS, where .csv files with fixed structure and column names will be dumped at the end of every day that may look like this: I have a hive table that should have new data appended to it, at the beginning of every day, with…
Naveen Reddy Marthala
  • 2,622
  • 4
  • 35
  • 67
1
vote
3 answers

Create Sqoop Hive Import Job

I successfully did an ad hoc import from an oracle table into Hive table using Sqoop. Now I want to create and save a Sqoop job with the exact parameters I used for the ad hoc import. But if I say: sqoop job --create myjob --import \ --connect…
tamara d
  • 320
  • 5
  • 18
1
vote
2 answers

Cannot Create table with spark SQL : Hive support is required to CREATE Hive TABLE (AS SELECT);

I'm trying to create a table in spark (scala) and then insert values from two existing dataframes but I got this exeption: Exception in thread "main" org.apache.spark.sql.AnalysisException: Hive support is required to CREATE Hive TABLE (AS…
NTH
  • 101
  • 2
  • 8
0
votes
1 answer

Can i filter the files(filenames) from which i wanted to create a hive table in databricks?

I have serverlogs enabled on s3 bucket. The log files have names as: 2023-02-16-00-16-16-A4210A3BBB675006. The first part of the filename is the date. And i extract various fields from the contents of the file using regex serde and create a hive…
Mourya Reddy
  • 1
  • 1
0
votes
1 answer

Hbase Tables not created in EMR cluster using Hive-Hbase Integration

I am new to AWS EMR and have created a Hive-Hbase table using the following code: CREATE EXTERNAL TABLE IF NOT EXISTS airflow.card_transactions(card_id bigint,member_id bigint,amount float,postcode int,pos_id bigint,transaction_dt timestamp,status…
hns_03
  • 1
  • 1
0
votes
0 answers

Add retention period to hive tables

Could you please let me know how to add retention period to Hive tables. In the below URL I could see partition discovery and retention is not recommended for use on managed tables. I don't understand why it is not recommended. I have created a…
Vijju
  • 37
  • 6
0
votes
2 answers

Hive - create an internal table from three external tables

I have three external tables in HIVE: table 1: CREATE EXTERNAL TABLE IF NOT EXISTS table_1( unique_key_column_1 VARCHAR, column_needed_1 DATE, redundant_column_1 VARCHAR, redundant_column_2 VARCHAR, redundant_column_3 VARCHAR, column_needed_2…
Naveen Reddy Marthala
  • 2,622
  • 4
  • 35
  • 67
0
votes
1 answer

Hive - create hive table from specific data of three csv files in hdfs

I have three .csv files, each in different hdfs directory. I now want to make a Hive internal table with data from those three files. I want four columns from first file, three columns from second file and two columns from third file. first file…
Naveen Reddy Marthala
  • 2,622
  • 4
  • 35
  • 67
1
2