Big Data Analytics on Excel files using Hadoop/Hive/Flink/Spark

2018-03-17 --- Jörn Franke

Today we have released HadoopOffice v1.1.0 with major enhancements:

• Based on the latest Apache POI 3.17
• Apache Hive: Query Excel files and write tables to Excel files using the Hive Serde
• Apache Flink support for Flink Table API and Flink DataSource/DataSink
• Signing and verification of signatures of Excel files
• Example to use the HadoopOffice library for writing files using Spark 1.x
• Provided universal converter from Excel cell content to basic data types and vice versa
• Improved support to read the “header” line (=first row) of an Excel from the first sheet or all sheets. Added options to skip the first n rows of the first sheet or of all sheets.
• Secure storage of credentials using keystores
• Migrated to Junit5
• Improvements related to memory in low footprint mode

Of course the usual features of the HadoopOffice library are still supported, such as Spark2 datasource/datasink support, encryption, linked workbooks, templates, low footprint mode etc.

This activity is part on the delivery of the overall HadoopOffice vision.