Homer 11 parquet files to hdfs

[recourse]

Homer 11 looks amazing can't wait to use it. Would it be possible for Homer to read and write it's parquet files to an existing Hadoop hdfs system?

[lmangani]

Hey Joseph!! Not quite. The backend can store parquet files locally and on object storage layers (progressively) but our DuckDB powered writer/query core uses our metadata store to resolve the parquet files and has a complex but elegant predicate pushdown mechanism to only scan relevant chunks from the resolved parquet files required by any given query. So we can use remote S3-compatible storage and the parquet files can be read with anything compatible with the format but at least at the beginning that's it - but as always this opensource so if there's sufficient demand or some brave developers anything is possible. Hope you're having a great time at Cluecon!