Zaloni Ideas
Status On the Backlog
Categories Profiling
Created by Sanjay Yadav
Created on Aug 19, 2019

Enable data profiling on Hive Views from profile of underlying Tables

Enable data profiling on views in Zaloni via linking with table instead of physically creating data profiling

From customer:

Views should not have their own profiling but profile information should come from the original table profiled from where these columns are being referenced in the view.
Customer Impact Major inconvenience
  • Attach files
  • Admin
    Matthew Monahan
    Aug 20, 2019

    We should take a look at this based on how the View is created:

    1. Effectively, no changes in the data (could be subset of columns or a joining of columns from more than one table)

    2. Some columns may be new and need to be profiled, but others remain the same

    3. View may be a subset by row or some other scenario where the data may produce a different profile - e.g. this type would contain a "where" clause

    Consider how profile date is determined at the View level - possibly track profile date at the field level