Calculate uniqueness inter and intra cluster records to measure the correctness of DME output
As with the Spark v 2.3.0 or later version, has a API which can calculate measure of Clustering Prediction Score - It will nice to integrate this in our DME plugin as a confidence scoring model out of box.More details about the Spark API :https://...
FDQ Reporting should have more feature to generated selective data
We have observed that when we run FDQ for multiple rows with multiple rules, the report generated is huge.We have been reported by a client that for a file of size about 1.5GB and 2.5 crore records, the fdq report generated was of about 350GB whic...
Hi,In Arena we can input Good, bad and report entity data path. If a user clicks Entity name text box after entering a custom value in entity path text box then the value in entity path text box is reset to /target schema/entity nameThis behavior ...
Enable Data Quality on Hive Views from DQ of underlying Tables
Enable data quality on views in Zaloni via linking with table instead of re-calculating DQFrom customer: Views should not have their own profiling/DQ but profile/DQ information should come from the original table profiled from where these columns ...
Clients Scenario:I have an entity with custom data format.This entity is associated with an EDQ action of a post ingestion WF. While executing the action, I got the exception "Entity with data file CUSTOM is not supported by EDQ"
Ability to compare data in trusted zone with source of truth data
As a data steward or member of governance team, I want the ability to compare data stored in trusted zone with golden record/source of truth data stored in source platform, so that I can ensure data quality and completeness.
almost 2 years ago
in Data Quality
On the Backlog