SQLdep joins Collibra to build the future of Data Lineage Learn more

Lineage Discovery and Migration

If you don't know where you are going, you might not get there.

“Many organizations are evaluating a shift from traditional data warehousing solutions to modern Big Data, often Hadoop-based, platforms. Yet vast bodies of existing code must be analyzed to support a strategic plan to embrace new technologies.”

To stay competitive, organizations are migrating to big data BI solutions that deliver business insights derived from vast quantities of data.

But, there are very few organizations who are starting with clean slate. Large organizations, with multiple, distributed divisions and development teams must wrestle with immense bodies of existing data. But along with the data there is code that performs transformation or data mediation between a data sources and a destinations. Often the goal is consolidation of multiple databases into a single database, with the goal of identifying redundant columns of data for consolidation or elimination. .

The key to a successful migration is the identification of data relationships as part of data lineage analysis across many sources. A significant impediment is the fact manually-maintained metadata managers are not able to account for metadata drift. To counter these limitations, SQLdep provides an instantaneous map of data relationship and transformations, relying directly on the code as a ‘source of truth’ for the data lineage. Armed with this easily comprehensible knowledge, data architects and BI managers are able to effectively plan and scope the migration from the data analysis of today to the insights of tomorrow.

SQLdep features

  • Data-lineage By reverse-engineering SQL code we show interactive data lineage/interactive data flows which otherwise have to be done manually by the developer.
  • Highlighting code When a developer is editing an SQL query we highlight relevant lines in the SQL code he might be interested in. This makes editing faster, but also helps to prevent mistakes by avoiding any edits being forgotten.
  • Meta data export Manually or programmatically export a 'single source of truth' for use as up-to-date documentation or to reconcile third-party metadata management repositories.

If you have any comments or queries, feel free to contact us.

Contact Us