SQLdep

Read a case study

SQLdep saved GE Money twelve months of work

Download

Partnership with Collibra

As a SQLdep founder, I am happy to announce our partnership with Collibra. Collibra is a leading Data Governance platform for Master Data Management. With the total funding amounting to $134M and their revenue doubling every year, Collibra is on the way to becoming the #1 solution for metadata management.

21 June, 2018 17:04:48 PM

Conditional lineage based on full-text search

A full-text search is now available for lineage and all underlying SQL queries. It  is a fantastic feature for narrowing down lineage complexity and showing only the relevant parts.

20 April, 2018 8:25:33 AM

IBM Netezza among supported SQL dialects

Our dev team implemented yet another database dialect. SQLdep now supports IBM Netezza which is used for Data Warehouse appliances and advanced analytics.

For early birds we offer a free unlimited tier till end of June 2018. You are welcome to upload your Netezza SQL scripts and we will take care of the rest. All metadata can be downloaded in CSV format as well. Get in touch and enjoy the free tier!

15 January, 2018 12:58:49 PM

Snowflake SQL dialect added. Try it.

We are happy to announce a support for Snowflake SQL dialect. All SQLdep features like automated documentation of ETL processes and capturing end-to-end data lineage are now available for Snowflake SQL dialect. 

Snowflake itself is a highly optimised database for Data Warehousing and it runs in cloud. The company was founded in 2012 and by early 2017 they have completed D round and raised $100M.

22 September, 2017 10:04:22 AM

Server migration

On Thursday 27th of April 2017 we have a scheduled downtime. The migration to a new server will start at 10:00am GMT and will be finished within 60 minutes. During the migration we will also update DNS records (domain name remains the same).

We are switching to a more powerful server and also changing internal infrastructure. Hypervisors will be installed to enable clustering and to obtain a fully redundant server platform. We do appologize for the inconvenience.

25 April, 2017 17:15:11 PM

Business glossary add-on

Our business glossary add-on helps the stakeholders in the company to actively manage and describe the data. The key feature is semi-automated functionality when most of the glossary is pre-populated from the existing sources. Thus manual input needed by data stewards is greatly reduced.

7 April, 2017 16:39:13 PM

SQL code checker: review and validation

Whenever you use SQL on daily basis you should make sure that everybody on the team uses the best SQL coding practises. Couple of well placed rules goes a great length when you need to analyse SQL code written by somebody else.

I have never understood how somebody can omit using a fully qualified column name. It just takes 2 seconds to place table alias in front of the column name. 

28 November, 2016 16:40:34 PM

Success story: tracking data lineage in Makro Cash & Carry company

The BI team in Makro faced a challenge with keeping track of the data lineage on their Teradata DWH. Just on the Czech market the company's data grew to 1.5TB in size and the data spread over 10,000 tables and views. It proved more and more difficult to perform impact analysis over multiple data layers, i.e. from the staging layer up to the reporting layer. The senior developer had to manually trace the lineage across the ETL job.

13 October, 2016 14:04:48 PM

SQL code parsing

A slight technical insight about how we reverse engineer the SQL code. It is to give you an overview of what is behind SQL code parsing and analysis. Those two parts are often called syntactic and semantic analysis.

15 August, 2016 10:13:15 AM

How do you like QueryScope dashboard?

We are moving forward to the launch of SQLdep QueryScope. In one week we are launching the private beta! Here is the first design of the user interface. How do you like it? We are looking forward to reading your comments or seeing you among the beta testers (sign-up here). 

9 May, 2016 13:53:02 PM

Data lineage based on SQL

Data lineage is essential for understanding data flows in your BI ecosystem (ETL jobs, SQL based reporting, ). A proper data lineage helps your developers to quickly identify impact of the change in SQL while keeping the audit trail behind. Finally BI end users are bound to have an access to data lineage as well so they can successfully leverage the value of data itself.

27 April, 2016 14:03:19 PM

Complex queries with an ease. Big shift in SQL world.

There is a big productivity shift about to happen in the SQL world. Till now, data analysts and developers were performing SQL code analysis manually and such an approach was a widely accepted standard. This performance killer ranges from 10 minutes on a moderately complex query till 2+ hours with a couple of complex queries.

31 March, 2016 13:57:16 PM

Catch me, if you can - Something you don’t want to hear from your know-how

10 years ago business intelligence became an inseparable part of every company. Every company heard that to achieve great success in the next years, they have to start using the data. So they created new BI teams, built new DWH architecture, tried to find the balance between servers and cloud, migrated to SaaS, and so on.

9 March, 2016 11:09:46 AM

“We didn’t go streaking” - an interview with SQLdep founders, Masi and Madhouse

It’s early in March 2014, and I’m in Prague, Czech Republic - one of the global capitals for delicious beer, beautiful architecture, and a growing technology community. I’m speaking with Martin “Masi” Masarik, and Miroslav “Madhouse” Semora about SQL development, data analysis, and their views on the SQL space, including their reasoning for devoting the last year of their lives to building SQLdep, a visualization and analysis tool for SQL developers and business intelligence analysts.

3 March, 2016 11:14:50 AM

Text based metadata? Have beautiful visualisation with zero effort.

Plenty of business intelligence teams maintain text based metadata. To write something is easy to start with and you gain results quickly. Speed bumps occurs with increasing metadata complexity and thus lineage becoming really intertwined. It is like if someone gives you a textual description how to drive from New York to Philli. Wouldn’t you be like -- “Hey, buddy, gimme a map instead!”?

18 February, 2016 10:56:11 AM

3 Hours a Day Your Developer is not Developing

Deadlines often are not met by developer teams regardless of what they are working on. Software development always takes longer than expected, mainly for three reasons. Scheduling is insufficient as it is done by managers, who often have little or no developer experience. Additionally, the team does not always have all the tools they need and of course, development always runs into problems that are not always accounted for. When management scopes out the timeline for a project, they often forget to consider time related to development where actual development does not take place.

16 February, 2016 11:04:47 AM

Stored procedures on the loose? Bring order to the chaos

Who has a blank stare even after 2 months in his new position? The answer is: a junior developer you hired and showed your ETL. I am talking  ETL with hundreds of intertwined procedures. To be fair, a senior guy would have to chew on it too but they’re less likely to choke.

4 February, 2016 10:51:23 AM

3 reasons why data documentation matters

The core of good business intelligence is a data warehouse, where information from various sources can be collected and organised in an effective manner to provide great results. Using a wealth of data helps provide analytical BI which not only allows you to react to what has happened, but predict what’s going to happen.

19 January, 2016 11:01:23 AM

Oh boy, am I asked again to compare two SQL based reports?

Do you know what’s difficult and prone to errors? Interpreting a complex SQL query that someone else wrote a year ago. Do you know what makes this scenario even worse? Having two different versions of such a SQL query, and it being up to you to figure out the difference.

14 January, 2016 10:58:56 AM

SQLdep presents Lionfish - the king of SQL code ocean

I love the SaaS services where you can immediately try the tool on a real life use case. Since SQLdep is focused on SQL code parsing a challenge was finding a really simple way to get SQL code to our REST API. The goal was that nobody should spend more than 5 minutes to submit their complete SQL codebase to us.

6 January, 2016 10:46:18 AM

Teradata Aster Database will move you to the big data world without impeding how your current team works

Alright, the buzzword again — big data. However this time from the perspective of a relational database. In particular, Teradata had a sweet idea to bring big data to relational databases. In 2011 they acquired Aster Data Systems enhancing their product portfolio with Teradata Aster Database.

27 December, 2015 11:06:59 AM

Data lineage on multiple databases? No problem.

Having your metadata under control and thus providing data lineage is a toughie. Not to mention another level of complexity when data travel across multiple databases. Especially through separate instances, thus when querying data through the database links are applied.

7 December, 2015 10:32:30 AM

SQLdep presents Wayback Machine for your DWH

Today, at SQLdep we proudly release a major feature that’s a must-have for any Business Intelligence ecosystem. To explain this feature in all its glory let’s talk about the Wayback Machine.The Wayback Machine is a service that keeps track of any page on the Internet over time. About a year ago the size of this archive was 15 petabytes.

25 November, 2015 11:03:20 AM

The dark side of generated code. Is there a new hope?

Stopping by the booths at Strata-Hadoop, I was struck by how many tools out there generate code. The idea is that non-technical users can create data flows with visual tools, and the nasty complexity of code is hidden from them. In theory, this sounds great. You save time, and, in the short-term at least, development resources.

28 October, 2015 10:43:14 AM

My current mission is to guide developers out of the stone age.

I was one of those developers in a large corporation who are running around undocumented code like headless chickens. I routinely analyzed thousands of lines of SQL code and a month later would have to do the ​*exact*​ same part from scratch. We were a team of 30 developers and every single day each of us spent 2 hours on SQL code analysis. Hideous routine for us and a really expensive task for the company.

24 September, 2015 10:38:29 AM

SQLdep and a Gartner Magic Quadrant

Gartner recently released their 2014 Magic Quadrant for Data Warehouse Database Management Systems, and perhaps not surprisingly, it describes both the rising influence of big data and cloud, as well as the investment by traditional vendors into their data warehouse offerings.

10 September, 2015 10:53:17 AM

Determining data lineage: SQL scripts vs documentation

So, here’s the issue: You’re a data analyst in a large company, working with a datawarehouse and you are part of the team responsible for reporting. You have a fairly complex set of SQL queries that set-up regular reporting, compute various metrics like profitability, prepare numbers according to account regulations like US GAAP, and much much more.

10 September, 2015 10:35:15 AM