The word “silo” gets bandied about far too much in the business world. Originally from the Greek siros or “corn pit”, anyone who has ever worked on a farm will know that a silo is a very useful thing indeed for keeping harvests dry, safe, stored appropriately and away from uninvited pests.
“Functional silo syndrome”, the term coined by Phil Ensor in the late 1980s to describe organisations structured with a “deeply layered vertical hierarchy” and “narrow, boring, highly specialized jobs designed to be easy to supervise (control)” continues to have some value however.
Data, of course, is not corn, any more than it is oil or any of the other commodities metaphor-lovers like to attach to it; but it is often held in isolated repositories with restricted access and limited visibility.
Sometimes, like literal silos, this is with good reason. Sometimes, like their metaphorical counterpart and per Ensor's formulation, it is problematic.
Getting value out of a data transformation does not have to involve pulling this asset into a single warehouse, however. It does typically require a unique blend of technology, cultural and process change and “narrow, boring, highly specialized’ jobs can be a blocker – sometimes intentionally.
Sitting down with The Stack at Snowflake’s Summit in Las Vegas, Satyen Sangani, CEO of data intelligence platform provider Alation notes that data and digital transformation programmes can create real people challenges.
Alation CEO Satyen Sangani: "Data represents power"
“Data represents knowledge and in a great sense represents power. If you are making data available to more people that was previously unavailable then you are distributing power. You are also relegating power to other individuals that may not have had it. That can be really scary for people who have made their jobs off doing one or several things particularly well… changing organisational human dynamics is tough,” he says.
Sangani, who has been running Alation since its founding in 2012 and who has grown it to over $100 million in ARR, is an economist by training who previously spend nearly a decade leading on analytical applications at Oracle. He has seen data transformation projects both fail and blossom and for a technology CEO, sounds (to your scribe) pleasantly world-weary, with very little “buy-this-shiny-widget-as-a-data-panacea” schtick.
Alation is mature at this point – with customers that include Cisco, Riot Games, Pfizer, Sainsbury’s among other bluechips – and asked about themes he has seen emerging across both technology (including AI) and drivers organisational success when it comes to data transformation, Sangani says he has seen the data world evolve a lot over the last decade.
“When we founded the company, Teradata was the biggest data warehouse out there, everybody was talking about 'Big Data', and Hadoop was all the rage. Then there was a moment where there were things like Netezza, and Vertica… then of course you move to the era of Databricks and Snowflake. This more recent theme of AI has brought in a lot of imagination around unstructured data, and in particular text, but also video and the like.
"I think that's going to fuel another level of growth."
What does Alation do?
So where does his company come in?
Alation’s role in helping enterprises gain value out of – and control over – their data is superficially straightforward but underpinned by significant technological nous. As a provider of data cataloguing software it aims to help customers “find, understand and trust data”: users can deploy it to gain control and stewardship over sprawling data estates and yes, "silos".
At the front-end is a useful user interface. At the back-end are over 100 connectors for a wide variety of data sources including BI tools, databases, and file systems; to pick 10 at random: AWS S3, Cassandra, Elasticsearch, Github, IBM DB2, MongoDB, SQL Server, SAP HANA, Teradata.
(“There are range of different mechanisms that you can use to talk to a database” Sangani says. “The most common one is to fire a SQL query directly into the database. But there are other interfaces, JDBC, ODBC...”)
Alation users connect to the database in question and use it to start issuing “a standard set of sort of requests into that database, like, ‘what are your tables?’ ‘What are the column names?’ ‘How many people have used this particular thing?’ ‘Who are the users on this server?’ ‘What are the queries that people have historically written?’ Sangani explains to The Stack.
Implementing that in Oracle, versus SQL Server, versus Snowflake is extraordinarily different and difficult. Getting further information about how to parse those queries and what SQL looks like in the context of Hive, versus SQL in the context of Databricks, or Snowflake… that can get extraordinarily complicated. The basic idea is we manage that complexity, and make the database look like something that's human-consumable.”
The company has baked a range of tools on top of this that let “people closest to the data can add rich context, describing what data really means and how it should be used... Machines can automate and scale tedious tasks, like translating technical to natural language, ranking data popularity, or identifying experts for guidance or data stewardship.”
It’s a platform and approach that won the company a leadership position in the Forrester Wave data governance solutions report – in which Alation sits alongside 11 other key peers in the space like Aim, Ataccama, Collibra, Congruity360, data.world, erwin, Infogix, OneTrust, SAP, Solix, and Syniti.
Questioned on the extent to which AI will disrupt this market (including the extent to which it might automate away the need for such tools and how Alation is deploying it) Sangani tells The Stack that "I think it's exciting.
"We're already doing things like automating documentation, taking SQL queries, and allowing people as they're authoring those queries to get suggestions so they can construct those queries without having to know the database environment very well... you will see more announcements from us, but we've chosen to take high trust, low hype strategy."
"The standard deviation is still quite broad..."
Returning to that theme of data transformation challenges and how he has seen those evolve on the customer side, he says: “I think that the challenge, of course, is that if you invest in accounting you know exactly what you're gonna get out of it. You invest in data, and it is often inherently speculative, you don't know if you're going to find insights...
“Even though on average, people are using data much more, there's been a lot of fits and starts, there's been a lot of failed expectations. The mean might be increasing, but the standard deviation is still quite broad, relative to investing in projects. Some of that's inherent to what you're trying to do, which is discover insights; knowledge generation is not a deterministic process. But some of that is also because people don't know how to do it.
"There's a lot more maturity now. Overall, it's a really exciting time. But I think a lot of people would like it to be a straighter line than it is."
On that sober note, customer meetings call. Are they going to get hype? It seems unlikely. Are they going to get quietly realistic insight? That seems more probable. Alation's CEO doesn't seem to be a big one for cliches and mercifully, silo doesn't seem to be a big part of his vocabulary either.