Introducing Atlan
The short of it⌠weâre building a home for data teams. And everyoneâs invited! đ
And now, hereâs the long of it⌠Like most good things in life, Atlan too started with a thought. An observation. A realization.
The way teams work with data is broken. Itâs outdated and unfair.
We know this through countless hours of research, the time weâve spent talking to data teams of all shapes and sizes, and our own experience of working with data. And while people may think that data jobs are among the most âsexiest jobs of the centuryâ, the thing isâŚ
Data teams today are overwhelmed and under enormous pressure
Just ask any data analyst, data scientist or chief data officer, and theyâll tell you about all the business expectations riding on them. All the sleepless nights theyâve spent trying to make disparate data sets talk to each other, solve for missing values in columns, solve for getting access to data, or answer the question that all business managers askâhow can your data add value to my business?
The truth is that while companies are investing significantly in big data and in setting up data teams, they are struggling to reap the returns.
Need the numbers? Here you go: companies were expected to invest $114 billion in big data in 2018, up from $31 billion in 2013. Yet only 27 percent of company executives described their big data projects as successful.
Why? BecauseâŚ
To win with data, diverse individuals need to come together
They say it takes a village to raise a child. This could not be more true than it is for data projects.
It takes an entire team of people with diverse skill sets to come together to make one data project successfulâfrom engineers solving for scale to scientists building models, business users who best understand the problem, machine learning engineers, data analysts and so on.
Look at the most successful data teams around the world⌠from the 400,000 âHumans of Dataâ at NASA (engineers, technicians and scientists) who spent years studying the lunar surface đ°ď¸ before humankindâs first step on the moon, to scientists from 20 research centers across six countries coming together to map the complete genetic blueprint of a human being.
The world has so, so many great stories of what data teams can achieve when they come together. But the problem is that all these Humans of Data have their own DNA and tool preferences, leading to what we callâŚ
Collaboration overheadâtoo many tools, workflows and variables!
And itâs not that companies arenât investing in data lakes, or infrastructure or tools. Most of the teams we work with routinely use S3, Azure, Hadoop, GitHub, Jupyter and more as part of their stacks. Yet their teams find it extremely difficult to find, access or share their data assets. And you can just forget about context and metadata.
And what about the time data peeps spend on high-value work vs. repetitive tasks aka human grunt work? A news report once pegged the time that data scientists spend on simply getting data ready for analysis at 50-80 percent. How can we reduce this âdata debtâ?
We might need toâŚ
Rethink every part of the human-data interaction.
Why do we say this? First, a little back story. We started out as a data team ourselves while building SocialCops.
As some of you may know (and were even a part of â¤ď¸ ), in the beginning we were just a small team of data scientists, engineers and economists who came together to solve big problems facing humanity by using data. As we got our hands dirty project by project, things started to break and chaos reigned.
Now, we were using the best-in-class open-source tools available at the time. That’s when we realised, these were human problems… not technology ones.
An engineer pushed the wrong data file to production. An analyst used the wrong variable in the analysis. An API spec was changed by a client without telling us. A data scientist left and rerunning the model they built was crazy. Our ML engineers struggled to explain how a model worked to the business users. Business users complained that the data team was a black box. Analysts complained the data pipeline was a black box. Engineers complained that the client problem statement was not properly communicated to them.
That’s how it all started. We began to build small tools to help our own team collaborate effectively.
These started with small fixesâlike opening up the data pipeline visually (we used Airflow) so that analysts could troubleshoot failures and to help build trust among the team when something failed. Automating basic alerts and notifications. Internal data discovery and access. Knowledge management. And so on.
By making these small, incremental changes in the human-data interactions for our own team, we were able to do big things. Our team was performing at 2x the quality, as we needed far fewer iterations to push a final product⌠in 50 percent of the time, thanks to the power of automation⌠and with 1/3rd the resources, given reduced engineering time.
And we went on to do some incredible work, including building the DISHA platform in record time for a platform of its size and scale, and being chosen as a global partner for the United Nations.
And that’s when we began to think. What if we opened up our tools to data teams around the world? Could we help them achieve their goals faster, just like we’d done with our own team⌠That’s how Atlan was born.
Introducing Atlanâa home for data teams.
Where you can bring together diverse data, tools and people to create a frictionless collaboration experience. Think of us as the glueâthe collaboration layer where you can work with data and all your tools.
Weâre trying to remove all the friction points between people and data by asking some fundamental questions.
For instance, code has a profile on GitHubâwhat would a âprofileâ of data look like? What if you could share data as easily as a Google Sheets link, without worrying about the size or format? Or what would a data versioning and approval workflow look like? What if data scientists could acquire external data within minutes, instead of the months it takes right now?
For the past 18 months, we’ve been working very closely with our beta customers to validate all these ideas and ensure that weâre adding value to these teams.
And we’re happy to share that…
Our customers are already winning with Atlan
It makes us so happy to share that today, more than 200 teams in 50+ countries rely on us to collaborate frictionlessly on data projects.
Here are some of their stories:
- âYour product saves 1.5 hours of my day, daily. Thank you!â
- âWith Atlan, we have built a predictive sales model to increase leads by 30%.â
- âAtlan is becoming the data spine on which data products in our company are built.â
If youâd like to experience data in a whole new way, come join us at www.atlan.com. We canât wait to see you!
With â¤ď¸ from Team Atlan
P.S. In case you’re wondering, our name, Atlan, is inspired by one of the earliest big data projectsâMatthew Fontaine Mauryâs work in the 1880s to map winds, currents and more for every 5-degree latitude-longitude square of the Atlantic Ocean. With daily data from 17 nations and thousands of ships, Mauryâs maps made ocean travel safer, reduced sailing time by up to 50 percent, and saved millions of dollars. Atlan is also derived from the name of Atlantisâthe worldâs most technologically advanced mythical civilization. The name reflects what we believe at Atlan; we believe in human achievement and progress, and in the power of data to get us there.
Oh, and Atlan is pronounced just like Atlas⌠only with an âNâ at the end! đ
