Elementl raises $33M Series B for its data orchestration platform based on Dagster

Mai 24, às 17:00


4 min de leitura


0 leituras

Elementl, a startup that is building a data platform based on the popular Dagster orchestrator, today announced that it has raised a $33 million Series B round led by Georgian. This round also saw...
Elementl raises $33M Series B for its data orchestration platform based on Dagster

Elementl, a startup that is building a data platform based on the popular Dagster orchestrator, today announced that it has raised a $33 million Series B round led by Georgian. This round also saw participation from new investors 8VC and Human Capital, as well as existing investors Sequoia, Index, Amplify, Hanover and Slow. The new round brings the company’s total funding to $48.8 million.

As is so often the case, Dagster founder Nick Schrock also founded Elementl after many years at Facebook, where he also co-created GraphQL. Schrock is currently the company’s CTO and chairman, with his former Facebook colleague Pete Hunt now the company’s CEO. As Hunt told me, he had invested in Elementl as part of its 2017 seed round — mostly as a bet on Schrock. Hunt admitted that at that point, he didn’t really understand the value proposition of Dagster but as he worked on more data problems at Facebook and then later at Smyte, the anti-abuse service he co-founded and later sold to Twitter, the need for better data orchestration quickly became clear to him.

Image Credits: Elementl

“I realized that there are these big complex data pipelines that are making very serious decisions — not just taking down social media posts but also deciding who gets a mortgage, all that stuff. Once you get to a certain size, every company is a data company and every company has a data platform,” Hunt said. This also means that managing their data pipelines is one of the biggest challenges for many companies.

Apache Airflow remains one of the most popular tools for building these pipelines (and there are plenty of startups that bet on it), but Schrock was looking to build a more modern system that was optimized for the world of cloud, DevOps and containers. But the team also rethought data pipelines from a high-level perspective. “The way people have historically built data pipelines is that they think in terms of tasks. So step A to step B — and then do step C. Within those steps, they could do anything and you don’t really know — they could write to some database in a way that you don’t expect and you have no way of controlling that or having observability into that step,” Hunt explained.

Image Credits: Elementl

Elementl rethought this with what it calls a data asset (which could be a table in a data warehouse or a file sitting in a data lake) at its core. So instead of thinking about tasks as the core abstraction, Elementl (and Dagster) focus on the data assets. “By centering this notion of an asset at the core of our system, we get a ledger of every data asset in the organization and every state transition that it’s gone through, along with all the metadata associated with it. That’s a mental model that developers love,” said Hunt.

Given that it competes with well-tested tools like Apache Airflow, Dagster also needs to work well for large organizations — and it needs to be a legitimate open source project, too. Like most open source startups, the company is layering enterprise features like single sign-on, role-based access and support for teams on top of the open source project as it builds out its commercial offering. And since Airflow is so popular, the team also recently launched a tool that allows current Airflow users to run data pipelines written for Airflow on Dagster.

Over the course of the last year, the number of active projects that use Dagster has tripled, the company says, and so has the overall open source community around it. Currently, companies like DoorDash, Flexport and Aritzia are using Dagster in production.

“Dagster was built from the ground up to provide a transformative developer experience while supporting the most demanding use cases in data engineering. Our unique abstractions and asset-first approach are really resonating with data practitioners, and we’re seeing this play out
in our key growth metrics,” said Schrock.

The company plans to use most of the new funding to build out its go-to-market organization.

“Our R&D team adopted Dagster for data orchestration over a year ago after an evaluation of the solutions in the space. We’ve been impressed with how Dagster has accelerated our engineering team’s productivity and ability to efficiently ship production-grade data pipelines.” said Emily Walsh, lead investor at Georgian.

Continue lendo

Marktechpost AI Research News

Perp-Neg: Unveiling Image Potential with Negative Prompts and Stable Diffusion
Despite the remarkable capabilities demonstrated by advancements in generating images from text using diffusion models, the accuracy of the generated images in conveying the intended meaning of the original text prompt is not always guaranteed, as found by recent research. Generating images that effectively align with the semantic content of the text query is a […]

Hoje, às 02:51

AI | Techcrunch

Marvel’s Spider-Man 2 features Miles with web wings and Peter in symbiote suit
Sony held its PlayStation Showcase on Wednesday, revealing for the first time 10 minutes of gameplay for the highly anticipated Spider-Man 2 video game. Fans also saw a short clip dedicated to the game’s...

Mai 24, às 22:28


Google celebra o Big Mango na Austrália com Doodle
O Google está celebrando o Big Mango da Austrália, um ícone de viagem e destino turístico construído em homenagem à indústria agrícola de Bowen. A estrutura é um dos muitos “Big Things” em todo o país – uma...

Mai 24, às 22:27

AI | Techcrunch

Elon Musk ushers in DeSantis’s 2024 campaign, signaling a new era for Twitter
Twitter, once synonymous with Trump, is Musk's media machine now Florida Governor Ron DeSantis filed the paperwork to officially launch his presidential campaign this afternoon, but opted to...

Mai 24, às 22:14


WhatsApp está desenvolvendo recurso de nome de usuário em sua última atualização beta para Android
O WhatsApp continua sua busca por melhorias na experiência do usuário, e agora está trabalhando em um recurso de nome de usuário. Na versão beta mais recente do WhatsApp para Android, o app ganhou...

Mai 24, às 21:46

AI | Techcrunch

Snowflake acquires Neeva to bring intelligent search to its cloud data management solution
Snowflake is all about managing data in the cloud, whether as part of a structured data warehouse, or less structured data lake. Neeva, a search startup from two former Google employees, has been all about...

Mai 24, às 21:35


These Technical Skills Are Extremely Useful for Test Automation | HackerNoon
These Technical Skills Are Extremely Useful for Test Automation [email protected] Long; Didn't ReadIf you work as a manual tester but want to transition to test automation, then you’ll need a few technical...

Mai 24, às 21:30


Meet the Writer: HackerNoon Contributor Jin Park - Technical Writer | HackerNoon
Meet the Writer: HackerNoon Contributor Jin Park - Technical Writer [email protected] Long; Didn't ReadJin is a freelancing whizz-kid with a love for tech, the intricate dance of neuroscience, and a knack for...

Mai 24, às 21:02


How Security Analysts Can Use Artificial Intelligence (AI) in Cybersecurity
Artificial intelligence (AI) has been a game-changer in cybersecurity. It has revolutionized the approach to cybersecurity by providing advanced techniques to detect and mitigate cyber threats. The use of AI in cybersecurity is increasing rapidly, with many companies adopting it as a key tool in their cybersecurity strategy.

Mai 24, às 20:58


TUDO sobre Final Fantasy XVI, aguardadíssimo RPG da Square Enix
Índice Estamos quase lá, só mais um mês de esperaO que se passa no mundo do jogoOs personagens do jogo e o decorrer de suas vidas Um gameplay cheio de ação e variedadeE os outros aspectos do jogo, como a...

Mai 24, às 20:00