Datasphere alternative
SAO has been experimenting with the optimal delivery of enterprise data for many years. The first attempt was SAP Data Hub, which was a good concept, but it consumed too many resources and was therefore not economical. SAP is currently trying the very complex Datasphere concept, see also page 48. But there is another way, as Richard Brouwer, Lead Sales Engineering Specialist for SAP at Fivetran, and Benedikt Engel, Solutions Architect EMEA at Snowflake, tell us in an exclusive E-3 double interview.
E-3: Snowflake and Fivetran, what brought them together and what connects them?
Richard Brouwer, Fivetran: Snowflake and Fivetran have been partners for about a decade, plus we are each other's customers. -Fivetran is considered one of Snowflake's elite technology partners and is Data Integration Partner of the Year 2022. Both companies have made it their mission to get data out of silos, and together we offer a solution that does that. Typically, the data does not come from Snowflake, but from other systems. Fivetran provides out-of-the-box, fully managed pipelines that feed data into Snowflake. In doing so, our unique ability to replicate SAP data also makes us the preferred integration partner for Snowflake. Many SAP customers know that delivering SAP data into Snowflake is cumbersome. Fivetran easily and efficiently delivers this data in near real-time. Our goal is to bring the data into Snowflake as a perfect replication of the source. Snow-flake then serves as a trusted, single repository for all data.
Angels: In the meantime, it is no longer said that "data is the new oil", but "time is the new oil". This is also what Snowflake and Fivetran are all about: Snowflake as a data platform and Fivetran as an ELT that makes data from countless sources available in almost real time. With Snowflake, the data is harmonized and analyzed, which provides valuable insights and brings an enormous increase in efficiency and effectiveness. As a result, we can now address new issues for which the cost-benefit ratio was previously not right.
E-3: And specifically, where do you complement each other?
Brouwer: With more than 400 connectors, Fivetran delivers data from a wide range of sources - and in a format that allows customers to get the most out of their data on the Snowflake platform. For example, in Salesforce, SAP data can be connected to more than 460 third-party sources. Because the data is available in the right format and at the optimal frequency, its value is much easier to leverage with the capabilities of the Snowflake platform than with other solutions. Even as the amount of data and the number of sources grow exponentially, our Snow-flake customers always have all their data at their fingertips.
Benedict Angel, Snowflake: Snowflake then provides the functionality to analyze the data and enrich it with more data about the Snowflake Market-place, or otherwise makes it profitable so customers can make the right decisions.
E-3: From the perspective of an existing SAP customer with ECC or S/4, what can you offer that SAP can't?
Angels: A scalable and cost-effective cloudnative data platform with an architecture that is easy to use, cloud agnostic and features workload isolation thanks to the separation of compute and storage and computing resources. Our customers can flexibly request compute clusters, which are provided on a sub-second basis and paid for in a pay-as-you-go approach - in other words, a true SaaS solution.
Brouwer: For SAP customers with ECC or S/4 systems, our comprehensive offering includes a range of superior solutions for replicating their SAP data. With our efficient multi-cloud data replication, we enable them to replicate seamlessly across different cloud platforms and provide the flexibility to choose the most appropriate environment. Our fully managed approach ensures hassle-free replication, while on-premises data processing improves performance and provides real-time insights. By securely migrating data and replicating in near real-time, we minimize the impact on source systems and customers can cost-effectively load their SAP data into Snowflake.
E-3: What is your relationship with SAP?
Brouwer: Fivetran is an SAP partner and the first certified connector will be available soon.
Angels: When it comes to the analytics platform, Snowflake often has joint customers with SAP. Synergies often develop here: SAP ERP is one of the data sources, while Snowflake supports customers in realizing use cases that in the past could hardly be implemented or only with a lot of effort. Our platform is used to analyze SAP and non-SAP data, sometimes extended with data from Snowflake Marketplace or streaming data from IoT devices.
E-3: In the broadest sense, you are involved in data management.
There are excellent SQL databases out there, where exactly do you see the current challenge in managing enterprise data?
Angels: Many customers choose Snowflake to solve the following problems: First, their data exists in many copies, some of which are inconsistent. The result is data redundancy. As a result, different analyses of the same data often deliver different results, making it virtually impossible to manage the business on a data-driven basis. Second, due to physical limitations or different data responsibilities, data is located in completely different databases - in other words, in data silos.
This means that data cannot be combined. Their benefits can then only be realized to a small extent. If they can be combined, opaque approval processes and different options within the respective system make uniform governance difficult, which quickly leads to security problems. And third, we see that existing solutions do not grow with the business - keyword limited scaling. Often, systems need to be planned for maximum utilization because they are not elastic. If you apply the usual depreciation over five years, it is often far too risky to start a project at all because it is difficult to estimate costs and benefits.
E-3: Fivetran says on its website: "The Automated Data Movement Platform", and Snowflake also offers data distribution. Does data have to be moved?
Brouwer: Yes - and with ever-growing data volumes and sources, automating data movement is the only way to succeed. Creating these pipelines manually leads to numerous challenges. Finding qualified data engineers who can create the pipelines is becoming even more difficult. In addition, they need to be continuously maintained and improved. Data movement is especially needed when data from different sources needs to be linked. If they are on one platform, you get answers much faster and more efficiently.
E-3: So it's all about data movement and transformation?
Brouwer: When we compare Data Movement with Data Distribution, we are talking about different forms of data movement. Fivetran mainly moves raw data or source data. Such data needs to be moved because it usually needs to be combined with other sources. This also includes the transformation of the data. Transforming this data and enriching it with other data sources goes much more efficiently in Snowflake. When we talk about Data Distribution, we are more talking about data products. This type of data is better suited for zero-copy data in Snowflake.
Angels: Snowflake aims to break down data silos by storing data only once and allowing anyone to access it. This is achieved through our unique architecture, including the separation of compute and storage. This gives us unlimited scalability in storage and compute.
E-3: So where is the origin of Snowflake?
Angels: Snowflake originated in the cloud - one of the fundamental building blocks of the Snowflake architecture. By making Snowflake available on AWS, GCP, and Azure, and linking the accounts together, we create what we call our Snowgrid, which enables us to provide zero copy data sharing, replication, and failover to our customers. This allows our customers to enter into a direct exchange with other customers and share their data directly, not only within the company boundaries, but also with other organizations, partners, third parties and other industries. With this approach, we break down data silos - geographically and also across cloud providers. The data is always live, there are no unnecessary copies. But the data has to be brought to Snowflake first, and Fivetran with its many connectors is the solution of choice for this.
E-3: SAP has developed a concept with the Data Hub so that the data can stay where it is stored, right?
Brouwer: That is true - but as a rule, the data there is neither in the format in which the company needs it, nor in the necessary frequency. In addition, setting up a data hub requires a setup of the SAP system. In addition, SAP data is very valuable. To leverage this value, it must be combined with other data sources. Snow-flake is a much better platform for this. This is because the Data Hub is based on the concept of data visualization. To combine data from different sources, you still have to move large amounts of data. So SAP Data Hub is good for examining data, but it is not ideal for a production environment.
E-3: Elevator Pitch: What challenges should an existing SAP customer definitely contact you about?
Angels: If it cannot or can only with difficulty implement requirements by the business departments, whether because of the price/performance ratio, because functionality is missing in the platform or because the scalability of the architecture is not sufficient at the point to support additional workloads.
Brouwer: If SAP data is not available in a timely manner or in the right format or level of detail, it is the right time to turn to Snowflake and Fivetran. We enable customers to extract not only SAP data, but also important or business-critical data from other external data sources and combine it into a central data warehouse. This is how we break down data silos and enable data-driven decisions.
E-3: Where and how does your offering optimally complement an ECC or S/4 architecture?
Brouwer: In the efficient, cost-effective and fast replication of data from SAP. We also support small and large use cases from ECC or S/4 to Snowflake. With Fivetran's unique CDC capabilities, transformations to Snowflake also run efficiently and quickly, delivering data in near real-time or in batch. By linking SAP data with data from other sources, customers can get much more value from their SAP data.
Angels: This is exactly where Snowflake comes in: We generate insights from all data from all linked sources. In that sense, we sit on top of the ECC source systems.
E-3: Thank you for the interview.
