A new partnership that was announced between Qubole, the cloud big data-as-a-service company, and Snowflake Computing, the only data warehouse built for the cloud, will enable organisations to use Apache Spark in Qubole with data stored in Snowflake. This new integration between cloud services allows data teams to build, train and put in production powerful machine learning (ML) and artificial intelligence (AI) models in Spark using information stored in Snowflake. It also enables data engineers to use Qubole to read and write data in Snowflake for advanced data preparation such as data wrangling, data augmentation and advanced ETL to refine existing Snowflake data sets.
Qubole provides an enterprise cloud data platform for all types of big data workloads. It enables organisations to operationalise data lakes through infrastructure automation and cloud optimisations for leading open source engines. The new partnership automates the connection between Qubole and Snowflake, eliminating the complexity of manually configuring Spark and reducing the time to train and deploy ML and/or AI models with Snowflake data. This integration also provides one-time, secure credential management between Qubole and Snowflake, and access to Snowflake data through Scala and Python via Qubole's Dataframe API for Apache Spark.
Today, more and more businesses are looking to build a cloud-based data infrastructure to gain agility, scale broader analytics capabilities, as well as lower cost of ownership. Concurrently, moving data warehouse infrastructures to the cloud and building data lakes can dramatically improve organisations' performance, concurrency and simplicity. With this announcement, enterprises have the best of both worlds. They will gain access to a simple, out-of-the-box integration between Qubole and Snowflake for the most performant, cost effective, and proven solution to any alternatives in the market.
Snowflake Vice President of Alliances, Walter Aldana, said, "For the first time our joint customers can leverage the computing power of Apache Spark and Snowflake together in an optimised environment. Customers can now benefit through our integration with Qubole to transform an organisation's decision-making abilities through more advanced analytics. Our customers who have been asking for additional ML capabilities can now build unique models to improve their business. We believe so much in this partnership and the value it brings to our customers that Snowflake is already using Qubole internally to augment our security capabilities using ML techniques."
According to Qubole COO, Kevin Kennedy, "Enabling AI and Machine Learning is a strategic imperative for every company. The fastest path to success is combining Qubole's cloud-optimised Apache Spark implementation with data stored in Snowflake to get frictionless deployment, lowest TCO, and unlimited scale."