Google Cloud Data Fusion

Google Cloud Data Fusion is a managed service for creating and managing data pipelines. This cloud-native solution enables users to build and control data pipelines quickly. It offers a web interface for constructing scalable data integration solutions, allowing users to clean, prepare, blend, transfer, and transform data without managing the infrastructure. Users can access it through its web interface or command-line tools. Tasks such as creating a Google Cloud project, creating and deleting Cloud Data Fusion instances, viewing instance details, and managing permissions can be performed in the Google Cloud Console. The Cloud Data Fusion web interface provides pages to design data pipelines and use Cloud Data Fusion functionality.

Command-line tools can also be used to create and manage Cloud Data Fusion instances and pipelines. The service offers a REST reference for creating and managing Cloud Data Fusion instances on Google Cloud and a CDAP reference for creating and managing pipelines and datasets. Google Cloud Data Fusion integrates with Google’s big data tools for data security and immediate data availability for analysis. This integration aids in developing data lakes with Cloud Storage and Dataproc, data movement into BigQuery for data warehousing, or data transformation for storage in a relational database like Cloud Spanner.