Google Cloud Dataflow Reviews & Insights

We use Google Cloud Dataflow as the primary ETL engine for our billing application. Our architecture ingests raw financial data stored in Cloud Storage (Excel format), which is then processed via Dataflow pipelines to handle data cleansing, schema mapping, and validation. We use Google Cloud Dataflow's batch processing to transform this unstructured data into structured datasets within BigQuery. This automatically triggers a generation of new invoice and keeps it ready for download.

It is best in cases where you have batch as well as streaming data. Also in some cases where you have batch data right now and in future you will get streaming data. In those cases Dataflow is very good.
Also in cases where most of your infra is on GCP.
It might not be good when you already are on AWS or Azure. And also you want in-depth control over security and management. Then you can directly use Apache beam over Dataflow.

In our company we are using Google Cloud Dataflow to create data pipe lines for data transformation and ingestion use cases before loading data into database. Flexibility to create our own flex templates for any special case handling. Capability to fit streaming and batch data loads are some benefits. We have some real time loads, which Dataflow helps alot.

Based on my experience, streaming / real time / machine learning / AI type of processing and batch processing which needs less transformation are very well suited. Work load that needs complex transformation / multiple hops gets very complicated to implement. New feature like Dataflow SQL option will come in handy for sql heavy users.