Dataminr Pulse is a Real-Time Event and Risk Detection solution for businesses, public sector, and news organization, that leverages AI to give users early indication of business-critical information about risks to people, a brand, and physical and virtual assets – so the user can respond with speed and confidence. For news, Dataminr's AI platform delivers real-time news analytics, that detects global events by processing billions of public data inputs per day in a wide range of formats and…
$15,000
per year
Google Cloud Pub/Sub
Score 9.1 out of 10
N/A
Google offers Cloud Pub/Sub, a managed message oriented middleware supporting many-to-many asynchronous messaging between applications.
For any news editor, Dataminr is a must. It is the only platform I have used which I would 100 percent recommend to anyone who needs constant access and notification to all that is going on in the world's media. Various other platforms provide information, but Dataminr collates all these other platforms together into one easy to use interface. It is remarkably quick to check Dataminr after receiving a desktop or email notification and find out what is going on regarding a specific subject. As a news editor, I am required to manage a news team that produces copy for breaking stories, and Dataminr allows me to find news and commission it faster than any other platform I have previously used.
If you want to stream high volumes of data, be it for ETL streaming or event sourcing, Google Cloud Pub/Sub is your go-to tool. It's easy to learn, easy to observe its metrics and scales with ease without additional configuration so if you have more producers of consumers, all you need to do is to deploy on k8s your solutions so that you can perform autoscaling on your pods to adjust to the data volume. The DLQ is also very transparent and easy to configure. Your code will have no logic whatsoever regarding orchestrating pubsub, you just plug and play. However, if you are not in the Google Cloud Pub/Sub environment, you might have trouble or be most likely unable to use it since I think it's a product of Google Cloud.
Collating various sources into one interface makes keeping track of breaking news remarkably easy.
The live update feature means updates from breaking stories are fed directly to the consumer. This means users do not have to go out their way to find out every last detail of a story, and can instead use their time writing the story.
With a pub/sub architecture the consumer is decoupled in time from the publisher i.e. if the consumer goes down, it can replay any events that occurred during its downtime.
It also allows consumer to throttle and batch incoming data providing much needed flexibility while working with multiple types of data sources
A simple and easy to use UI on cloud console for setup and debugging
It enables event-driven architectures and asynchronous parallel processing, while improving performance, reliability and scalability
Setting up Dataminr can be relatively obtuse. Although basic functions are easy to use, the more in-depth aspects require research and training to use correctly.
Topics are very broad, which can be a negative at times. It would be useful to have 'football' as a topic rather than 'sport', for instance.
It serves all of our purposes in the most transparent way I can imagine, after seeing other message queueing providers, I can only attest to its quality.
It has many libraries in many languages, google provides either good guides or they're AI generated code libraries that are easy to understand. It has very good observability too.
Connection issues/downtime occur but they are rare and generally quickly fixed. There are frequent seminars and tutorials to teach the platform. This is an excellent decision from the company and has made me more willing to learn how to use Dataminr correctly. As a result of this, I use the platform more than I would have if I had been forced to learn everything myself.
They have decent documentation, but you need to pay for support. We weren't able to answer all our questions with the documentation and didn't have time to setup support before we needed it so I can't give it a higher rating but I think it tends to be a bit slow unless you're a GCP enterprise support customer.
Before using Dataminr we did not use a similar platform, with stories discovered by simply crawling the web (TweetDeck/Reddit/news sites, etc). Because of this, Dataminr has been a game-changer for us and has revolutionized the way we have worked since it was introduced. Being able to completely remove one aspect of the job (searching for news) has increased productivity in all other areas.
Having used Amazon Web Services SNS & SQS I can say that even if the latter may offer more features, Google Cloud Pub/Sub is easier to use. On the other hand, usage of SNS & SQS as well as documentation and troubleshooting is easier with the AWS solution. Since we are not using GCP only for Pub/Sub the choice depends on other variables.
You can just plug in consumers at will and it will respond, there's no need for further configuration or introducing new concepts. You have a queue, if it's slow, you plug in more consumers to process more messages: simple as that.
Since starting to use Dataminr, our traffic has vastly increased. Though not entirely to thank for the growth, Dataminr has played a role in talkSPORT.com's ComScore increasing by 150 percent over the past 18 months.
Increased traffic is partly due to producing more content, and Dataminr has directly led to discovering a number of stories we previously would have missed.
