The best enterprise testing platform available.
July 09, 2014

The best enterprise testing platform available.

Gwen Lafleur | TrustRadius Reviewer
Score 10 out of 10
Vetted Review
Verified User

Modules Used

  • Maxymiser

Overall Satisfaction with Oracle CX Marketing (formerly Oracle Marketing Cloud)

We use Maxymiser in our digital department as our global online testing solution. We test creative treatments and user experience as part of our continuous improvement initiative on all of our websites.
  • Customer service - we have a wonderful solutions team that helps us with managed testing and strategy, as well as technical support for our self-service tests. They're responsive, respectful, and helpful. We have yet to run across an issue they couldn't help us with.
  • Platform integration - it's very easy to integrate into our websites, and one of the things that I most appreciate is that the single line of javascript code makes it so that we don't have to continue going back to our dev teams for further integration... we can just code our tests and launch them. It also makes it so we can test just about anything we can imagine... from simple button tests to all of our product pages to any kind of price testing we can think of, to any segments we want to test to.
  • Reporting - the reporting tools are robust and easy to use, and one of my favorite things is the filtering capability that comes out of the box so that I don't have to add filters to my individual campaigns in order to understand impact by traffic channel, etc...
  • There's a bit of a learning curve, especially if you're used to using a different platform like Test & Target. This is especially true for our developers who still need to get the advanced training to enable them to code some of our more complex tests without help.
  • We can test just about anything we want, and there are also great merchandising capabilities. However, this is a testing platform first, so some things require a bit of a workaround. That hasn't been an issue yet, however... you can still do it, but it's a bit more involved (IE testing badging on specific products.) They are working in this area... they've already added some functionality I asked them about a year ago such as targeting by weather.
  • Much better insight into customer behavior - we've been able to get a better idea of how audiences for our various websites differ from each other, even in the same geographic regions. This helps with our employee efficiency as we're better able to make decisions on how to address our customers online. This information is also shared with our offline marketing teams to help create a more cohesive overall approach.
  • We've seen significant decreases in our bounce and exit rates, helping us to widen the funnel and bring more customers deeper into our sites. We've also tested into higher adds to cart on product pages and higher overall conversion, leading to lifts in our target metrics.
  • Adobe Test and Target,Monetate
I was a power user for Adobe Test and Target in a previous position, and have found that I definitely prefer Maxymiser for pure ease of implementation and setup. No mboxes, no creating parameters that have go to into the code, no having to set up segments in my campaign in order to filter later on. Testing is more streamlined, and test setup and reporting are much faster and more efficient. We're also able to implement a much wider range of campaigns with greater capabilities on the merchandising and segmentation fronts. You do still need to do front end (and sometimes backend) coding, but you're eliminating steps up front and during campaign setup. They also offer the Visual Campaign Builder if you're doing simple testing on a single page - you don't need dev at all to use this tool, and we've had very successful tests using this... it's a great option for those who are comparing with Optimizely, etc... for a solution that doesn't require development. Button and image swaps, copy, even some basic coding adjustments can be made without any dev. This was another great benefit that T&T didn't offer at the time. They also continually update and improve this tool as well, so functionality is excellent.

For Monetate, we evaluated them both initially and at our first renewal. I found that they were about testing second, merchandising first, and they weren't able to give us a working prototype of the type of price testing we would need for our sites. I was also very disappointed by the follow-through of their sales teams... promised feedback and information were never provided leaving me to wonder what their customer service would be like after signing a contract. Between that and the technical gaps, we felt this was not the right fit for us.
Very well suited when you want to test across multiple pages and / or run heavily targeted or segmented campaigns. Lots of out of the box capabilities, and they can help add the ability to do anything else. You can create reusable custom scripts for targeting or tracking. Ease of implementation is great, and as I mentioned, the campaign reporting is top notch. If you still want additional insight, you can tie it in to 3rd parties like ClickTale or Google Analytics.

Using Oracle CX Marketing (formerly Oracle Marketing Cloud)

Our eCommerce Website Managers and Web Specialists all use Maxymiser to some extent, and as a primarily self-service client, our development teams also use it. We currently have 7 developers who have basic training and are able to code and help set up campaigns, and we will be providing advanced training to them as well as to a second development team in Europe.
2 - We have a project manager who oversees our global testing program and runs our bi-weekly testing review meetings. He works as a liaison with our services team, helping to facilitate meetings and solutions as needed, but he does not use the tool at all. I'm the second "support" user. Although I'm currently the manager for a single website, initially I was hired to run all testing and so now help provide internal support for questions on setup and testing in general. All other required support is handled by our Maxymiser solutions team.
  • Price testing
  • User experience testing on key site pages (homepage, category pages, product pages, checkout)
  • Customer segmentation / targeted experiences
  • It seems that our price testing was rather innovative, as none of the other solutions available could do what we were asking. We were able to give alternate pricing options for our products (which products differed by site) and to display new or strike-thru pricing both on the product pages and on category pages and anywhere else that product appeared with a price... all in the same campaign.
  • We found that integrating Maxymiser with ClickTale gave us some surprising insights. By looking at user visit recordings by test experience, we were able to uncover user experience issues further down the funnel that explained unexpected test results.
  • Our use of Maxymiser itself wasn't innovative, but the results of our testing were unexpected; we ran several very similar tests on two different websites serving the same geographic region with very similar products. We found that the audiences responded very differently to the same approach, which has helped us to stop making assumptions that those sets of customers were inherently similar.
  • Personalization - our program isn't mature enough yet to really use this, but we definitely want to start creating personalized user experiences in the future.
  • Mobile - this audience isn't growing as quickly for us as it is in other industries, but it's definitely growing, and this is on our radar as an opportunity for the future. Maxymiser has some great testing capabilities to help us understand this new-to-us audience so see how that experience should differ from our main site.
We've already renewed a few times - each time we look at renewal, we find that there isn't another solution available that has the testing capabilities combined with the customer service that we need for our business. I've been testing online for about 13 years and I've been very impressed with Maxymiser. They're always willing to work with us to find solutions, be it technical, training help, campaign or strategy questions.

Oracle CX Marketing (formerly Oracle Marketing Cloud) Implementation

I give it a 10 because the only issue we had was a result of not following the guidance we were given. Maxymiser provided a customized implementation guide for each site where we were adding the code. On our site implementations when we followed that guide to the letter, it was extremely fast and easy and has worked very well.
Change management was minimal - The biggest issue was in trying to implement something with remote dev teams. Key takeaway: implement the code exactly as it's specified in the implementation guide that the team will provide to you. We had some problems with trying to adjust to make the mmcore.js fit within the current format, and had to make changes down the road. This was an issue on our side and not Maxymiser, but we've learned the importance of better communication between dev and business during this phase.
  • The mmcore.js file has to be loaded synchronously... we had a 3rd party program dynamically loading asynchronous files in front of it, and had to add some exclusions. Once we made the adjustments to make sure it was implemented exactly as Maxymiser had recommended, it worked flawlessly on that and other sites where we subsequently implemented.

Oracle CX Marketing (formerly Oracle Marketing Cloud) Support

They're very responsive, helpful, and respectful. From my perspective, they're a partner in our success - I always get the feeling that they care about what we're doing and helping us to succeed. Emails are usually answered quickly and I've never had to escalate an issue. The team is always willing to help us with anything we might need... I've had them on the phone with my entire dev team in more than one instance, helping to resolve tricky coding for complicated tests. Our dev team is in Manila, and they've also been very helpful in working with us on remote training which has been extremely helpful and also effective. When they're providing managed services, the communication is always excellent, and I've always appreciated the strategic support that they provide... they don't just execute on what I hand them, they make an effort to help us make it even better.
ProsCons
Quick Resolution
Good followup
Knowledgeable team
Problems get solved
Kept well informed
No escalation required
Immediate help available
Support understands my problem
Support cares about my success
Quick Initial Response
None
Yes - In some cases, our dev backlog gets long enough that we prefer to pay for services support rather than go through the process of getting it prioritized with our dev team. We also use services for more complex tests that our team hasn't yet been trained to set up. We also have a standing weekly meeting with our technical support team to resolve any issues that surface during the regular course of testing, and we use technical services resources to help support our dev team if they run into difficulties with our self-service test setup.
Yes - Yes - communication and follow-through is excellent.
There have been several, but the most recent was when we were setting up a campaign to test a new version of one of our product page templates. Our dev team had most everything working, the campaign was set up and ready, but we couldn't get one of the tracking scripts to work and we couldn't get the campaign to display correctly in the sandbox. This was a time sensitive project, and our technical lead was quick to make himself available pretty immediately - he got on the phone with me and our dev team in Manila and helped walk through every aspect of the test and code. Within 45 minutes, he was able to pinpoint the problem with both issues and provide alternate code that would make them work. We had the test launched the next morning, met our deadline, and have seen extremely positive results from the campaign.

Using Oracle CX Marketing (formerly Oracle Marketing Cloud)

Personally, I find it quite easy to use. But for those members of our team who have little or no testing experience, it's been a bit more difficult. There's also training required for development teams in order to have your campaigns coded and set up in the most efficient way. Our developers have been able to do basic and intermediate tests with no difficulty, and they find the interface itself quite intuitive... it's just the extremely complex tests that require a bit more understanding.
ProsCons
Like to use
Relatively simple
Easy to use
Technical support not required
Well integrated
Consistent
Convenient
Feel confident using
None
  • Reporting - it's so easy it's almost just intuitive. They have training available, but I found that I didn't really need it.
  • Campaign setup. It does help if you've had basic training or previous testing experience, but I've found that I can set up a fairly complex campaign in about 5 minutes (caveat is that I write my testing brief first.)
  • Adding basic segments to your campaign - there are a lot of out of the box segments available with a wizard-like interface that makes it extremely easy to set up what you want.
  • Using the Visual Campaign Builder (VCB.) This is a very streamlined tool that makes it extremely quick and easy to set up simple tests that don't require coding. I can set up and launch a basic test in 10 minutes.
  • Not necessarily difficult or cumbersome, but it takes a bit of practice at first to get used to creating and mapping site and campaign scripts. Once you understand how it works, it's really pretty easy.
  • Obviously, the more complicated your test, the more complicated some of the setup will be. We've found that for things like testing all of a certain type of product page, there's some scripting involved that's fairly advanced. Our services team has helped us a lot, but it's a good idea to get the advanced technical training for your dev team.