Jupyter Notebook vs. TensorFlow vs. Teradata Vantage

I've created a number of daisy chain notebooks for different workflows, and every time, I create my workflows with other users in mind. Jupiter Notebook makes it very easy for me to outline my thought process in as granular a way as I want without using innumerable small. inline comments.

Incentivized

TensorFlow is great for most deep learning purposes. This is especially true in two domains: 1. Computer vision: image classification, object detection and image generation via generative adversarial networks 2. Natural language processing: text classification and generation. The good community support often means that a lot of off-the-shelf models can be used to prove a concept or test an idea quickly. That, and Google's promotion of Colab means that ideas can be shared quite freely. Training, visualizing and debugging models is very easy in TensorFlow, compared to other platforms (especially the good old Caffe days). In terms of productionizing, it's a bit of a mixed bag. In our case, most of our feature building is performed via Apache Spark. This means having to convert Parquet (columnar optimized) files to a TensorFlow friendly format i.e., protobufs. The lack of good JVM bindings mean that our projects end up being a mix of Python and Scala. This makes it hard to reuse some of the tooling and support we wrote in Scala. This is where MXNet shines better (though its Scala API could do with more work).

Incentivized

Teradata Vantage is well suited for large scale ETL pipelines like the ones we developed for anti money laundering risk matrices. It handles heavy joins, aggregations, and transformations on transactional data efficiently. We generate alert variables, adjust for inflation, and monitor establishments monthly with it, all integrated with Python and Control-M for a centralised automation across the company. For less appropriate, I would say that heavy resource demands might slow down experimentation for iterative work.

• Simple and elegant code writing ability. Easier to understand the code that way.
• The ability to see the output after each step.
• The ability to use ton of library functions in Python.
• Easy-user friendly interface.

Incentivized

• A vast library of functions for all kinds of tasks - Text, Images, Tabular, Video etc.
• Amazing community helps developers obtain knowledge faster and get unblocked in this active development space.
• Integration of high-level libraries like Keras and Estimators make it really simple for a beginner to get started with neural network based models.

Incentivized

• ETL (Extract - Transfor - Load)
• NOS to send data from Teradata Vantage to S3 and from S3 to Teradata Vantage
• Teradata GeoSpacial feature
• Bulk reading and writing in huge tables
• MPP capacity already mature
• Temporal Capacity more mature that other solutions
• TASM

Incentivized

• Need more Hotkeys for creating a beautiful notebook. Sometimes we need to download other plugins which messes [with] its default settings.
• Not as powerful as IDE, which sometimes makes [the] job difficult and allows duplicate code as it get confusing when the number of lines increases. Need a feature where [an] error comes if duplicate code is found or [if a] developer tries the same function name.

Incentivized

• RNNs are still a bit lacking, compared to Theano.
• Cannot handle sequence inputs
• Theano is perhaps a bit faster and eats up less memory than TensorFlow on a given GPU, perhaps due to element-wise ops. Tensorflow wins for multi-GPU and “compilation” time.

• Teradata is an excellent option but only for a massive amount of data warehousing or analysis. If your data is not that big then it could be a misfit for your company and cost you a lot. The cost associated is quite extensive as compared to some other alternative RDBMS systems available in the market.
• Migration of data from Teradata to some other RDBMS systems is quite painful as the transition is not that smooth and you need to follow many steps and even if one of them fails. You need to start from the beginning almost.
• Last but not least the UI is pretty outdated and needs a revamp. Though it is simple, it needs to be presented in a much better way and more advanced options need to bee presented on the front page itself.

Incentivized

Teradata is a mature RDBMS system that expands its functionality towards the current cloud capabilities like object storage and flexible compute scale.

Incentivized

Jupyter is highly simplistic. It took me about 5 mins to install and create my first "hello world" without having to look for help. The UI has minimalist options and is quite intuitive for anyone to become a pro in no time. The lightweight nature makes it even more likeable.

Incentivized

Support of multiple components and ease of development.

Incentivized

Teradata Vantage allows us to create a scalable infrastructure to support our strategic initiatives. The dedicated compute power ensures reliable performance with isolated workloads and dedicated resources, optimizing workflows for faster, more efficient data transfers. The compute clusters support ETL processes and OSF’s developers and data science team with the flexibility to create self-service analytics, to spin up/down at any time, driving better performance and minimizing costs.

I haven't had a need to contact support. However, all required help is out there in public forums.

Incentivized

Community support for TensorFlow is great. There's a huge community that truly loves the platform and there are many examples of development in TensorFlow. Often, when a new good technique is published, there will be a TensorFlow implementation not long after. This makes it quick to ally the latest techniques from academia straight to production-grade systems. Tooling around TensorFlow is also good. TensorBoard has been such a useful tool, I can't imagine how hard it would be to debug a deep neural network gone wrong without TensorBoard.

Incentivized

We have meetings at the beginning with the technical team to explain our requirements to them and they were really putting in a lot of effort to come up with a solution which will address all our needs. They implemented the software and also trained a few of our resources on the same too. We can get in touch with them now as well whenever we run into a roadblock but it's very less now.

Incentivized

Use of cloud for better execution power is recommended.

Incentivized

With Jupyter Notebook besides doing data analysis and performing complex visualizations you can also write machine learning algorithms with a long list of libraries that it supports. You can make better predictions, observations etc. with it which can help you achieve better business decisions and save cost to the company. It stacks up better as we know Python is more widely used than R in the industry and can be learnt easily. Unlike PyCharm jupyter notebooks can be used to make documentations and exported in a variety of formats.

Incentivized

Keras is built on top of TensorFlow, but it is much simpler to use and more Python style friendly, so if you don't want to focus on too many details or control and not focus on some advanced features, Keras is one of the best options, but as far as if you want to dig into more, for sure TensorFlow is the right choice

Incentivized

Teradata is way ahead of its competitor because of its unique features of ensuring data privacy and data never gets corrupted even in worst case scenario. In most cases, the data corruption is a major issue if left unused and it leads to important data being wiped off which in ideal case should be stored for 3 years

Incentivized

• Positive impact: flexible implementation on any OS, for many common software languages
• Positive impact: straightforward duplication for adaptation of workflows for other projects
• Negative impact: sometimes encourages pigeonholing of data science work into notebooks versus extending code capability into software integration

Incentivized

• Learning is s bit difficult takes lot of time.
• Developing or implementing the whole neural network is time consuming with this, as you have to write everything.
• Once you have learned this, it make your job very easy of getting the good result.

Incentivized

• Moving to Teradata in the Cloud-enabled a level of agility that previously didn't exist in the organization. It also enabled a level of analytic competency that was not achievable using other options on the aggressive timeline that was required. We didn't want to settle for reinventing a wheel when we had a super tuned performance capable beast readily available in Teradata. Teradata lets us focus on our business rather than spending money and effort trying to design software or database foundations features on an open source or lower performance platform.

Incentivized