1. Home
  2. Open-Source Database Software
  3. SQLite Reviews
  4. User Review of SQLite
SQLite: Instrumental in Performant ETL Pipeline
July 07, 2022

SQLite: Instrumental in Performant ETL Pipeline

Bradley Wilson | TrustRadius Reviewer
Score 10 out of 10
Vetted Review
Verified User

Overall Satisfaction with SQLite

SQLite has been instrumental in our ETL pipeline. We utilize it heavily to ingest JSON and XML from multiple sources like Rest API, SFTP, and WebDAV repositories. It is performant for our needs and allows our development staff to work in a familiar language for them, reducing our technical debt and improving our response time. We manipulate our data on the client and package it for export or import to our primary RDBMS solution. The extra data transformation tools SQLite has available made it an easy choice to make use of.
  • Direct data transformation from JSON/CSV to table
  • Easy to format data exported to desired layout and file type
  • Direct ingestion from Rest API
  • Familiar language syntax for SQL Server developers
  • Lack of procedures makes for convoluted data manipulation via nested triggers
  • Linked Server with MySQL/SQL Server would be very much appreciated
  • Basic Table manipulation, not drop and recreate
  • Direct JSON/CSV integration
  • Portability. We can develop on multiple platforms, then ship and get the same consistent behavior
  • Performant. Some of our integrations are in processed in real time from our internal application to an external source. If SQLite were not performant, we would explore an alternative solution.
  • We have had to invest more time in our staff to learn and make use of SQLite properly
  • We have incurred more technical debt as we have introduced additional technology and complexity to our systems.
  • We are able to respond more rapidly to our end users needs.
We looked at other traditional RDBMS products, but found them to be cumbersome to deploy. They take up more space, and consume more computing resources than SQLite does. While the performance or direct integration to our primary applications may have been better or easier if we had gone with a traditional RDBMS, the performance of SQLite has been more than acceptable. The performance and speed to deploy made SQLite a much more attractive option for us than a traditional RDBMS.

Do you think SQLite delivers good value for the price?

Yes

Are you happy with SQLite's feature set?

Yes

Did SQLite live up to sales and marketing promises?

Yes

Did implementation of SQLite go as expected?

Yes

Would you buy SQLite again?

Yes

The portability and ease of use has us doing local development, and then shipping the database to production for implementation. This compact deliverable object is great. Until multiple developers need to access it at a time. It is a caveat, but we create a database per integration as it is easier to manage. Using a SQLite database to manage more than one integration or task is not something we would recommend.