Storing information

I have been using HSQL for many years now, and by-and-large it has been fine.  Yes it struggled under heavy load and its periodic cache flushing just aggravated a busy system, but it worked and was easy to install, set-up and use.

More recently I have been using PostgreSQL (or I should say re-visiting), which is also a good data store. Better than HSQL, more "grown up", able to cope under load and bettering tooling.  There are also pre-built "stacks" that offer lots of functionality "out-of-the box".  Of course there are some downsides, its more difficult to install, set-up and configure and its not platform independent.....but the benefits are worth the extra effort.

During this time I have also had "encounters" with Oracle, which is a different beast entirely.  BIG, time consuming to set-up and difficult to manage.  It has always left me asking "Why is it this hard ?".

I have been looking at data storage and manipulation for a number of different scenarios recently and have started to look at some of the other options,  including Cassandra, MongoDB and Solr. Problem is I don't have the time to do an in-depth evaluation of all the possibilities..... :(

Of those I did try; I found Solr easy to setup and use; piping multiple RSS feeds, via Camel and XSLT, into Solr  and performing queries.

Cassandra looks interesting, but I don't believe I ready for it just yet.  MongoDB looks very interesting....

I found the following quote useful to categorise information stores...

"What am I going to build my next app on? Probably Postgres. Will I use NoSQL? Maybe. I might also use Hadoop and Hive. I might keep everything in flat files. Maybe I’ll start hacking on Maglev. I’ll use whatever is best for the job. If I need reporting, I won’t be using any NoSQL. If I need caching, I’ll probably use Tokyo Tyrant. If I need ACIDity, I won’t use NoSQL. If I need a ton of counters, I’ll use Redis. If I need transactions, I’ll use Postgres. If I have a ton of a single type of documents, I’ll probably use Mongo. If I need to write 1 billion objects a day, I’d probably use Voldemort. If I need full text search, I’d probably use Solr. If I need full text search of volatile data, I’d probably use Sphinx."

Here :

I am going to spend some quality time with MongoDB, I will let you know how I get on....


Popular Posts