Wednesday, September 26, 2012

What About Small Data?

This is an other post along the lines of We Are Making Things Too Hard more than the more obvious Thinking About Databases. And there may perhaps be a bit of old man “get off of my yard” nonsense in it as well. Back in the day (I warned you) we had only very simple says of dealing with data. In fact on many systems all we had were “flat files” that could be addressed either sequentially or, if we were lucky, randomly. One had to more or less know where the data was. Sorting files so that one could do a binary search though the whole data set was not uncommon.
In my first job out of university I worked for a company that used a system with two files for each set of data. One file had the data in unsorted order while the second file had a key structure that had to be frequently reorganized as efficacy would fall apart rapidly as more data was added to the files. Oh is was a kludge. I don’t really want to go back to those days.
Later I worked with operating systems that had fairly simple but reliable ISAM (Indexed Sequential Access Method) file systems. Initially they only supported a single key for a file but they rapidly grew more sophisticated to support multiple keys, duplicate keys, transaction support such as journaling. These systems were then developed to support simple relational databases that while not true databases the way we think of today but faked it pretty well. Oh and they were easy to use.
Today it seems that most of our options involve real databases. Tools like Microsoft SQL Server and MySQL are full blown relational database management systems. MUCH power! With power comes complexity though. At times and for some applications I feel like my choice is between a chainsaw and a dull knife when what I really want is a scroll saw.
What I want is to be able to declare a record type with various data and a key or two, have the system create or open the associated file and let me place or get data with a simple write or read statement. That’s it. No high priced DBA to set up a complex database after long and careful study. No complex and powerful SQL statements to build. No daemon or server application to run and communicate with. Just simple straight forward indexed access to a simple file. The sort of thing that I could do 30 years ago using Record Management Services on various Digital Equipment operating systems.
It’s out there right? Someone please tell me its out there and where to find it. Thanks.

No comments: