a project just for fun

I've started a project of my own.

I've decided to use a bunch of technologies I'm actually interested in, rather than the usual crap I have to use:

• scala - general programming
• gradle - build
• mongodb - persistence
• swing from scala - UI
• vlc / winamp / wmp - media playback
• lift - web interface, if I get around to it

I've used scala before, but just for experiments and tools.

I've been reading up on gradle, as I've been using ant fairly extensively. I am not a fan of maven, but I do like the ideas behind the dependency management.

I've used swing a fair bit before, and the application will have a fairly simple interface. I thought about trying out SWT, but I wasn't sure how it would integrate with scala.

I wanted it to work with any of the media players on my PC.

There'll probably be some sort of admin interface, if I get around to it done in lift. Lift seems to be the obvious choice in scala. I've been told that it's very different from struts, which I have used and am not a fan of.

The biggest technology leap has been moving from an SQL db to mongodb. I've heard a fair bit about nosql, and it sounded like a much better persistence mechanism for a small project like this one.

The difference between mongodb and sql is bigger than people make out.

It's similar to the difference between a strong/statically typed language and a strong/dynamically typed language.

With both, you need to be sure of what type you are expecting.

With the strong/statically typed language (sql), you have to be very careful with the exact data structures that have been defined, and be sure that the exact nullable/mandatory/empty list rules match between the data type and what the code assumes.

With the strong/dynamic  typed language (mongodb), each time you do a method call (query or update) you need to be aware that a method missing (eg, "mandatory" data missing), could actually happen with any action. The syntax doesn't allow you to just assume that something will work, you need to think about how the error cases will be handled.

With mongodb, there's no guarantees put on the data structure or content at all.
Just this simple difference,  puts a fair bit more of the data consistency checking onto the application programmer.

This trade off is perfect for a exploratory project like mine, which has a single client application. But in any data store that had multiple client applications, this trade off would be horrible to manage. There's no way that three different application could be consistent in their consistency checking, especially if any of them attempted to rectify data problems.

I've previously worked on an LDAP db, that had a provisioning interface used to ensure data consistency. All updates would go through that app, and reads would go directly to the LDAP interface.

I can imagine a similar configuration used for nosql data stores. A shared provisioning interface (library/REST/SOAP) that ensures the specific data consistency required for that application domain, instead of the SQL's one and only structure. That way the queries could make certain assumptions about the data structure.

I think that the flexibility of nosql, offers us a new way to abstract over data; which requires us to think differently about the best way of managing that data.

what is it you are really doing?

I was at a family event, and my very young cousin asked my uncle what my job was.

Instead of talking about computers or games or the internet, or any of those things that people normal fall back on when trying to explain what software engineering/development/programming is; he described the actual purpose. He said described it as :

Automating all the boring jobs, so that everyone can have a better life.

Which is a much bigger job than just making websites, games or anything like that.

It reminded me of the SICP lectures, where they broke down the title "Computer Science".
They talked about Geometry, as derived from the Greek for measuring the earth. Modern Geometry has nothing to do with actual measuring of the earth, it's just how the abstract modern geometry started out.

In the same way, Computer Science has nothing to do with computers; That's just how it's starting out.

What Computer Science (Which isn't really a science either, but I'll leave that for a later discussion) is really about is understanding process; thinking about how we do stuff.

This more abstract definition, feeds directly into my uncle's description of the true purpose of software engineering/development and programming.
It has nothing to do with computers, it's about working out how people do things, and automating (simplifying) them.
The upshot is:

• If you don't know how to do it manually, you can't automate it.
• Just because you're using a computer to do it, doesn't mean it's better than doing it manually.

Everyone knows what happens when the requirements are wrong, but people don't think enough about if the solution is actually an improvement.

There's been many times when people have shown me software solutions, that don't actually simplify (automate) the process at all. It a few cases, the software was actually more complicated than the manual process.
Those ideas always struck me as stupid, but I couldn't think of why it was fundamentally wrong.

You need to ask yourself, is this actually going to help?