Wonderful changes in Tiburon-RESTful

I’ve been rewriting Tiburon to provide a completely sane restful interface. It still does what it did before, but now … it does it so much more nicely! First: I got rid of the config file. Some folks were having trouble with JSON config files. Creating them is very easy, they are key value stores in 90% of the cases, with the remaining 10% being a “default” key, and then the value. That is, parsing them should be trivial, but we had the other problem of many editors, lots of bad brace matching. Really, the config file are a collection of JSON documents. So, my brilliant (not really, but…) concept was, why not, I dunno, use an actual JSON document store. Like, I dunno, MongoDB? Looking at the way it was being used, we had 3 document collections in a single file. So start out with 3 collections. Same document structure. Then I’ve got to write boilerplate code to query/insert/update/remove entries. I tried Mango, but I never was able to get some of the functions working. Major issues with even the examples. Next I tried the canonical MongoDB modules. And then everything worked. I wrote generic functions for the CRUD, and had one method per collection/function. Not great. More on this in a moment. Second: I altered the logic of the code so that most everything accepts/returns JSON documents by default. This is a system that has to fit into other systems, so being able to receive and send JSON is very important. This means a significant simplification in the output pathways (not generating HTML by default, and I don’t have to generate many different display pages). Since this is the server side, and it talks to db’s, I can leverage that fact and build a nice/beautiful client side to talk to the server side. Both cli and html. And have them consistent, as the clients are by definition, thin. Third: building nice unit tests. Very simple web clients to do one function, and return results. I’ve always done something like this, but now I am adding more heft to this and making this part of “test after any change”. Fourth: With all of this done, I noted that my controllers all looked the same. I mean, apart from a few labels, and other things I could handle algorithmically in a single controller, they really were identical. All that was required on the module side was a function router, a Grand Central switch to route the calls to the correct method. Sorry about this, but I like one screen full or less for my functions these days. I used to write big huge ones, but that makes debugging much harder. So I wrote the GrandCentral method, and handled the algorithmic bits I needed to, and … now I have 3 controllers in the server side. I could probably knock it down to 2 if I worked a little harder, or even 1 if I went nuts. But I can manage 3. Make things as simple as possible. But no simpler. This was Einstein’s advice to scientists everywhere, and I take it to heart. Ok, so now I have this GrandCentral method. Testing everything out. All works. But …. … I noticed something. I was developing the CRUD portions. I had them targeted to a specific collection. So I’d need 3 of each function. Which is 3x the debugging work. As the collections are all in DB’s, and the way I’ve arranged the db access would allow me to trivially switch between them … all I’d need would be a way to index them. Like, say, by noun. So the URLs now look like /verb/noun . /list/boot or /add/targ or /update/mach. And the get/put/post/del are all handled correctly in the GrandCentral, and passed to each method. So all I really need are 4 methods, with the noun indicating which collection I am working on. Sonnofagun! Serendipity! Now I’ve got unit tests to write up, and some additional logic to handle, but I may be finished with the server side far sooner than anticipated. And once this is done (this is the control plane), I’ve got the data storage plane, and the client side to develop. But the data storage plane … been thinking about this for a long time, in a different context, and I think I have a solution for it. Far more than that, but this would be the first phase in the much bigger picture, as I think I’ve solved some of the big/huge problems I need to solve at this point. Not sure, but have to implement it and try it out. This is getting very exciting …