Past Project: Linguacode

When I was working for IQNavigator (now Beeline), I was heavily involved in FrontOffice integrations. Many of our larger customers would send us data from their systems, usually in the form of some sort of text file, which contained lists of users, contractors, jobs, time sheets, and so forth, and we would feed that into our database. (Since we used a single production database for all our customers in a multi-tenant fashion, we needed something pretty beefy, which was Oracle Exadata.) For the most part, these integrations were individual pieces of Java code that would read the files and process them in a "streaming" fashion (applying some functional programming techniques) to put them into the right place.

Needless to say, this was a maintenance nightmare. New integrations took at least a week to code and test, and needed to wait for us to deploy code to production to stand one up for a customer. And dealing with the inevitable bugs in that code was always a headache. I was charged with coming up with a better solution.

There was one constraint placed on me, though. Our CTO was particularly enamored with a product called Dell Boomi, which was an integration and automation platform, one I knew nothing about. But soon...a plan came to mind.

What if we didn't have to support all those integration formats in code? What if we had just one format of data, that outside data files were translated to before we saw them? Then we could support just one pipeline, or, at least, one per type of data we accepted. And someone else could develop the means of translating outside integration data to ours, using Boomi.

Influenced as I was by my previous work at Jabber Inc. (now part of Cisco Systems), I styled this new format as an XML-base one, along the same lines as the Jabber/XMPP protocol. Like XMPP, my format was divided into "stanzas," with a "header stanza" containing information about the specific integration, followed by one or more "data stanzas," each equivalent to a single record, like a line of text in a CSV or pipe-delimited¹ file. I named this new data format "Linguacode," after the Star Trek concept of a culturally-neutral "anti-encrypted" language form easy for non-human races to decode. (Fun fact: In-universe, Linguacode was invented by Hoshi Sato, communications officer for the Enterprise NX-01.)

Once we had constructed the needed pipelines (one per integration data type) using XML instead of text, and once I had explained to our professional services group what the requirements were for translating their integration data, the superiority of this approach became readily apparent. New integrations now took a day to stand up, rather than a week...and that day's effort was being expended by non-developers, producing the translation routine using Boomi. They could also be brought online at any time, independent of new deployments of FrontOffice. Developer involvement was limited to extending the Linguacode format as necessary to accommodate more special cases and types of data--not to mention extracting and deleting much of the old integration code. After all, as I've said many times, "The easiest code to maintain is code that doesn't exist. "

In companies like this, integrations are almost always an effective form of "vendor lock-in"; after all, once someone has made the effort to link their systems to yours in such an intimate fashion, it'll be that much more difficult for them to switch to one of your competitors and redo all that integration work (if it can be done at all!). Linguacode closed a very big gap we had in our integrations strategy, to great acclaim. I left Beeline in 2019 for Carbon Black, but I bet they're still using Linguacode-based integrations today.