The DEATH and REBIRTH of the Scoop™ Newsroom System

The current situation:
 
Martha and I have some news for you about the future of the Scoop newsroom system SCS supplied you and that we have supported and you have used for so many years.
 
We call what's installed at various SCS Scoop sites Scoop 5.5.  (Or Scoop 5 for short.) And the short, sad story is that Scoop 5 is reaching end of life.  While all the good things of Scoop remain good, the bad stuff is getting out of hand.   Moving to workstations running Windows 7 and later saw cracks in the GUI.  (E.g., drag­ n­drop seemed to have dropped off the face of the earth.)   Mac and PC versions of the clients could never seem to stay in sync.  Especially troublesome was the lack of plugins that were compatible with current versions of InDesign and modern workstation platforms.   What we've heard from many sites was "We are just sitting on it."
 
Believe me, we tried to fix this.  We shelled out big bucks to buy Scoop Publishware's Scoop 5.5 source code.  What a fiasco!  I agreed to take the code "as is". Only the real situation was the code was "as isn't".
 
We tried to build it from source and found that many of the "module libraries" (i.e. the components that WinEdit, etc. used) were a mix of custom code, Borland code, and licensed, proprietary third­ party binary components. These weren't compatible with any build tools, either current ones or legacy ones.
 
One typical and disturbing situation had to do with eLibrary.  We all know how valuable your library is. At some of our sites it is the only recorded history of the local community. You could try to replace the application, but what do you do about the database?  So way back Scoop licensed Surfinity for the library's full­ text indexing engine. A nice tool for 1999, however what about today?  Where is its source code?  We never found it.  We couldn't even find the Surfinity company. We were really stuck in a software swamp.
 
BTW ­ I told our daughter, Sharon, about this and she said, "Dad, you at SCS are so lucky. You keep complete control of and distance from the platforms you use. Most of the rest of the software world makes junk like you are seeing.   If it runs, just ship it and let others worry about ongoing, long­-term support."
 
The issues with the Windows GUI were severe.  We talked to the former Scoop team members. They said that Scoop's drag and drop problems lie deep in the internals of the proprietary Microsoft's foundation classes that Scoop 5 uses.  It is impractical to try and fix them.
 Ulf Wilkenson, formerly Scoop Publishware's owner, once employed a team of experienced and competent Swedish  developers. Things began falling apart when he fired them and contracted with an overseas development company to save money. They were responsible for the failed Scoop 6 effort.
 
This isn't the first time I've seen off­shoring development kill a product or even a company.

There's a phrase that begins "Up the creek..," that
comes to mind. The situation with Scoop 5 is intolerable.
 
Reject the old ways
 
When we think of Scoop 5, we see something that clashes with our culture and business model.  If you really checked it out, you would agree with us.
 
It allowed five storage management solutions: ­ the file system (i.e., no database), MS SQL, a licensed, relational database management system (RDMS) from Microsoft and only for Microsoft OS’s, Oracle SQL, also proprietary and, alternatively, both My SQL and PostgreSQL, popular, platform­i ndependent, free and open source RDMS’s.
 
Why you would want to offer five mechanisms when one will do is beyond me.  Especially when two of them require costly licenses and restrictions on resellers (like us).
 
The information retrieval storage system from Surfinity, which was used by eLibrary, was the source of its own special problems.
 
Then there was the offering of both InDesign and QuarkXPress support.  Well, you know what happened to QuarkXPress.   I don't know a single reseller or XTension developer that liked dealing with them.  We certainly didn't.
 
You can't imagine how both Mac and Windows client workstations were supported. There were two separate programs and two separate development teams.  The Mac guys were quite good. The Windows team, not so much.
 
Not only that, each required special coding to use the proprietary GUI tool kits each had.
 
I could go on for hours.  Scoop 5 can't be where we start from if we want to go anywhere.
 
The proposed future:
 
We took a bold step.  We forswore any attempt to incrementally progress step­ by ­step from the existing (almost) working Scoop 5 system and its code base. To do so would just have placed us in the position of being upwardly compatible with previous errors. And forever fixing ancient bugs.
 
The first step was to say goodby to QuarkXPress, then the file system and MS SQL or Oracle for data management.  Junking Surfinity was a no­brainer and so were platform dependent GUIs and proprietary operating systems.
 
What a relief!  Now we could concentrate on doing the good stuff. The choice of platform was easy, just use Linux on NUCs, like what our current low­ cost, high performance advertising systems run on every day. Or support cloud­based deployment as an alternative. Scoop 7 uses TLS (Transport Layer Security) to encrypt all communications between the server and the client. TLS is the same technology used in web browsers to provide the "S" in HTTPS. Effectively this means that, if properly configured, any and all data exchanged through Scoop cannot be read by any eavesdroppers.
 
Kicking out Surfinity was easy.  We substituted ElasticSearch, currently a very popular FOSS search engine. Unplug Surfinity, plug in ElasticSearch and voila, searches are better, more intelligent and faster. And it's free.  No wonder Surfinity is gone.  BTW ­ We have built ElasticSearch databases with over a terabyte (over 10 year's worth) of data from Scoop newsroom systems.
 
One of the ugliest parts of Scoop was its rights management technology.  Not only could we not find all of it, what we did find sucked big time.  We junked it all. If you want to keep someone from using software, it should not be done module by module, but at a higher level. We've never done this before and don't want to start now.   What if it breaks?  What kind of vendor would be willing to jeopardize a customer's publishing for a payment?
 
To unify the GUI across workstations, be they Macs, Windows PCs or Linux, we chose Qt.  Qt is a GUI toolkit that allows a single code base to be used for multiple platforms.  Qt is dual­ licensed under both open source and proprietary/commercial licenses. We are using it under the free, open source (FOSS) license.
 
With our unification of all database access through an SCS developed front end to PostgreSQL, we could hide the differences in file and operating systems.  Qt helps with this and many other multi­platform issues.
 
Just how special is Qt?  Well, it has built­in components for internationalization that allow a product line to be deployed using multiple languages and localizations. Everything from alternate strings (words and phrases) to how dates, times, numbers and money are displayed can be done through Qt.  What a super tool!

Here's the surprise!
 
We have a partnership to announce.
 
There was new coding done after Ulf sent us the source we bought.   It was done by his former 25% partner Juha Siintola (Jussi) in Finland.  Jussi bought out Ulf, acquiring all of Scoop Publishware.  On Thursday August 3, 2017 we inked a deal with Jussi that changes everything.
 
Our agreement gives us joint copyright ownership of the Scoop 5.5, 5.7 and 6 code (and now Scoop 7) and the rest of the intellectual property of Scoop Publishware.
 
Jussi will supply development resources focused on InDesign plugins.  We will focus on the rest.  We will not compete with each other. Our territory is the Western Hemisphere and his is the rest of the world.
 
We will be setting up Scoop to run for those who speak Spanish, French and Portuguese.  SCS already has customers in 21 countries, including ones in North, Central and South America and the Caribbean, so the internationalization is appropriate.  (William knows all these languages, plus about a half­dozen more.)
 
With the partnership with Jussi, we start with installed bases of Scoop in over six countries.  Clearly the system is well liked.  It's just suffering from bit rot.
 
Going forward, Scoop 7 is in alpha release. Interestingly, it currently has about 75% of the functionality of Scoop 5 and yet has a code base of about 50,000 lines of code.  Scoop 5 has over 450,000 LOC.   What the heck!
 
Call Kurt Jackson (610 746­7700) to talk further. He can arrange demos and provide additional information.