The current situation:
Martha and I have some news for you about the future of the Scoop newsroom system SCS supplied you and that we have supported and you have used for so many years. We call what's installed at various SCS Scoop sites Scoop 5.5. (Or Scoop 5 for short.) And the short, sad story is that Scoop 5 is reaching end of life. While all the good things of Scoop remain good, the bad stuff is getting out of hand. Moving to workstations running Windows 7 and later saw cracks in the GUI. (E.g., drag ndrop seemed to have dropped off the face of the earth.) Mac and PC versions of the clients could never seem to stay in sync. Especially troublesome was the lack of plugins that were compatible with current versions of InDesign and modern workstation platforms. What we've heard from many sites was "We are just sitting on it." Believe me, we tried to fix this. We shelled out big bucks to buy Scoop Publishware's Scoop 5.5 source code. What a fiasco! I agreed to take the code "as is". Only the real situation was the code was "as isn't". We tried to build it from source and found that many of the "module libraries" (i.e. the components that WinEdit, etc. used) were a mix of custom code, Borland code, and licensed, proprietary third party binary components. These weren't compatible with any build tools, either current ones or legacy ones. One typical and disturbing situation had to do with eLibrary. We all know how valuable your library is. At some of our sites it is the only recorded history of the local community. You could try to replace the application, but what do you do about the database? So way back Scoop licensed Surfinity for the library's full text indexing engine. A nice tool for 1999, however what about today? Where is its source code? We never found it. We couldn't even find the Surfinity company. We were really stuck in a software swamp. BTW I told our daughter, Sharon, about this and she said, "Dad, you at SCS are so lucky. You keep complete control of and distance from the platforms you use. Most of the rest of the software world makes junk like you are seeing. If it runs, just ship it and let others worry about ongoing, long-term support." The issues with the Windows GUI were severe. We talked to the former Scoop team members. They said that Scoop's drag and drop problems lie deep in the internals of the proprietary Microsoft's foundation classes that Scoop 5 uses. It is impractical to try and fix them. Ulf Wilkenson, formerly Scoop Publishware's owner, once employed a team of experienced and competent Swedish developers. Things began falling apart when he fired them and contracted with an overseas development company to save money. They were responsible for the failed Scoop 6 effort. This isn't the first time I've seen offshoring development kill a product or even a company. There's a phrase that begins "Up the creek..," that comes to mind. The situation with Scoop 5 is intolerable. Reject the old ways When we think of Scoop 5, we see something that clashes with our culture and business model. If you really checked it out, you would agree with us. It allowed five storage management solutions: the file system (i.e., no database), MS SQL, a licensed, relational database management system (RDMS) from Microsoft and only for Microsoft OS’s, Oracle SQL, also proprietary and, alternatively, both My SQL and PostgreSQL, popular, platformi ndependent, free and open source RDMS’s. Why you would want to offer five mechanisms when one will do is beyond me. Especially when two of them require costly licenses and restrictions on resellers (like us). The information retrieval storage system from Surfinity, which was used by eLibrary, was the source of its own special problems. Then there was the offering of both InDesign and QuarkXPress support. Well, you know what happened to QuarkXPress. I don't know a single reseller or XTension developer that liked dealing with them. We certainly didn't. You can't imagine how both Mac and Windows client workstations were supported. There were two separate programs and two separate development teams. The Mac guys were quite good. The Windows team, not so much. Not only that, each required special coding to use the proprietary GUI tool kits each had. I could go on for hours. Scoop 5 can't be where we start from if we want to go anywhere. The proposed future: We took a bold step. We forswore any attempt to incrementally progress step by step from the existing (almost) working Scoop 5 system and its code base. To do so would just have placed us in the position of being upwardly compatible with previous errors. And forever fixing ancient bugs. The first step was to say goodby to QuarkXPress, then the file system and MS SQL or Oracle for data management. Junking Surfinity was a nobrainer and so were platform dependent GUIs and proprietary operating systems. What a relief! Now we could concentrate on doing the good stuff. The choice of platform was easy, just use Linux on NUCs, like what our current low cost, high performance advertising systems run on every day. Or support cloudbased deployment as an alternative. Scoop 7 uses TLS (Transport Layer Security) to encrypt all communications between the server and the client. TLS is the same technology used in web browsers to provide the "S" in HTTPS. Effectively this means that, if properly configured, any and all data exchanged through Scoop cannot be read by any eavesdroppers. Kicking out Surfinity was easy. We substituted ElasticSearch, currently a very popular FOSS search engine. Unplug Surfinity, plug in ElasticSearch and voila, searches are better, more intelligent and faster. And it's free. No wonder Surfinity is gone. BTW We have built ElasticSearch databases with over a terabyte (over 10 year's worth) of data from Scoop newsroom systems. One of the ugliest parts of Scoop was its rights management technology. Not only could we not find all of it, what we did find sucked big time. We junked it all. If you want to keep someone from using software, it should not be done module by module, but at a higher level. We've never done this before and don't want to start now. What if it breaks? What kind of vendor would be willing to jeopardize a customer's publishing for a payment? To unify the GUI across workstations, be they Macs, Windows PCs or Linux, we chose Qt. Qt is a GUI toolkit that allows a single code base to be used for multiple platforms. Qt is dual licensed under both open source and proprietary/commercial licenses. We are using it under the free, open source (FOSS) license. With our unification of all database access through an SCS developed front end to PostgreSQL, we could hide the differences in file and operating systems. Qt helps with this and many other multiplatform issues. Just how special is Qt? Well, it has builtin components for internationalization that allow a product line to be deployed using multiple languages and localizations. Everything from alternate strings (words and phrases) to how dates, times, numbers and money are displayed can be done through Qt. What a super tool! Here's the surprise! We have a partnership to announce. There was new coding done after Ulf sent us the source we bought. It was done by his former 25% partner Juha Siintola (Jussi) in Finland. Jussi bought out Ulf, acquiring all of Scoop Publishware. On Thursday August 3, 2017 we inked a deal with Jussi that changes everything. Our agreement gives us joint copyright ownership of the Scoop 5.5, 5.7 and 6 code (and now Scoop 7) and the rest of the intellectual property of Scoop Publishware. Jussi will supply development resources focused on InDesign plugins. We will focus on the rest. We will not compete with each other. Our territory is the Western Hemisphere and his is the rest of the world. We will be setting up Scoop to run for those who speak Spanish, French and Portuguese. SCS already has customers in 21 countries, including ones in North, Central and South America and the Caribbean, so the internationalization is appropriate. (William knows all these languages, plus about a halfdozen more.) With the partnership with Jussi, we start with installed bases of Scoop in over six countries. Clearly the system is well liked. It's just suffering from bit rot. Going forward, Scoop 7 is in alpha release. Interestingly, it currently has about 75% of the functionality of Scoop 5 and yet has a code base of about 50,000 lines of code. Scoop 5 has over 450,000 LOC. What the heck! Call Kurt Jackson (610 7467700) to talk further. He can arrange demos and provide additional information. No newspaper executive needs to ask us to buy into the reality of today's newspaper market. We get it and we are committed to help.
We hope our customers and prospects in the newspaper industry want to do more with less, while still well supporting the mission of the free press. We appreciate the difficulty of adapting to the new Internet realities that challenge their traditional business model. How are we trying to help? A large part of our research and development effort is dedicated to making newspapers more efficient. It's tricky. For example, we grew a multi-million dollar business around automating classified pagination only to see newspaper classifieds decimated by Internet classifieds. Funny, not all newspaper classifieds are dying. Some of our South American customers still run classified sections with over 100 pages. What's their secret? So SCS/ClassPag was a highly successful technology that almost invariably allowed a single operator to do the work of many. What's happened with it now? At corporate design centers, every operator can paginate multiple newspaper classified sections. They call it consolidation and supporting it well is an increasingly important part of our business. SCS/ClassPag is the kind of practical artificial intelligence technology that makes efficient what was once tedious. What about the people? Think of it this way. If you ask a person to do a job a machine can do better, you de-humanize the person. By-the-way, I just hate when someone refers to reducing the labor needed to make a product as "culling." It's essential productivity growth. There is no denying that print advertising is declining, just as there is no denying that it remains the top revenue source for nearly all newspapers. SCS's Layout-8000 ad dummying technology, PaperCheckAdBoss (PCAB), SCS/Track and other reliable tools help newspapers produce their print products more efficiently. We would be gratified if these contributed to preserving the quality of their products and retaining the trust of their readers and advertisers. SCS builds trusted newspaper systems. Did you read the story called Tronc's Star Executives Launch Newsroom Tour?
One line in there caugth my eye. "The execs told Sun Staff that the machine learning and artificial intelligence the company is emphasizing will serve as a way to free up time so staffers can create more content without getting bogged down in other, more tedious processes." Back in the mid 70's when I was new at the ANPA/RI (the research arm of what is now the NAA), we identified one of those "more tedious processes", display ad dummying. It was obvious that before newspaper pages could be paginated (a critical goal then), the page geometries would need to be computed and passed to a pagination engine. Remember, this was a time when there was no QuarkXPress, InDesign or even desktop computers. After prototyping a laser page imager for the ANPA/RI's patented page compositor invention, I thought ad dummying was ripe for automation. Some work on automated dummying had been done by my predecessor, Dave Reed. The ANPA had also funded similar work that was done at MIT. Why not pick up from there and make a practical solution? So here we are in 2016 and the struggle for automated dummying of newspapers still isn't resolved. Why? Michael Ferro talks of artificial intelligence and machine learning. Been there. Done that. (Well, maybe not for the audience analysis as Mr. Ferro envisions.) Still, with thousands of publications being dummied each week with Layout-8000™ (including all of Mr. Ferro's papers), why do only about one in five Layout-8000 users let the computer do most of the work? Based on my 40 years of experience with this issue may I humbly suggest that perhaps technology adoption isn't as easy as Mr. Ferro might think. It's a topic of endless fascination to me. I learn something new every day. At many sites dummying an edition with Layout-8000 takes about 10 minutes. Much of that time is spent just double checking and tweaking what Layout-8000 proposes. At other sites operators spend as much as four times that long. What's the difference? Does Layout-8000 need to be even smarter? Isn't a huge state space search coupled with a finite state machine with 133 transitions enough? Is it about better GOFAI (Good Old Fashioned Artificial Intelligence)? It's something I studied. My early 1970's research papers on chess programming appeared in SIGArt. In 1980 we at the ANPA/RI installed the first production version of Layout-80 at the Pittsburgh Post-Gazette. It could and did auto-dummy. So, Mr. Ferro, is it just a little AI and ML that will reduce the tedium of ad dummying at the tronc papers? There is good reason to doubt it. Before I tell you what I've concluded is the overarching impediment to progress, let' consider what typically happens. Layout operators know stuff that isn't told to Layout-8000 (or, for that matter, recorded anywhere.) Consider the interface problem: 1) advertiser requests aren't entered into front ends, or, if they are, they aren't passed to Layout-8000; (Would you believe that some front-end vendors claiming a Layout-8000 interface actually sabotage their interface to Layout so as to favor their own dummying products?) 2) Many former MEI ALS sites have a very hard-to-break culture of manually placing every ad; 3) quick and dirty manual placement takes precedence over doing a one-time, more complete product description and setup, and so it goes. As we help corporate newspaper groups like Gannett, tronc (Tribune), Lee Enterprises and others consolidate dummying into design centers, we have found the search for more automation continues. Recently we added what we call LayoutHistoryAdBoss™. This is a module that records the decisions of Layout designers in context. It learns their dummying expertise. (Tasks, not people, are moved to design centers. What those people know needs to be moved to the design centers.) So we do artificial intelligence and machine learning. Behind the scenes we gather statistics on every Layout-8000 dummying session. We know how long operators take, the description of the product, the completeness of the ad interface data, the product setup, etc. And still this isn't enough. What's really missing? Mr. Ferro, I've come to believe progress depends most on agility. Being able to quickly make and deploy small improvements to systems is absolutely critical. So this is an argument for less of the bureaucracy we see at corporate sites. Through speeding change one maximizes improvements. Consider a corporate testing cycle that lasts longer than the interval between releases. Of what use is that? Does it reduce problems with upgrades? Unfortunately the contrary is true. I'm reminded of a story my daughter, Sharon, told me of her programming job at Dell. She was responsible for a supply chain application. A new release was deployed and folks in Ireland and Italy started having trouble accessing the system. Turns out that O'Leary, O'Mally, D'Orazio, had problems as did others with names with single quotes. They were messed up by an error in coding an MS SQL query statement. The fix took five minutes to code. The approvals to deploy the fix took an entire day of hand carrying paperwork for signatures by managers who were unlikely to understand any technical explanation of the problem or its fix. We have many similar stories we could tell about the organizations we serve. The lesson - more bureaucracy degrades rather than improves the yield from new computing technology. It is wrong thinking to believe that a longer deployment time frame avoids problems. It just increases the difficulty of finding and repairing them. My recommendations: 1) be more agile, and 2) recognize that the issue isn't one of technology but of management. For the tedium to go away, what people do with technology must change. |
Richard J. Cichelli
SCS President & CEO Archives
January 2019
Categories
|