Why is high availability something users of any mission-critical software need to think about? The simple answer is that if the application is unavailable, its users would be in a world of hurt. Software Consulting Services, LLC (SCS) takes high availability very seriously. Pay special attention if you are using VMWare.
One typical approach for data processing applications is to provide recovery from failures by taking snapshots. Snapshots come in two varieties. Full snapshots are those where an entire copy is made of the current state, i.e. all data managed by the application. Delta snapshots copy over only the differences between one context and the prior one.
Using the full snapshot approach, recovery takes you back to the state when the last snapshot was done. Snapshots taken the night before lose the session work back to the nightly backup should there be some sort of failure. For applications that fail almost never, this method usually provides adequate reliability.
Delta snapshots are different. Like a full snapshot, you are able to get back to the state of the latest snapshot. Unfortunately, the path there involves already having a full backup copy and then applying snapshot deltas one at a time until you have the state of the computation as up to date as it can be.
Delta snapshots are tricky. And as the VMWare documentation (1025279) for "Best practices" states, "Snapshots are not backups. A snapshot file is only a change log of the original virtual disk. Therefore, do not rely on it as a direct backup process."
For snapshots to work, databases need to be quiescent when a snapshot is made. Usually, this implies that the service being provided by an application is interrupted. Choices need to be made that trade mean time to recover (MTTR) against the cost of service interruption. Thus you hear the term "nightly backup" rather than "15-minutes-ago backup."
To meet the needs for even higher availability with larger active computational states, our applications keep two (or more) application environments in sync in near real time. We call this “mirroring”. You can switch from one to the other and be assured you will lose little if anything when an application server fails. This is done by log shipping.
Log shipping is a reliable mechanism for building recoverable databases. Here is how it works. Log shipping is built into SCS's data management subsystem. Two separate applications environments are kept in sync. When a record is updated in the primary environment, a transaction is made and passed to the secondary environment. The secondary is always running so it is active and sort of "on standby". It can take over the work by a simple "make primary" command. We contrast active standby availability to a "powered off" backup machine. These are in no way the same in terms of disaster recovery.
It is important that the primary and secondary environments have nothing in common but their network connection. Anything that fails with one should never impact the other. This is called "shared nothing servers." We don't even put these servers on the same UPS! Nothing is shared, not even the underlying data management algorithms. The transmitted log-based transactions have their own updating, consistency and validation logic. This is designed to be both more bullet proof and idiot proof.
Just as a side note – With stable software systems, the actual need for switching to a secondary server may be so rare that we recommend that switching take place on a scheduled basis. Without practicing, it is probable that during an emergency more problems will be created. (You should read the VMWare recovery procedure if you want a really good scare.) I really wish more attention were paid to recovery and high availability issues. Just last month we were helping one of our customers and we found their secondary server was last updated in 2008. An untested and unpracticed disaster recovery plan is no recovery plan at all.
The primary and secondary servers don't need to be in the same location, although since server class technology is so inexpensive these days, both are likely to be deployed in the same server room. (They might be separate VMs on separate servers.) Should enhanced disaster recovery be desired, additional secondary servers can be deployed in remote secure locations.
All the articles and their links starting from this URL are relevant if you wish to dig deeper.
Originally published 08/03/2015
We at SCS are sending our last Sun server to the recycler.
The Bee Group Newspapers of Amherst, NY, a Layout-8000™ user for many years, will be getting its next Layout-8000 upgrade on modern Intel NUCs running Linux. Trey Measer, Bee Group publisher, was somewhat surprised to learn that SCS kept a Sun server just to support his newspaper.
Our Sun needed to be sent to a better place. When running, it put off nearly 3000 BTU of heat per hour. An extra ton of expensive A/C cooling equipment was needed just for it. Our secondary backup A/C system was failing and taking care of our Sun could cost $12,000 in new cooling equipment.
The Sun consumed much electrical power and delivered not much computing power, at least not in current terms. Our Sun 3500 Enterprise Server had two 400 MHz processors and about 100 GB of disk space. Prices for bare-bones Sun servers started at over $4,000. The three year total cost of ownership of such a fully configured Sun server was estimated to be $88,000.
Here's a server network, larger than the one going to The Bee Group, made up of 6 NUC servers (on the right side of the monitor) and 2 large storage devices (on the left).
The Bee Group will be getting 2 NUCs with 1.7GHz processors, 8GB RAM and 240 GB SSD. They will also be getting one 2TB Western Digital MyCloud NAS. This is all for less than $2000.
And now - what was that device sitting on top of the Sun?
The Banana Pi, the newest computer on which Layout-8000 runs, consumes less power than a 100 W incandescent light bulb (about 350 BTU) and, of course, requires no special environmental equipment. (People consume about 400 BTU/hr each.) Nevertheless the computing power that the Banana Pi delivers is extraordinary. As pictured, the BPi has a 1 GHz dual core processor, 1 GB of RAM, 1 TB of fast SSD, 1 GB Ethernet connection and support for various peripheral devices. In raw computing power, the BPI has about 10 times that of the Sun server and, fully tricked out, costs under $500. The bare bones Banana Pi single board computer can be ordered from Amazon for under $35.
Support revenue has always been very important to SCS. We want to keep customers much longer than their hardware is likely to last. It is not just Suns that have become obsolete. Many versions of Windows and the computers they ran on have reached end of life. Long before Windows appeared on the scene, SCS software ran on DEC and HP computers. And, of course, it runs on the Sun.
Each new generation of technology brings new challenges. We expect specific computers and operating systems to become obsolete. SCS guards against being negatively impacted by this by investing in keeping our software platform independent and using either commodity or open source operating systems with no third party layered products. This distinguishing difference makes adopting next generation platforms easier while reducing the total cost of ownership of SCS's systems.
As Bee publisher Trey Measer learned first-hand, SCS supplies enduring customer support.
Originally published on 07/13/2015
A sense of immediacy, universal buy-in and personal involvement: these are some of the shared attributes The Lawton (OK) Constitution management team brought to the table as the daily newspaper embarked with Software Consulting Services (SCS) on an ambitious, six-week timetable to overhaul an antiquated lineup of advertising and accounting systems.
The project managers for SCS credited the expedited rollout and successful go-live to the mentality of the paper’s management team. “There’s only so much we can do with technology,” said Keith Keffer, SCS business systems manager. “We have to have the site want to go forward and push it forward.”
The paper went live in June with what SCS describes as its “Community Advertising System Enterprise Edition Subscription”. The solution is marketed as a monthly service in which SCS provides hardware, software and managed support for an array of advertising-based applications that had traditionally been sold by site license.
Already a customer of SCS’s automated advertising dummying system, Layout-8000™
, The Lawton Constitution added:
The first phase focused on setup, training and implementing the AdMAX, SCS/ClassPag and Layout-8000 products. SCS/Track and CAS were slated to follow later this summer.
“From the first week of training [in late April] to go-live was about six weeks,” Keffer said, “[That] wouldn’t have been possible if the site didn’t have such a good attitude. Everyone involved in the decision-making for purchasing the system was 100% behind it and they made that clear to everyone who worked for them.”
“Not only did they buy into it, but they were very involved,” said Jonathan Ebling, SCS’s pre-press project manager, who participated in the installation and go-live.
For The Constitution, there was an immediacy to implementing a new classified system, in particular. “[The preexisting system] was on life support, and we had to do something in a hurry,” said Mike Owensby, general manager.
The paper was experiencing issues with server stability. “When we were down there for the site survey [in March], I think they rebooted their classified system six times, because it crashed and they couldn’t take ads,” Keffer said. “That was pretty much a daily occurrence for them. I don’t know about you, but if my server was dying that often during the day, I’d be wanting to get off it, too.”
Owensby said the vendor and newspaper agreed on what both considered an aggressive timetable for transitioning.
He acknowledged some initial hand-wringing by front-line users over giving up a system with which they had become “comfortable”, but said doubts were short-lived. “As with any change, everybody was just a little bit apprehensive, but it helped having [the SCS team] around to hold our hands,” Owensby said. “That first day everybody was pulling their hair out. By the third day, everybody [was saying], ‘This isn’t too bad.’ By the fourth day, everybody was just flying through it.”
Kim Dodds, advertising director, observed that the team’s confidence is “building every day.”
In retrospect, it was easier for the management team to be on board from the start, having fretted over the imminent failure of the legacy system. “I mean, there was a couple times when we weren’t sure we were going to get [the paper] out,” Owensby said. “That’s the reason [we] were comfortable with it going in.”
A demo of the vendor’s classified product helped distinguish SCS from the paper’s other prospective choices. “We looked at them and a couple of others,” Owensby said. “It looked like it was a pretty easy system. We didn’t feel like the learning curve would be quite as much as with some of the others. That played a lot into it. It seemed to be something we could implement quickly – and fortunately, that’s the way it’s been.”
It didn’t hurt that the paper was already familiar with SCS as a Layout-8000 customer, Owensby acknowledged.
Aside from swift implementation, another key advantage gained in the transition to SCS’s Community Advertising System, according to Jim Cottingham, The Constitution’s business manager, has been the “elimination of duplication of effort.”
Phil Curtolo, director of sales for SCS, helped the paper justify its investment in the system by demonstrating how it would refine workflow to eliminate a three-part form that was being used to enter redundant data into separate systems.
It’s hard to overstate, Curtolo said, “just how manual the workflow was” and “how many hoops they had to jump through” to schedule, dummy, build and invoice a display ad prior to the conversion. “Their workflow consisted of a hand-filled-out sheet that the sales reps would fill in. They’d pass that around to the different parties – order entry in accounting and production – and then the production folks would grab that sheet and start using that to build the ads. We’ve gotten rid of that sheet for two of those steps. There’s one more step to [eliminate] and then the sheet will go away,” replaced by a single data entry step.
“It streamlined all our processes,” Cottingham said.
Subscription Model and Managed Services
SCS’s shift in focus from selling software site licenses to providing an array of services – software, hardware, support and managed services – via a subscription model was also seen by the Lawton management team as a major plus. “That was a huge part of our decision,” Cottingham said.
“All the servers are theirs; all the software is theirs, and we just pay the monthly service. We also get all the updates,” Owensby said. “They have remote capability if there’s something that’s going haywire. They’re able to go in there and take care of it remotely, which is handy. And also, 24/7 tech support doesn’t hurt.”
“It’s the latest version of the full-function software that we’ve installed for years and years for customers who purchased the site licenses outright,” Curtolo explained. “The difference is that now we’re taking this approach of packaging it together, doing the subscription service and providing the hardware.”
“One of the reasons we’ve had success” with this approach, according to Keffer, is that “we take ownership of a lot of the day-to-day” tasks that an on-site applications specialist would otherwise perform. “We’ve had a few sites where because of staffing issues – where they’ve lost people or had to downsize – they’ve been able to take advantage of us.’’ It costs a lot less for SCS’s managed services, he said, than “having a body on site,” and it’s an added benefit to newspaper customers that “they don’t have to worry about managing all that stuff.”
Likewise, not owning the servers relieves the customer of the prospect of hardware headaches. “We provide on-premise equipment, and it’s as though it’s in the cloud from the customers’ standpoint,” Curtolo said. “They don’t do anything with the on-premise equipment other than unpackage it when it arrives, plug it into their power supply and hook it up to their network. Once it’s on the network, we maintain and monitor it in real time from SCS, going forward. It’s got all of the thumbs-up and benefits of the cloud, without any of the risks having hardware not being on-premise and not being able to access it for some reason.”
The bundling of what used to be stand-alone software products also has had a beneficial impact on the implementation process, according to Curtolo. “We eliminate what was always the wild card when we were installing one system at a time: the interface with third-party systems,” he said. “The more of that we can cut out, the better the installs go and the more control over the installs we end up having.”
The vendor has an installed customer base of 20 newspapers that are utilizing its full suite of advertising products, Curtolo said. This is in addition to the more than 200 that have one or just a few of the modules.
Having become one of them, the management team at The Constitution has expectations that in addition to productivity savings, the newspaper will realize revenue gains from the new upsell features in SCS’s classified product. “We’ve already generated some,” Cottingham said. “I was looking at some reports this morning, and it looks like the boxes and logos are up from what they were.”
Owensby acknowledged that the gains will take time. “It’s a slow process, because everybody’s got to get comfortable with [the new system], but at some point we’re going to get there, and we feel like there’ll be enough additional revenue generated to take care of the monthly cost.”
Originally published 07/13/2015
As the primary news source for the state of New Hampshire and a newspaper with a national audience, the Union Leader is a complex operation. With statewide penetration, a multi-faceted web presence and four regional papers, the Manchester-based paper needed an advertising accounting and designware package as solid as the bedrock from which the Granite State takes its nickname.
That's when SCS entered the picture.
SCS has been working with the Union Leader for more than a decade and the biggest paper in New Hampshire makes good use of our entire package of software for its display and classified advertising departments. From AdMAX™ to Community Advertising Services, the Union Leader has what it takes to meet the integrated, cost-efficient demands of today's newspaper industry.
"What's terrific is that it's all integrated. It's extremely flexible and we do everything we can with the system," said Joyce Levesque, chief financial officer at the Union Leader.
SCS Business Systems Manager, Keith Keffer, who has been working with Levesque and her team for the past year, says the Union Leader is a great example of what can be done with SCS's wide array of offerings.
"They use everything in the system the way I'd hope people would use it," Keffer said. "This site is one that really wanted to take advantage of the best practices available through our applications."
Among the packages already in use are:
They have just gone live with the self-service classified module, Keffer says, noting that it allows sales representatives to input ad orders as well as allowing members of the public to order and create their own ads.
Levesque, who has been involved in almost all of the implementation and oversees the operations from the accounting side, said the programs have been integral to the paper's efforts to create efficiencies and expand the services to its own customers.
Adding the SCS/Track and AdMAX applications allowed the Union Leader to eliminate the multiple platforms previously required to coordinate the ad orders, placement and billing.
One of the biggest advantages of the AdMAX application, said Levesque, is that it keeps track of current customer billing and lets newspaper representatives know when a delinquent customer tries to post a new ad.
"The advertising system travels all the way through to print and web, customer credit and ad sizing," she said. "This has helped us manage revenue, too. We are very, very pleased."
Originally published 01/07/2013
Color Factory with SmartColor supports automated work flow in the toning area for multiple publications. The biggest benefit coming down the pike is for consolidation efforts.
With eight daily newspapers in northeast Pennsylvania and almost a half-dozen alternative publications spread across the eastern United States and Saint Thomas, Times-Shamrock Communications needed a robust toning center to handle its growing workload.
They found it in Color Factory, provided by Software Consulting Services, LLC (SCS).
Created by FotoWare of Norway and sold in North America by SCS, Color Factory's toning center is essentially a one-step process that offers automatic reproduction and image enhancement, format conversion, sorting and duplication, PDF processing, automatic metadata processing, scalability and advanced file routing to print clients throughout the world.
The strength and flexibility of the package, which includes SmartColor for automatic color enhancement and SmartClean for noise reduction, color saturation and contrast, has enabled papers across the United States and Canada to cut costs and improve quality of their photo reproduction in both print and on-line products.
"We did not reduce manpower but we did increase our workload. Additionally, Color Factory is also embedded into our Ad Tracking system, as well," said Michelle Ross, Publishing Services Manager for Times-Shamrock, explaining that Color Factory is used to standardize photos and convert them to PDF for publication on the company's various publication websites. "Everything is workflow-based without a lot of manual intervention. Everything just flows through the system."
As the publisher of the Times-Tribune, Citizens Voice and Standard-Speaker in northeastern Pennsylvania, the Virgin Island Daily News and alternative publications in locations as disparate as San Antonio, Detroit, Baltimore and St. Thomas, the company needed a strong system when it looked to replace an aging software package that was no longer able to keep up with the company's growth, Ross said..
Despite their broad geographic spread, they were able to put in workflow management technology that offered significant efficiencies. "We are very, very integrated here," Ross said, adding that Color Factory is used even for promotional and related materials for the company's radio stations.
Kurt Jackson, Vice President of Operations for SCS, said the company has been distributing Color Factory for about 10 years. The timely introduction of the SmartColor package five years ago proved a boon for many newspapers that at the same time were starting to wrestle with significant revenue losses as more advertising went electronic. SmartColor expanded the capability of Color Factory's toning center applications beyond a single publication and gave them the opportunity to transition to automated work flow in the toning area for multiple publications, Jackson said.
The biggest advantage to the program is the one-step toning capability, which has allowed many clients to reduce staff while improving quality. "The work we are saving there is what would have been done by the photographers or toning staff. The biggest benefit I see coming down the pike is for consolidation efforts," Jackson says. "The system provides incredible value when you consider the FTE savings and quality automation and toning"
Color Factory is used by clients such as Getty Images in Seattle, Wash., where all incoming photos are routed through the system; and the Toronto Globe & Mail, where 97 percent of all photos now go through SmartColor for toning that previously required a staff of seven, Jackson said.
Another newspaper that has taken advantage of the consolidation benefits is Tulsa World in Oklahoma.
"It's been fantastic," said Tulsa Pre-Press Manager David Bridges. "It eliminated the need for eight employees, but we were probably over-staffed when we got it.
"The routing capability inside Color Factory is fantastic. If you can think it, you can do it," he continued, noting the streamlining and versatility of the software as the chief benefits. "The routing gives us all sorts of different options and the speed is unbelievable. We have spoiled our newsroom with this."
Originally published 2012-10-31
Updated 6/21/2016 - Gannett to add 15 new sites to its Layout-8000 environment
In June 2016, Gannett has licensed 15 new sites to its corporate Layout-8000 system. This supports Gannett's acquisition of Journal Media Group, a deal that was finalized in April 2016. Journal Media Group was formed in 2015 through a merger of the newspaper operations of The E.W. Scripps Company and Journal Communications, Inc.
After this addition, Layout-8000 will be used by Gannett to dummy more than 100 daily newspapers in 34 states and Guam. New publications include The Milwaukee Journal-Sentinel, Wisconsin's largest newspaper, two of Tennessee's largest dailies in The Knoxville News Sentinel and The Memphis Commercial Appeal and The Ventura County Star in Camarillo, CA.
Updated 04/14/2016 - This installation is completed and in full production. More Gannett sites have been absorbed into the design centers as papers were purchased. Layout-8000 is the corporate standard.
Originally published: 06/27/2012 - Gannett Co. Inc. is installing Layout-8000 to anchor display ad dummying at its 83 dailies and 450+ weekly publications.
A single server farm, or Gannett "Cloud" in the eastern US will host users nationwide.
The Layout-8000 deployment is integrated with the roll out of CCI Europe’s Enterprise NewsGate CMS, which Gannett is implementing to consolidate content for system wide sharing. Gannett will also create 5 design centers groupwide. SCS and CCI have developed an enhanced XML feed that will allow all daily and weekly publications to use a single XML data scheme.
Design centers slated to go into production with Layout-8000 are those in Asbury Park, NJ, Des Moines, IA, Phoenix, AZ and Nashville, TN. The hubs will also be responsible for paginating content for sister U.S. Community Newspaper sites within their regions.
What is Faceted Ad Search?
Newspaper ads contain two kinds of attributes: attributes of an ad order and attributes of the ad content. Attributes of an ad order include the ad number, classification, customer id, name, and address, run schedule, cost of the ad, and so on. The relational database system allows ads to be indexed for quick search on these attributes. Such searches are useful to newspaper personnel for providing management information through queries and reports.
Attributes of the ad content depend on the category of the ad. For example, auto ads have a make, model, year, mileage, price, and so on. Real estate ads have a price or rent, number of bedrooms, location, and so on. The Spice system allows each newspaper to specify rules for recognizing such attributes in textual ad content. Once attributes are recognized (by a program, called oddly enough a recognizer), ads are added as documents in a text indexing system that indexes ads for quick search on those attributes.
These attributes can be added manually by either newspaper staff or by advertisers in a self-service mode, but one of the real strengths of the SCS search offering is that our recognizer will add them automatically using a technique called “named entity recognition” or NER.
We consider the automatic assignment of ad content attributes a significant innovation. Not having to enter fielded data to know that, for example, an ad describes a car with two doors and costs between $7,000 and $8,000 makes supporting faceted search practical for newspapers, where the labor costs of manually entering such data are prohibitive.
Because we use this technique, we need only a classified extract (as one might send to a pagination system) to have the raw data needed to build a newspaper owned and controlled faceted search web site for classified ads.
Searching ads by content is especially useful for users of an on-line advertising system. It allows them to specify criteria that narrow down the list of available autos, homes, apartments, or jobs to those that are of interest to the reader. The search technique that SCS uses for content searches is known as faceted search.
Let’s use as an example a search for a used Chevrolet (any model) for sale in the Lehigh Valley, with air conditioning, for sale for less than $10,000.
The first step in a faceted search of classified ad content is to get to the right classification. In general, getting to the right classification is called hierarchical or taxonomy searching. Yahoo Directory uses this method. With it you can iteratively narrow your search to get to the desired information.
For example, we can start at the top of the directory and then select Business & Economy, Shopping and Services, ...
until, eventually, we reach car listings.
The second step is to indicate search criteria that make sense for ads in that classification. This step is similar to direct keyword text search, as Google supports, with several advantages. With Google, you just throw a bunch of words into a text box.
Chevrolet cars in Lehigh Valley with air conditioning less than $10,000
The user interface provides no guidance on what to type to specify ad search criteria. In fact, there is really nothing you can type into a text box that will, for example, find all ads for autos in a certain price range. And you can’t be sure all of your criteria are used.
Following the first non-sponsored link returned by that last search, for instance, returned 5,051 cars and the first two were in Tennesee and Florida. The listed prices were under $10,000 but one of them represented the current bid in an incomplete eBay auction.
With an attribute-based search, the recognizer adds to the text index an attribute that indicates the price range. And since the classification has already been determined by the first step in the search, the user interface for the second step can guide the user in specifying search criteria (e.g., price, make, mileage) that make sense for that classification.
Thus faceted search combines the hierarchical and direct search techniques in such a way as to provide a superior classified search experience. Faceted search enables users to navigate a large, online database of classified ads by combining a progressive narrowing of choices (i.e., by classification) with text search. Faceted search has become the prevailing user interaction mechanism in e-commerce sites.
SCS provides faceted search using new technology that allows full text searching along with attribute based searching. With it you can, for example, find ads for the apartments for rent in a given neighborhood, in a given price range with the right number of bedrooms and baths. Combined with the Ajax-enabled interactivity that SCS uses in its on-line applications, faceted search provides a unique and wonderful user experience.
We can also use the SCS system to look for our used Chevrolet with air conditioning, that costs less than $10,000. We start by selecting the category Transportation and then Cars.
At this point, sub-topics that are appropriate only for cars are presented. We can select Auto Makes and then Chevrolet.
Notice the “breadcrumb” menu that shows how we got here and also how many ads currently meet our search criteria.
We can now check off our price criteria and our request for air conditioning.
Finally we can see that a small number of ads meet our criteria and we can ask to see them. If we hover over a specific ad, we see it as it appeared in the newspaper.
Never has there been a way to browse content so rapidly while getting acquainted with the scope and nature of the content. You will never feel lost no matter how much data there is. More than a search interface, faceted search is an information navigation and discovery tool.
The Spice formula language’s selection command allows developers to build a query that specifies a selection of ads using ad content criteria specified in a faceted search with ad order criteria from the relational database, thus seamlessly integrating information retrieval and relational database functionality. The rows (ads) resulting from such a query can include a join of ad order attribute columns from the relational database table and ad content attributes derived by the content recognizer.
Perhaps the earliest reference for a system supporting faceted search of classified advertising and similarly organized information is Patent number: 4429385 “Method and apparatus for digital serial scanning with hierarchical and relational access.” Filed Dec. 31,1981; issued Jan. 31, 1984 by inventors Richard J. Cichelli and Michael O. Thompson then with the American Newspaper Publishers Association.
Originally published 2009-07-01.
Learn about SCS
Published articles that are written by, about, or for SCS are collected here for your viewing.